Domain adaptation stereo matching method based on multi-distribution color conversion and consistency

Abstract

This invention relates to a domain-adaptive stereo matching method based on multi-distribution color transformation and consistency, belonging to the field of computer vision technology. It includes the following steps: transferring the color style of a target domain image to a source domain image, and training a pre-defined first model using the acquired source domain dataset. Constructing and initializing a teacher model and a student model with the exact same structure as the first model. Inputting an unlabeled image of the target domain into the teacher model to generate a pseudo-disparity map, constructing mixed training data based on the epipolar geometry of stereo matching, and inputting the pseudo-disparity map and the mixed training data into the student model. Continuously optimizing the parameters of the student model using consistency learning, and deploying the optimized student model as a domain-adaptive stereo matching model. This invention effectively solves the domain difference problem in the prior art and improves the domain-adaptive stereo matching capability through the above methods.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a domain-adaptive stereo matching method based on multi-distribution color transformation and consistency. Background Technology

[0002] Stereo matching is a fundamental problem in binocular computer vision, aiming to find corresponding pixels in a stereo pair (left and right images), where the distance between corresponding pixels is called disparity. Stereo matching utilizes epipolar geometry to obtain stable depth perception from the estimated disparity, and therefore has been applied to various scenarios such as scene understanding, object detection, visual odometry, and SLAM. Classical stereo matching methods typically employ manually designed features and predefined windows to calculate and aggregate matching costs. Due to the limited representational power of features, most classical methods perform poorly in textureless regions, scenes with perspective distortion, and scenes with varying lighting conditions.

[0003] With the significant breakthroughs achieved by deep neural networks in image classification tasks, researchers pioneered the use of deep networks to extract image features and calculate matching costs, demonstrating performance that significantly surpasses traditional methods. To further improve the accuracy and efficiency of stereo matching, current techniques generally employ fully convolutional networks to directly regress disparity maps. These methods utilize massive amounts of synthetic data for pre-training and real-world benchmark data for fine-tuning, achieving state-of-the-art performance in publicly available stereo matching benchmarks.

[0004] However, these methods for adapting synthetic data to real-world scenarios have limited performance. For example, a stereo matching method pre-trained on the labeled SceneFlow dataset achieves good performance on that dataset, but fails to produce equally good results on other real-world datasets such as Middlebury and KITTI, meaning that the datasets used for model training and the datasets used for model testing are not in the same domain.

[0005] Therefore, rather than designing high-precision models for specific datasets, the more pressing need is to obtain effective domain-adaptive stereo matching models that can accelerate the promotion and practical application of stereo matching technology.

[0006] The information disclosed in this background section is intended only to enhance the understanding of the overall background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to overcome the above-mentioned defects and provide a domain adaptive stereo matching method based on multi-distribution color transformation and consistency.

[0008] To solve the above technical problems, the present invention provides a domain-adaptive stereo matching method based on multi-distribution color transfer and consistency, comprising the following steps: S100, based on a preset multi-distribution color transfer rule, the color style of the target domain image is transferred to the source domain image, thereby obtaining the source domain dataset, and the preset first model is trained using the obtained source domain dataset; S200, a teacher model and a student model with the same structure as the first model are constructed, and the parameters of the teacher model and the student model are initialized to the parameters after the first model is trained; S300, the unlabeled image of the target domain is input into the teacher model to generate a pseudo-disparity map, and a hybrid training data is constructed based on the epipolar geometric characteristics of stereo matching. The pseudo-disparity map and the hybrid training data are input into the student model, and the parameters of the student model are continuously optimized using consistency learning; S400, the student model with optimized parameters is deployed as a domain-adaptive stereo matching model, thereby improving the domain adaptation performance in the stereo matching network.

[0009] Optionally, in step S100, where the color style of the target domain image is transferred to the source domain image based on a preset multi-distribution color transfer rule to obtain a source domain dataset, and the preset first model is trained using the obtained source domain dataset, the domain adaptive stereo matching method may further include: S110, dividing the target domain image into multiple target domain sub-images along the vertical direction, and calculating the color parameter distribution of each target domain sub-image; S120, updating the color parameter distribution of the target domain sub-images using an exponential moving average method to obtain a global multi-color distribution set for the target domain; S130, dividing the source domain image into multiple source domain sub-images along the vertical direction, and calculating the color parameter distribution of each source domain sub-image; S140, randomly selecting color distribution parameters from the global multi-color distribution set of the target domain and mapping them to the corresponding source domain sub-images to complete the style adaptation of the source domain image; S150, inputting the style-adapted source domain sub-images and the corresponding real disparity maps into the first model, and training the first model using a smooth L1 paradigm.

[0010] Optionally, the color parameter distribution of the target domain sub-image includes at least the mean of the target domain sub-image. and variance The color parameter distribution of the source domain sub-image includes at least the mean of the source domain sub-image. and variance .

[0011] Optionally, the calculation process of the mean and variance of the target domain sub-image is expressed by the following formula:

[0012]

[0013]

[0014] in, This represents the mean of the target domain sub-image. This represents the variance of the target domain sub-image. Indicates the first The k-th sub-image of the target domain of the target domain image is E(·), which represents the expectation operation.

[0015] Optionally, the process of updating the color parameter distribution of the target domain sub-image using an exponential moving average to obtain the global multi-color distribution set of the target domain is expressed by the following formula:

[0016]

[0017] Among them, among them, , Let be the mean and variance of the i-th color distribution in the updated global multi-color distribution set of the target domain, respectively. , , respectively, are the mean and variance of the historical target domain sub-images, and γ is the EMA attenuation coefficient, with a value range of 0.9~0.99.

[0018] Optionally, the process of randomly mapping the color distribution parameters in the global multi-color distribution set of the target domain to the corresponding source domain sub-image, thereby completing the style adaptation of the source domain image, is expressed by the following formula:

[0019]

[0020] in, This represents the k-th source domain sub-image of the j-th source domain image after style adaptation. This represents the k-th source domain sub-image of the original j-th source domain image. , , respectively, are the mean and variance of the source domain sub-images. , These are color distribution parameters randomly selected from the global multi-color distribution set of the target domain.

[0021] Optionally, in step S300, where the unlabeled image of the target domain is input into the teacher model to generate a pseudo-disparity map, hybrid training data is constructed based on the epipolar geometric characteristics of stereo matching, and the pseudo-disparity map and the hybrid training data are input into the student model, and the parameters of the student model are continuously optimized using consistency learning, the domain adaptive stereo matching method may further include: S310, selecting a strip-shaped region with a width consistent with the image width, bidirectionally mixing the style-adapted source domain image and the target domain image to generate two sets of hybrid images and corresponding hybrid disparity maps, and performing channel random arrangement enhancement on the hybrid images; S320, inputting the two sets of hybrid images into the student model, and optimizing the parameters of the student model using a joint loss function.

[0022] Optionally, the process of selecting a strip-shaped region with a width consistent with the image width, bidirectionally blending the style-adapted source domain image and the target domain image to generate two sets of blended images and corresponding blended disparity maps, and performing channel random arrangement enhancement on the blended images is expressed by the following formula:

[0023]

[0024]

[0025] Where M is the mask for the striped region. For the target domain image, For the source domain image after style adaptation, The target domain pseudo-disparity map generated for the teacher model. This is the true disparity map of the source domain image. , , and These are two mixed images and their corresponding disparity maps.

[0026] Optionally, the process of inputting two sets of mixed images into the student model and optimizing the student model parameters using a joint loss function is expressed by the following formula:

[0027]

[0028]

[0029]

[0030]

[0031]

[0032] in, To monitor losses, Losses due to pseudo-supervision This is the loss due to the fusion of luminance and parallax smoothness. For parallax smoothing loss, The total loss for the target domain dataset; , These are the weighting coefficients. The source domain disparity map output for the student model. The left image, SSIM(·) is the reconstructed image mapped from the right image to the left image, and SSIM(·) is the structural similarity index. , Let be the partial derivatives in the x and y directions, respectively. For disparity maps, The balancing coefficient is set to 0.8; therefore, the loss on the target domain dataset is... .

[0033] Optionally, the student model uses pseudo-random 3D convolution to replace part of the traditional 3D convolution. The pseudo-random 3D convolution generates the convolution kernel through preset pseudo-random kernel parameters, which reduces the number of model parameters while maintaining stereo matching performance.

[0034] The beneficial effects of this invention are as follows:

[0035] This invention splits the source and target domain images vertically into multiple sub-images, calculates the mean and variance of each sub-image, and then randomly transfers the style of the target domain sub-image to the source domain sub-image. This multi-distribution color transfer allows the source domain image to reflect the style of the target domain image more precisely, while randomly selecting the style of the target domain sub-image for transfer to the source domain sub-image increases the diversity of the training images.

[0036] Meanwhile, based on the characteristics of stereo matching, this invention employs a Teacher-Student network architecture combined with CutMix to learn consistency between the teacher and student models, further improving model performance. Unlike the original CutMix, which uses random boxes to mix the source and target domain images, this invention randomly selects a strip-shaped region (sub-image), where the width of the strip-shaped region is equal to the width of the image, to bidirectionally mix the source and target domain images, forming two sets of mixed inputs. This method, instead of random boxes, is more suitable for stereo matching and can effectively solve the problem of not being able to determine the position of a randomly selected box in the left image in the right image.

[0037] Furthermore, to further reduce the number of parameters in the 3D convolutional stereo matching model, pseudo-random 3D convolution is used to replace part of the 3D convolution, which not only reduces the number of parameters but also maintains performance. Experimental results show that the algorithm proposed in this invention achieves excellent performance. Attached Figure Description

[0038] Figure 1 This is a flowchart of the domain adaptive stereo matching method based on multi-distribution color conversion and consistency provided in the embodiments of the present invention; Figure 2 This is the domain adaptive stereo matching method based on multi-distribution color transformation and consistency provided in the embodiments of the present invention. Figure 1 A diagram of the sub-steps in step S100; Figure 3 This is the domain adaptive stereo matching method based on multi-distribution color transformation and consistency provided in the embodiments of the present invention. Figure 1 A diagram of the sub-steps in step S300; Figure 4 This is a diagram of the overall architecture of the domain adaptive stereo matching method based on multi-distribution color conversion and consistency provided in this embodiment of the invention. Figure 5 This is a schematic diagram of multi-color distribution style transfer of the domain adaptive stereo matching method based on multi-distribution color transformation and consistency provided in the embodiments of the present invention; Figure 6 This is a schematic diagram of image fusion based on the cutMix method of the domain adaptive stereo matching method based on multi-distribution color conversion and consistency provided in the embodiments of the present invention. Detailed Implementation

[0039] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses consistent with some aspects of this disclosure as detailed in the appended claims.

[0040] As mentioned earlier, stereo matching is a fundamental problem in binocular computer vision, aiming to find corresponding pixels in a stereo pair (left and right images), and the distance between corresponding pixels is called disparity. Stereo matching utilizes epipolar geometry to obtain stable depth perception from the estimated disparity, and therefore it has been applied to various scenarios such as scene understanding, object detection, visual odometry, and SLAM.

[0041] Classical stereo matching methods typically employ manually designed features and predefined windows to calculate and aggregate matching costs. Due to the limited representational power of these features, most classic methods perform poorly in textureless regions, scenes with perspective distortion, and scenes with varying lighting conditions.

[0042] With the significant breakthroughs achieved by deep neural networks in image classification tasks, researchers have begun to incorporate deep networks to extract image features and calculate matching costs, resulting in performance that significantly surpasses traditional methods. To further improve the accuracy and efficiency of stereo matching, recent methods have widely adopted fully convolutional networks to directly regress disparity maps. These methods utilize massive amounts of synthetic data for pre-training and real-world benchmark data for fine-tuning, achieving state-of-the-art performance in public stereo matching benchmarks.

[0043] However, these methods for adapting synthetic data to real-world scenarios have limited performance, a problem known as the domain adaptation problem. For example, a stereo matching method pre-trained on the labeled SceneFlow dataset may achieve good performance on that dataset, but fail to produce equally good results on other real-world datasets such as Middlebury and KITTI. This is the domain disparity problem, where the dataset used for model training and the dataset used for model testing are not from the same domain.

[0044] To address this issue, a domain-adaptive stereo matching method based on multi-distribution color transformation and consistency is provided, which effectively solves the above problems. The present invention solves the problem in the following way.

[0045] Example 1:

[0046] Please refer to the instruction manual appendix. Figures 1 to 6 This invention provides a domain-adaptive stereo matching method based on multi-distribution color transformation and consistency. The execution mechanism of this domain-adaptive stereo matching method can be a controller, which includes the following steps:

[0047] S100. Based on the preset multi-distribution color transfer rules, the color style of the target domain image is transferred to the source domain image, thereby obtaining the source domain dataset, and the preset first model is trained using the obtained source domain dataset.

[0048] In this first embodiment, step S100 may further include:

[0049] S110. Divide the target domain image into multiple target domain sub-images along the vertical direction, and calculate the color parameter distribution of each target domain sub-image.

[0050] The multi-color distribution variable of the target domain sub-image is defined as follows: , K represents the number of sub-images into which each image is split, such as... Figure 5 As shown.

[0051] Subsequently, a batch of target domain images were read from the dataset. Each batch has the following dimensions: B×C×H×W, where B is the batch size, and C, H, and W represent the image channels, height, and width, respectively. Weak enhancement operations such as image luminance transformation and inversion are performed on each batch.

[0052] The color parameter distribution of the target domain sub-image includes at least the mean of the target domain sub-image. and variance .

[0053] The calculation process of the mean and variance of the target domain sub-image is expressed by the following formula:

[0054]

[0055]

[0056] in, This represents the mean of the target domain sub-image. This represents the variance of the target domain sub-image. Indicates the first The k-th sub-image of the target domain of the target domain image is E(·), which represents the expectation operation.

[0057] S120. The color parameter distribution of the target domain sub-image is updated by exponential moving average, thereby obtaining the global multi-color distribution set of the target domain.

[0058] The process is expressed by the following formula:

[0059]

[0060] Among them, among them, , Let be the mean and variance of the i-th color distribution in the updated global multi-color distribution set of the target domain, respectively. , , respectively, are the mean and variance of the historical target domain sub-images, and γ is the EMA attenuation coefficient, with a value range of 0.9~0.99.

[0061] S130. Divide the source domain image into multiple source domain sub-images along the vertical direction, and calculate the color parameter distribution of each source domain sub-image.

[0062] The multi-color distribution of the source domain sub-image is defined as follows: , K represents the number of sub-images that each image is split into.

[0063] Subsequently, a batch of target domain images were read from the dataset. Each batch has the following dimensions: B×C×H×W, where B is the batch size, and C, H, and W represent the image channels, height, and width, respectively, and each is weakly enhanced.

[0064] The color parameter distribution of the source sub-image includes at least the mean of the source sub-image. and variance

[0065] The calculation process of the mean and variance of the source domain sub-image can then be expressed by the following formula:

[0066]

[0067]

[0068] in, This represents the mean of the source domain sub-image. This represents the variance of the source domain sub-image. Indicates the first The k-th sub-image of the target domain of the target domain image is E(·), which represents the expectation operation.

[0069] S140. Randomly select color distribution parameters from the target domain global multi-color distribution set and map them onto the corresponding source domain sub-image to complete the style adaptation of the source domain image.

[0070] This process is expressed by the following formula:

[0071]

[0072] in, This represents the k-th source domain sub-image of the j-th source domain image after style adaptation. This represents the k-th source domain sub-image of the original j-th source domain image. , , respectively, are the mean and variance of the source domain sub-images. , These are color distribution parameters randomly selected from the global multi-color distribution set of the target domain.

[0073] S150. Input the style-adapted source domain sub-image and the corresponding real disparity map into the first model, and train the first model using the smooth L1 paradigm.

[0074] S200. Construct a teacher model and a student model with the exact same structure as the first model, and initialize the parameters of the teacher model and the student model to the parameters of the first model after training, such as... Figure 4 As shown.

[0075] S300. Input the unlabeled image of the target domain into the teacher model to generate a pseudo-disparity map. Construct hybrid training data based on the geometric characteristics of stereo matching epipolar lines. Input the pseudo-disparity map and the hybrid training data into the student model. Continuously optimize the parameters of the student model using consistency learning.

[0076] In this first embodiment, step S300 may further include:

[0077] S310. Select a strip-shaped region with the same width as the image width, and perform bidirectional mixing on the source domain image and the target domain image after style adaptation to generate two sets of mixed images and corresponding mixed disparity maps. Then, perform channel random arrangement enhancement on the mixed images.

[0078] The traditional Cutmix method is used for stereo matching because it cannot determine the position of the box selected in the left image in the right image. To solve this problem, the method proposed in this invention, based on the epipolar geometry of stereo matching, randomly selects a strip region with the same width as the image and performs a bidirectional copy of the source and target domain images. That is, the strip region selected in the target domain image is swapped with the corresponding strip region in the source domain image, generating two hybrid images. This process is expressed by the following formula:

[0079]

[0080]

[0081] Where M is the mask for the striped region. For the target domain image, For the source domain image after style adaptation, The target domain pseudo-disparity map generated for the teacher model. This is the true disparity map of the source domain image. , , and These are two mixed images and their corresponding disparity maps.

[0082] S320. Input the two sets of mixed images into the student model and optimize the parameters of the student model using the joint loss function.

[0083] This process is expressed by the following formula:

[0084]

[0085]

[0086]

[0087]

[0088]

[0089] in, To monitor losses, Losses due to pseudo-supervision This is the loss due to the fusion of luminance and parallax smoothness. For parallax smoothing loss, The total loss for the target domain dataset; , These are the weighting coefficients. The source domain disparity map output for the student model. The left image, SSIM(·) is the reconstructed image mapped from the right image to the left image, and SSIM(·) is the structural similarity index. , Let be the partial derivatives in the x and y directions, respectively. For disparity maps, It is a balance coefficient with a value of 0.8. The loss is for disparity smoothing; therefore, the loss on the target domain dataset is... .

[0090] S400: Deploy the optimized student model as a domain-adaptive stereo matching model, thereby improving the domain-adaptive performance in the stereo matching network.

[0091] The algorithm is implemented in PyTorch, and all models are trained using the Adam optimizer on two NVIDIA GeForce RTX 4090 GPUs. To verify the superiority of this method, a series of quantitative analysis experiments were designed. Three general datasets in the stereo matching domain were selected: SceneFlow, KITTI2012, and KITTI2015. SceneFlow was used as the source domain dataset, and the others as the target domain datasets for validation. All original images were randomly cropped to a uniform resolution of 336×504. All models were trained for 20 epochs with a batch size of 8 (including 8 pairs of source domain images and 8 pairs of target domain images, applicable to this model and the AdaStereo model). For this model, the first 15 epochs used supervised training of the student model based on the real disparity map, and the remaining 5 epochs used consistency training through a teacher-student network. The initial learning rate was set to 0.001, and decreased by a factor of 2 after the 12th, 16th, and 18th epochs, respectively. The experiment used End-Point Error (EPE) and Disparity Outlier Rate as evaluation functions. The experimental results are shown in Table 1.

[0092] Table 1. Performance Comparison of SceneFlow -> KITTI 2012-2015

[0093]

[0094] The experimental data above show that the algorithm can significantly improve the performance of domain adaptive stereo matching. Moreover, on both datasets, both performance metrics outperform other comparative algorithms. When applied to tasks such as measurement, autonomous driving, and 3D reconstruction, it will further improve the performance of task processing.

[0095] It should be understood that, in various embodiments of the present invention, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0096] Furthermore, the terms "system" and "network" are often used interchangeably in this paper. The term "and / or" in this paper merely describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " in this paper generally indicates that the preceding and following related objects have an "or" relationship.

[0097] It should be understood that, in the embodiments of the present invention, "B corresponding to A" means that B is associated with A, and B can be determined based on A. However, it should also be understood that determining B based on A does not mean that B is determined solely based on A; B can also be determined based on A and / or other information.

[0098] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0099] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0100] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, apparatuses, or units, or they may be electrical, mechanical, or other forms of connection.

[0101] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0102] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0103] From the above description of the embodiments, those skilled in the art will clearly understand that the present invention can be implemented in hardware, firmware, or a combination thereof. When implemented in software, the above-described functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of a computer program from one place to another. Storage media can be any available medium accessible to a computer. For example, but not limited to, computer-readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code having the form of instructions or data structures and accessible to a computer. Furthermore, any connection can suitably be a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of the medium. As used in this invention, disk and disc include compressed optical discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, wherein disks typically magnetically copy data, while discs optically copy data using lasers. The combinations described above should also be included within the scope of protection for computer-readable media.

[0104] In summary, the above description is merely a preferred embodiment of the technical solution of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A domain-adaptive stereo matching method based on multi-distribution color transformation and consistency, characterized in that, Includes the following steps: S100. Based on the preset multi-distribution color transfer rules, the color style of the target domain image is transferred to the source domain image, thereby obtaining the source domain dataset, and the preset first model is trained using the obtained source domain dataset. S200. Construct a teacher model and a student model with the same structure as the first model, and initialize the parameters of the teacher model and the student model to the parameters of the first model after training. S300. Input the unlabeled image of the target domain into the teacher model to generate a pseudo disparity map, construct hybrid training data based on the geometric characteristics of stereo matching epipolar lines, and input the pseudo disparity map and the hybrid training data into the student model, and continuously optimize the parameters of the student model using consistency learning. S400: Deploy the optimized student model as a domain-adaptive stereo matching model, thereby improving the domain-adaptive performance in the stereo matching network.

2. The domain-adaptive stereo matching method according to claim 1, characterized in that, In step S100, based on a preset multi-distribution color transfer rule, the color style of the target domain image is transferred to the source domain image to obtain a source domain dataset. The obtained source domain dataset is then used to train a preset first model. The domain adaptive stereo matching method may further include: S110. Divide the target domain image into multiple target domain sub-images along the vertical direction, and calculate the color parameter distribution of each target domain sub-image. S120. The color parameter distribution of the target domain sub-image is updated by exponential moving average, thereby obtaining the global multi-color distribution set of the target domain. S130. Divide the source domain image into multiple source domain sub-images along the vertical direction, and calculate the color parameter distribution of each source domain sub-image. S140. Randomly select color distribution parameters from the target domain global multi-color distribution set and map them onto the corresponding source domain sub-image to complete the style adaptation of the source domain image. S150. Input the style-adapted source domain sub-image and the corresponding real disparity map into the first model, and train the first model using the smooth L1 paradigm.

3. The domain-adaptive stereo matching method according to claim 2, characterized in that, The color parameter distribution of the target domain sub-image includes at least the mean of the target domain sub-image. and variance ; The color parameter distribution of the source sub-image includes at least the mean of the source sub-image. and variance .

4. The domain-adaptive stereo matching method according to claim 3, characterized in that, The calculation process of the mean and variance of the target domain sub-image is expressed by the following formula: ； ； in, This represents the mean of the target domain sub-image. This represents the variance of the target domain sub-image. Indicates the first The k-th sub-image of the target domain of the target domain image is E(·), which represents the expectation operation.

5. The domain-adaptive stereo matching method according to claim 2, characterized in that, The process of updating the color parameter distribution of the target domain sub-image using an exponential moving average method, thereby obtaining the global multi-color distribution set of the target domain, is expressed by the following formula: ； Among them, among them, , Let be the mean and variance of the i-th color distribution in the updated global multi-color distribution set of the target domain, respectively. , , respectively, are the mean and variance of the historical target domain sub-images, and γ is the EMA attenuation coefficient, with a value range of 0.9~0.

99.

6. The domain-adaptive stereo matching method according to claim 2, characterized in that, The process of randomly mapping the color distribution parameters in the global multi-color distribution set of the target domain to the corresponding source domain sub-image, thereby completing the style adaptation of the source domain image, is expressed by the following formula: ； in, This represents the k-th source domain sub-image of the j-th source domain image after style adaptation. This represents the k-th source domain sub-image of the original j-th source domain image. , , respectively, are the mean and variance of the source domain sub-images. , These are color distribution parameters randomly selected from the global multi-color distribution set of the target domain.

7. The domain-adaptive stereo matching method according to claim 1, characterized in that, In step S300, where the unlabeled image of the target domain is input into the teacher model to generate a pseudo-disparity map, hybrid training data is constructed based on the epipolar geometric characteristics of stereo matching, and the pseudo-disparity map and the hybrid training data are combined and input into the student model, and the parameters of the student model are continuously optimized using consistency learning, the domain adaptive stereo matching method may further include: S310. Select a strip-shaped region with the same width as the image width, and perform bidirectional mixing on the source domain image and the target domain image after style adaptation to generate two sets of mixed images and corresponding mixed disparity maps. Then, perform channel random arrangement enhancement on the mixed images. S320. Input the two sets of mixed images into the student model and optimize the parameters of the student model using the joint loss function.

8. The domain-adaptive stereo matching method according to claim 7, characterized in that, The process described by the following formula expresses the selection of a strip-shaped region with a width consistent with the image width, bidirectional blending of the style-adapted source domain image and the target domain image to generate two sets of blended images and corresponding blended disparity maps, and the calculation process of channel random arrangement enhancement of the blended images: ； ； Where M is the mask for the strip region. For the target domain image, For the source domain image after style adaptation, The target domain pseudo-disparity map generated for the teacher model. This is the true disparity map of the source domain image. , , and These are two mixed images and their corresponding disparity maps.

9. The domain-adaptive stereo matching method according to claim 7, characterized in that, The process of inputting two sets of mixed images into the student model and optimizing the parameters of the student model using a joint loss function is expressed by the following formula: ； ； ； ； ； in, To monitor losses, Losses due to pseudo-supervision This is the loss due to the fusion of luminance and parallax smoothness. For parallax smoothing loss, The total loss for the target domain dataset; , These are the weighting coefficients. The source domain disparity map output for the student model. The left image, SSIM(·) is the reconstructed image mapped from the right image to the left image, and SSIM(·) is the structural similarity index. , Let be the partial derivatives in the x and y directions, respectively. For disparity maps, The balancing coefficient is set to 0.8; therefore, the loss on the target domain dataset is... .

10. The domain-adaptive stereo matching method according to claim 1, characterized in that, The student model uses pseudo-random 3D convolution to replace part of the traditional 3D convolution. The pseudo-random 3D convolution generates the convolution kernel through preset pseudo-random kernel parameters, which reduces the number of model parameters while maintaining stereo matching performance.

Images