A domain generalization stereo vision matching method based on hierarchical visual transformation

Abstract

The application discloses a domain generalization stereo vision matching method based on hierarchical visual conversion, and the steps include: 1, left and right eye stereo matching image data acquisition and preprocessing; 2, generating different levels of transformation images; 3, constructing loss for training image visual processing network T to learn parameters; 4, extracting image features and constructing loss for training robust feature extraction network f(·); 5, calculating matching cost; 6, inputting the matching cost into a cost aggregation module and a disparity subdivision module to obtain a predicted disparity map; 7, constructing a smooth-l1 loss; 8, constructing a total loss function; 9, training a complete stereo matching model using the total loss function. The application can significantly improve the generalization performance of the model on different real data sets under the condition of only using existing synthetic data sets for training.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a domain generalization stereo matching technology. BACKGROUND

[0002] Stereo matching technology is a basic task in computer vision, which aims to find the matching corresponding relationship of the same pixels between a given stereo image pair, so as to calculate the disparity result at the pixel level. In the field of three-dimensional vision, according to the disparity value and camera parameters obtained by the stereo matching technology, the depth value and three-dimensional coordinates of each pixel in the stereo image can be obtained. Therefore, in recent years, stereo matching technology has been widely used in robot navigation and automatic driving fields, and has attracted more and more attention in the field of computer vision.

[0003] With the development of deep learning, deep stereo networks based on convolutional neural networks have shown impressive performance due to their powerful feature representation ability. However, due to the lack of sufficient labeled real training data, the existing state-of-the-art stereo networks are usually trained on large-scale synthetic data, such as SceneFlow. However, there is usually a large difference (color, illumination and background) between the synthetic dataset and the real dataset used for testing, so the trained model usually performs very poorly in the real dataset, which seriously hinders the application of stereo matching technology in the real world.

[0004] Based on this problem, the existing solutions can be divided into two categories: 1) Fine-tune the stereo matching network trained with synthetic data using labeled target domain real data. 2) Train domain adaptive stereo matching network jointly using synthetic data and unlabeled target domain real data. Although these attempts improve the performance of the model on the real dataset, these methods require real data of the target domain during training. However, in the practical application of stereo matching technology (such as the field of automatic driving), real stereo image pairs usually have a variety of styles, such as different driving weather, scenes, illumination, etc. It is difficult for the model to access sufficient and diverse real data during the training phase to improve the generalization performance on various real datasets. Therefore, it is necessary to design an effective method to significantly improve the generalization performance of the model on different real datasets under the condition of only using existing synthetic datasets for training. SUMMARY

[0005] In order to solve the above problems existing in the prior art, a hierarchical visual transformation domain generalization stereo matching method is proposed, so as to generate more diverse visual transformation images of different levels for network training under the condition of only using existing synthetic datasets for training, thereby significantly improving the generalization performance of the model on different real datasets.

[0006] The application achieves the above-mentioned purposes by adopting the following technical scheme:

[0007] The domain generalization stereo vision matching method based on hierarchical visual conversion has the characteristics that the following steps are included:

[0008] Step 1, acquisition and preprocessing of left-right stereo matching image data:

[0009] Step 1.1, obtaining a large-scale synthetic stereo matching dataset as a training set, wherein any pair of left-right stereo matching images of the training set is defined as {X l ,X r}, wherein X l represents a left eye image, and X r represents a right eye image;

[0010] Step 2, any one of the images in {X l ,X r} is recorded as an input image X and input into an image visual processing network T to generate transformed images of different levels:

[0011] Step 2.1, constructing an image visual conversion module T G (·) in the image visual processing network T;

[0012] Defining that the image visual conversion module T G (·) includes four visual attribute transformation units, including: a brightness transformation unit a contrast transformation unit a saturation transformation unit and a hue transformation unit

[0013] After the input image X is sequentially processed by the four randomly ordered visual attribute transformation units, the transformed image T G (X) of the overall level is obtained;

[0014] Step 2.2, constructing an image visual conversion module T L (·) in the image visual processing network T;

[0015] The image visual conversion module T L (·) first divides the input image X into N'×N' non-overlapping image blocks wherein, represents the i-th image block, and N' represents the number of rows or columns divided in the horizontal or vertical direction;

[0016] The i-th image block Input to the whole level image visual conversion module T G The i-th whole level transformed image block T

[0017] All image blocks after the whole level image visual conversion module T G The local level transformed image T L (X) is obtained by processing using formula (5);

[0018]

[0019] In formula (4), Merge(·) represents a merging operation;

[0020] Step 2.3, constructing the image visual conversion module T P (·) of the pixel level in the image visual processing network T;

[0021] The image visual conversion module T P (·) is obtained by perturbing the input image X using formula (6) to obtain the pixel level transformed image T P (X):

[0022] T P (X) = X + (μσ(W) + β)·P (6)

[0023] In formula (6), is a random generated Gaussian perturbation matrix with mean 0 and standard deviation 1, is a parameter matrix to be learned, h represents the height of the input image X, w represents the width of the input image X, and 3 represents the number of channels;

[0024] Step 3, inputting the input image X and the generated transformed images T G (X) of different levels, T L (X) and T P (X) into the discriminator network φ(·) for processing, thereby constructing the domain label classification cross-entropy loss L ce and the feature difference loss L sim The to-be-learned parameters in the image visual processing network T are used for training:

[0025] Step 3.1, the domain label classification cross-entropy loss L ce is constructed using formula (7):

[0026]

[0027] In formula (7), CE(·,·) represents a standard cross-entropy loss, T J(X) represents the generated J-level transformed image; J e {G, L, P}, wherein G represents the global level, L represents the local level, and P represents the pixel level; Y d represents the domain label or the level J corresponding to the transformed image T J of the input image X;

[0028] Step 3.2, constructing the feature difference loss L sim using formula (8)

[0029]

[0030] In formula (8), Cos(·,·) represents a cosine similarity function;

[0031] Step 4: inputting the input image X and the generated transformed images T G (X) of different levels into a feature extraction network f(·) composed of a 2D full convolutional network for processing to obtain four sets of left-right eye feature maps, thereby constructing a feature distance loss L L using formula (9) P (X) and T dist (X) are input into a feature extraction network f(·) composed of a 2D full convolutional network for processing to obtain four sets of left-right eye feature maps, thereby constructing a feature distance loss L pred using formula (9) for training the robust feature extraction network f(·):

[0032]

[0033] In formula (9), ||·||2 represents an L2 norm;

[0034] Step 5: calculating the matching cost of each of the four sets of left-right eye feature maps;

[0035] Step 6: constructing a cost aggregation module composed of a 3D full convolutional network and a disparity subdivision module composed of a 2D full convolutional network;

[0036] The four sets of matching costs are sequentially input into the cost aggregation module and the disparity subdivision module for processing to obtain four sets of predicted disparity maps;

[0037] Step 7: constructing a smooth-l1 loss

[0038]

[0039] In formula (10), Y pred represents any one set of predicted disparity maps; Y gt represents the corresponding real disparity map; represents a smoothed L1 loss;

[0040] Step 8: constructing a total loss function of the stereo matching network using formula (11)

[0041]

[0042] In formula (11), λ1, λ2 and λ3 are three hyperparameters;

[0043] Step 9, based on the training set, the stereo matching network composed of the image visual processing network T, the discriminant network φ(·), the feature extraction network f(·), the cost aggregation module and the disparity subdivision module are trained by using the gradient descent method, and the total loss function is calculated to update the network parameters until the total loss function converges, thereby obtaining the optimal stereo matching network for matching the left and right stereo matching images.

[0044] The domain generalization stereo vision matching method based on hierarchical visual conversion has the characteristics that the four visual attribute conversion units in the image visual conversion module T G (·) are processed according to the following process:

[0045] Brightness conversion unit Contrast conversion unit Saturation conversion unit All use formula (1) to perform image visual conversion on the input image X to obtain the output image after changing any one visual attribute I , and I∈{B,C,S} respectively represent brightness, contrast and saturation;

[0046]

[0047] In formula (1), o I (X) represents a transformed image that completely converts any one visual attribute I of the input image X, and when I=B, o I (X)=X·O, wherein O represents a zero matrix; when I=C, o I (X)=Avg(Gray(X)), wherein Gray(·) represents converting an RGB image to a grayscale image, and Avg(·) represents calculating the average value of all pixels; when I=S, o I (X)=Gray(X); represents the weight of any one visual attribute I of the input image X in the corresponding visual attribute of the output image , and the value is randomly selected within the adjustable range , wherein, represents the lower bound of the value, represents the upper bound of the value, ​​An upper limit of the value, and is obtained by formula (2) respectively.

[0048]

[0049] In formula (2), sigma (·) represents a sigmoid function. And And are two parameters to be learned, and mu and beta are two hyperparameters.

[0050] The global sub-transformation Using formula (3) to perform overall level image visual conversion on the input image X, to obtain the output image after changing the hue attribute

[0051]

[0052] In formula (3), h represents a hue component, s represents a saturation component, v represents a transparency component, and Rgb (·) represents converting an image from an HSV space to an RGB space. The hue attribute of the input picture X occupies the weight in the output image Corresponding hue attribute, and the value is randomly selected in the adjustable range Wherein, Indicates The lower limit of the value, Indicates The upper limit of the value, and is obtained by formula (4) respectively:

[0053]

[0054] In formula (4), And Are two parameters to be learned.

[0055] The electronic device comprises a memory and a processor, and the memory is used for storing a program supporting the processor to execute the domain generalization stereo visual matching method, and the processor is configured to execute the program stored in the memory.

[0056] The computer readable storage medium stores a computer program, and when the computer program is run by a processor, the steps of the domain generalization stereo visual matching method are executed.

[0057] Compared with the prior art, the beneficial effects of the present application are as follows:

[0058] 1. The image visual processing network T is proposed, and the original stereo matching image is input into the image visual processing network T to generate three different levels (overall level, local level and pixel level) of transformed images, and the three levels of transformed images are also used as training samples for training, thereby solving the problem of single style of the existing synthetic training data set, and realizing more robust stereo image matching.

[0059] 2. The domain label classification cross-entropy loss and the feature difference loss are designed to train the to-be-learned parameters in the image visual processing network T, so that the visual difference between the generated three different levels of transformed images and the original images is maximized, thereby further improving the style diversity of the existing training data set, avoiding the extraction of non-robust apparent style features by the existing stereo matching network, and improving the generalization ability of the existing stereo matching network.

[0060] 3. The feature distance loss is designed to directly reduce the difference between the original image features extracted by the feature extraction network f(·) and the three different levels of transformed image features, so that the existing stereo matching model can use more robust semantic or structural features to realize correct matching between different stereo image pairs, and such features are reliable and applicable for both synthetic data sets and real data sets; this design further improves the domain generalization performance of the stereo matching network. BRIEF DESCRIPTION OF DRAWINGS

[0061] Figure 1 is a flowchart of the domain generalization stereo vision matching method based on the hierarchical visual conversion method of the present application. DETAILED DESCRIPTION

[0062] In this embodiment, as shown in Figure 1 , a domain generalization stereo vision matching method based on hierarchical visual conversion includes the following steps:

[0063] Step 1, acquisition and preprocessing of left and right stereo matching image data:

[0064] Step 1.1, obtaining a large-scale synthetic stereo matching data set as a training set, wherein any pair of left and right stereo matching images of the training set is defined as {X l ,X r}, wherein X l represents a left image, and X r represents a right image; in this embodiment, the selected synthetic stereo matching data set is the SceneFlow data set, and the real stereo matching data sets KITTI 2012, KITTI 2015, Middlebury and ETH3D are obtained as test data sets.

[0065] Step 2, any one of {X l ,X r} is recorded as input image X and input into image visual processing network T to generate transformed images of different levels:

[0066] Step 2.1, constructing image visual conversion module T G (·) in image visual processing network T

[0067] Defining image visual conversion module T G (·) contains four visual attribute conversion units, including: brightness conversion unit contrast conversion unit saturation conversion unit and hue conversion unit

[0068] After input image X is processed by the four randomly ordered visual attribute conversion units, the overall level transformed image T G (X) is obtained; in this embodiment, the purpose of randomly ordering the four visual attribute conversion units is to enhance the style diversity of the generated overall level transformed image;

[0069] The four visual attribute conversion units in image visual conversion module T G (·) are processed as follows:

[0070] Brightness conversion unit contrast conversion unit saturation conversion unit All use formula (1) to perform image visual conversion on input image X to obtain output image after changing any one visual attribute I and I∈{B,C,S}, respectively representing brightness, contrast and saturation;

[0071]

[0072] In formula (1), o I (X) represents the transformed image of input image X after complete conversion of any one visual attribute I, and when I=B, o I (X) = X·O, where O represents a zero matrix; when I=C, o I (X) = Avg(Gray(X)), where Gray(·) represents converting an RGB image to a grayscale image, and Avg(·) represents calculating the average value of all pixels; when I=S, o I (X) = Gray(X); represents any one visual attribute I of input image X in output image The corresponding weight of visual attributes, and within the adjustable range. A value is randomly selected from within, where, express The lower bound of the value, express The upper bound of the value is obtained from equation (2);

[0073]

[0074] In equation (2), σ(·) represents the sigmoid function; and There are two parameters to be learned, and μ and β are two hyperparameters; in this embodiment, and Each is initialized with a random real number that follows a Gaussian distribution with a mean of 0 and a variance of 1. The hyperparameters μ and β are set to 0.1 and 0.15, respectively.

[0075] Global subtransformation Equation (3) is used to perform a global hierarchical image visual transformation on the input image X to obtain the output image with altered tone attributes.

[0076]

[0077] In equation (3), h represents the hue component, s represents the saturation component, v represents the transparency component, and Rgb(·) represents converting the image from HSV space to RGB space; This represents the tonal attribute of the input image X in the output image. The weight of the corresponding hue attribute, and within the adjustable range. A value is randomly selected from within, where, express The lower bound of the value, express The upper bound of the value is obtained from equation (4):

[0078]

[0079] In equation (4), and These are two parameters to be learned; in this embodiment, and Each is initialized with a random real number, which follows a Gaussian distribution with mean 0 and variance 1;

[0080] Step 2.2: Construct the local-level image visual transformation module T in the image visual processing network T. L (·);

[0081] image visual conversion module T L (·) first divide the input image X into N'xN' non-overlapping image blocks wherein, represents the i-th image block, and N' represents the number of rows or columns divided in the horizontal or vertical direction; in this embodiment, N' is set to 4;

[0082] the i-th image block is input into the overall-level image visual conversion module T G (·) for processing to obtain the i-th overall-level transformed image block In this example, the initialized to-be-learned parameters of the overall-level image visual conversion module T G (·) passed through by different image blocks are different, so that the change degrees of the four image visual attributes (brightness, contrast, saturation, and hue) of different image blocks are different, thereby enhancing the style diversity between different image blocks generated.

[0083] All image blocks after passing through the overall-level image visual conversion module T G (·) are processed using formula (5) to obtain the local-level transformed image T L (X);

[0084]

[0085] In formula (5), Merge(·) represents a merging operation;

[0086] Step 2.3, constructing the image visual conversion module T P (·) of the pixel level in the image visual processing network T;

[0087] The image visual conversion module T P (·) performs perturbation on the input image X through formula (6) to obtain the pixel-level transformed image T P (X):

[0088] T P (X) = X + (μσ(W) + β)·P (6)

[0089] In formula (6), is a randomly generated Gaussian perturbation matrix with a mean of 0 and a standard deviation of 1, is a to-be-learned parameter matrix, h represents the height of the input image X, w represents the width of the input image X, and 3 represents the number of channels;

[0090] In this embodiment, the original left and right binocular fusion stereo matching images are respectively input into three different levels of image visual conversion modules to generate different levels of conversion images. These generated conversion images and the original images will be used together for subsequent stereo matching network training, which greatly enhances the style diversity of existing training data, thereby avoiding the learning of non-robust apparent style features by the network. In addition, this operation is not performed in the test stage, so that the present application greatly enhances the generalization of the existing stereo matching network without introducing additional time and memory overhead.

[0091] Step 3, input the generated different levels of conversion images T G (X) and the original image X into the discriminant network φ(·) for processing, thereby constructing the domain label classification cross-entropy loss L L (X) and T P (X) into the discriminant network φ(·) for processing, thereby constructing the domain label classification cross-entropy loss L ce and the feature difference loss L sim The parameters to be learned in the image visual processing network T for training are as follows:

[0092] Step 3.1, the domain label classification cross-entropy loss L ce is constructed by using formula (7):

[0093]

[0094] In formula (7), CE(·,·) represents the standard cross-entropy loss, T J (X) represents the generated J-level conversion image; J e {G, L, P}, wherein G represents the overall level, L represents the local level, and P represents the pixel level; Y d represents the domain label of the input image X or the domain label corresponding to the conversion image T J (X) of level J;

[0095] In this embodiment, the domain label classification cross-entropy loss L ce is used to train the discriminant network φ(·), so that the discriminant network φ(·) can correctly identify the domain category (original image, overall level conversion image, local level conversion image and pixel level conversion image) to which different input pictures belong.

[0096] Step 3.2, the feature difference loss L sim is constructed by using formula (8):

[0097]

[0098] In formula (8), Cos(·,·) represents the cosine similarity function.

[0099] In this embodiment, under the premise that the discrimination network φ(·) can correctly identify the domain categories to which different input pictures belong, the feature difference loss L sim to maximize the feature distance between the original image and the transformed images at different levels, thereby maximizing the visual style difference between the original image and the transformed images at different levels, and thereby further compensating for the style diversity of the existing training data.

[0100] Step 4: input the input image X and the generated transformed images T G (X), T L (X) and T P (X) into the feature extraction network f(·) composed of a 2D fully convolutional network for processing to obtain four sets of left-right eye feature maps, thereby constructing the feature distance loss L dist for training the robust feature extraction network f(·):

[0101]

[0102] In formula (9), ||·||2 represents the L2 norm;

[0103] In this embodiment, the feature extraction network f(·) can be composed of different network architectures according to the different selected stereo matching benchmark networks, and the feature distance loss L dist The smaller the distance between the original image features and the transformed image features at different levels extracted by the feature extraction network f(·) is directly constrained, which promotes the feature extraction network f(·) to better learn robust semantic or structural features, greatly improving the generalization of the existing stereo matching benchmark network.

[0104] Step 5: select different matching cost calculation methods according to the different selected stereo matching benchmark networks, i.e., calculate the matching costs of the four sets of left-right eye feature maps respectively.

[0105] Step 6: construct a cost aggregation module composed of a 3D fully convolutional network and a disparity subdivision module composed of a 2D fully convolutional network;

[0106] Input the four sets of matching costs into the cost aggregation module and the disparity subdivision module respectively for processing to obtain four sets of predicted disparity maps;

[0107] In this embodiment, the cost aggregation module and the disparity subdivision module can be composed of different network architectures according to the different selected stereo matching benchmark networks.

[0108] Step 7: construct the smooth-l1 loss L

[0109]

[0110] In formula (10), Y pred represents any one set of predicted disparity map; Y gt represents the corresponding real disparity map; represents the smoothed L1 loss;

[0111] In this embodiment, for each input image X, three transformed images T G (X), T L (X) and T P (X) of different levels are generated first. The complete smooth-l1 loss G (X), T L (X) and T P (X) are calculated from the predicted disparity map obtained by inputting X, T

[0112] Step 8: the overall loss function of the stereo matching network is constructed by using formula (11)

[0113]

[0114] In formula (11), λ1, λ2 and λ3 are three hyperparameters; in this example, the hyperparameters λ1, λ2 and λ3 are taken as 1, 0.5 and 0.5 respectively.

[0115] Step 9: based on the training set, the stereo matching network composed of the image visual processing network T, the discriminant network φ(·), the feature extraction network f(·), the cost aggregation module and the disparity subdivision module is trained by using the gradient descent method, and the overall loss function is calculated to update the network parameters until the overall loss function converges, thereby obtaining the optimal stereo matching network for matching the left and right stereo matching images.

[0116] In this embodiment, an electronic device includes a memory for storing a program supporting the processor to execute the above method and a processor configured to execute the program stored in the memory.

[0117] In this embodiment, a computer readable storage medium has a computer program stored thereon, and the computer program is executed by a processor to perform the steps of the above method.

[0118] The test results of the present application are further described in combination with the following charts:

[0119] In order to verify the generalization performance improvement effect of the present application on the existing stereo matching benchmark network, four kinds of conventional stereo matching benchmark networks are selected, including: PSMNet, GwcNet, CFNet and RAFT-Stereo, and Ours represents the method proposed in the present application embedded into the corresponding benchmark stereo matching network. All models are trained only using the synthetic dataset SceneFlow in the training stage, and are tested using KITTI 2012, KITTI 2015, Middlebury and ETH3D in the test stage, respectively. The EPE (mean pixel error) and D1 (pixel ratio with error greater than n) indicators are used as test indicators, wherein for KITTI 2012 and KITTI 2015, n = 3; for Middlebury, n = 2; for ETH3D, n = 1. The experimental results are shown in Table 1.

[0120] Table 1 Comparison of generalization results of benchmark stereo matching network and stereo matching network using the method of the present application

[0121]

[0122] From the experimental results in Table 1, it can be seen that: under the verification conditions of all benchmark stereo matching networks and all test datasets, the method proposed in the present application significantly improves the generalization performance of the benchmark stereo matching network. In addition, the method proposed in the present application only participates in the calculation in the training stage and does not execute in the test stage, thereby avoiding introducing any time and calculation overhead.

Claims

1. A domain-generalized stereo vision matching method based on hierarchical visual transformation, characterized in that, Includes the following steps: Step 1: Acquisition and preprocessing of left and right eye stereo matching image data: Step 1.1: Obtain a large-scale synthetic stereo matching dataset as the training set. Any pair of left and right eye stereo matching images in the training set is defined as... ,in, Represented as the left eye image, Represented as the right eye image; Step 2, Let any one of the images be denoted as the input image. The images are then input into the image vision processing network T to generate transformed images at different levels. Step 2.1: Construct the overall hierarchical image visual transformation module in the image visual processing network T. ; Define image visual conversion module It contains four visual attribute transformation units, including: brightness transformation unit. Contrast conversion unit Saturation transformation unit and tone conversion unit ; The input image After being processed by four randomly ordered visual attribute transformation units, the overall hierarchical transformed image is obtained. ; Step 2.2: Construct local-level image visual transformation modules in the image visual processing network T. ; The image visual conversion module First, input image Divided into Non-overlapping image blocks ,in, Represents the i-th image patch. Indicates the number of rows or columns in the horizontal or vertical direction; The i-th image patch Input to the overall hierarchical image visual transformation module The image is processed to obtain the transformed image patch of the i-th overall level. ; Using formula (5) to perform image visual transformation module at the overall level All subsequent image patches are processed to obtain a local-level transformed image. ; (5) In equation (4), Indicates a merge operation; Step 2.3: Construct the pixel-level image visual transformation module in the image visual processing network T. ; The image visual conversion module The input image is processed by equation (6). Perturbation is applied to obtain a pixel-level transformed image. : (6) In equation (6), It is a randomly generated Gaussian perturbation matrix with a mean of 0 and a standard deviation of 1. It is a parameter matrix to be learned, where h represents the height of the input image X, w represents the width of the input image X, and 3 represents the number of channels; This represents the sigmoid function; and These are two hyperparameters; Step 3: Input image and the generated transformed images at different levels , and Input authentication network This is processed to construct the domain label classification cross-entropy loss. and feature difference loss The parameters to be learned in the image vision processing network T: Step 3.1: Use equation (7) to construct the domain label classification cross-entropy loss. : (7) In equation (7), Represents the standard cross-entropy loss. Indicates the generated Image transformation at different levels; Where G represents the global level, L represents the local level, and P represents the pixel level; Indicates the input image domain tags or hierarchy Corresponding transformed image The tags for each domain; Step 3.2: Construct the feature difference loss using equation (8). : (8) In equation (8), Represents the cosine similarity function; Step 4: Input image and the generated transformed images at different levels , and Input is a feature extraction network consisting of a 2D fully convolutional network. The feature maps of the left and right eyes are processed to obtain four sets of feature maps, and then the feature distance loss is constructed using equation (9). Used to train robust feature extraction networks : (9) In equation (9), Indicates L2 normal form; Step 5: Calculate the matching cost for each of the four sets of left and right eye feature maps; Step 6: Construct a cost aggregation module consisting of a 3D fully convolutional network and a disparity subdivision module consisting of a 2D fully convolutional network; The four sets of matching costs are input into the cost aggregation module and the disparity subdivision module respectively for processing to obtain four sets of predicted disparity maps. Step 7: Construct smooth- using equation (10) loss ; (10) In equation (10), A disparity map representing any set of predictions; This represents the corresponding true disparity map; This represents the smoothed L1 loss; Step 8: Construct the overall loss function of the stereo matching network using equation (11). : (11) In equation (11), , and There are three hyperparameters; Step 9: Based on the training set, use gradient descent to process the image visual processing network T and the discrimination network. Feature extraction network The stereo matching network, consisting of a cost aggregation module and a disparity subdivision module, is trained, and the overall loss function is calculated. To update network parameters until the overall loss function is reached. The process continues until convergence, thus obtaining the optimal stereo matching network for matching left and right eye stereo images.

2. The domain generalization stereo vision matching method based on hierarchical visual transformation according to claim 1, characterized in that, The image visual conversion module The four visual attribute transformation units are processed according to the following process: Brightness conversion unit Contrast conversion unit Saturation transformation unit Equation (1) is used for all input images. conduct Image visual transformation to obtain any visual attribute Modified output image ,and These represent brightness, contrast, and saturation, respectively. (1) In equation (1), This indicates the input image. Any visual attribute The transformed image undergoes a complete transformation, and when hour, = O, where O represents the zero matrix; when hour, = ,in This represents converting an RGB image to a grayscale image. This represents calculating the average value of all pixels; when hour, = ; Indicates input image Any visual attribute I in the output image The corresponding weight of visual attributes, and within the adjustable range. A value is randomly selected from within, where, express The lower bound of the value, express The upper bound of the value is obtained from equation (2); (2) In equation (2), and These are two parameters to be learned; The color transformation unit Use equation (3) to process the input image Perform a holistic hierarchical image visual transformation to obtain an output image with altered tone attributes. : (3) In equation (3), h represents the hue component. 'v' represents the saturation component, and 'v' represents the transparency component. This indicates that the image has been converted from HSV color space to RGB color space; Indicates input image The hue attribute in the output image The weight of the corresponding hue attribute, and within the adjustable range. A value is randomly selected from within, where, express The lower bound of the value, express The upper bound of the value is obtained from equation (4): (4) In equation (4), and These are two parameters to be learned.

3. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store a program that supports the processor in executing the domain generalization stereo vision matching method of claim 1 or 2, and the processor is configured to execute the program stored in the memory.

4. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program is executed by the processor to perform the steps of the domain generalization stereo vision matching method of claim 1 or 2.

Images