A robust stereo matching method and device based on frequency domain adaptive filtering optimization

Abstract

The present invention relates to a robust stereo matching method and device based on frequency domain adaptive filtering optimization, the method comprising: acquiring binocular images; inputting the acquired binocular images into a stereo matching model, completing pixel matching to obtain a disparity map; the stereo matching model comprising: a feature extraction network for extracting features of the binocular images to obtain left image features and right image features; an adaptive filtering module for converting left image features and right image features to the frequency domain, and adaptively filtering the left image features and right image features, and re-transforming the filtered left image features and right image features to the spatial domain to complete cost body construction; a frequency compensation module for frequency compensation of the left image frequency domain features, and fusing them with aggregated features after cost body aggregation to obtain fused features; a disparity prediction module for performing disparity prediction according to the fused features, completing pixel matching to obtain a disparity map. The present invention can improve the robustness of the stereo matching model to correction errors.

Description

Technical Field

[0001] The present invention relates to the field of stereo matching technology, and in particular to a robust stereo matching method and device based on frequency domain adaptive filtering optimization. Background Art

[0002] Stereo matching is a commonly used method to directly obtain scene depth from images. Its basic principle is to search for corresponding pixels in a pair of rectified binocular images along the horizontal direction, complete the matching of pixels and obtain a disparity map. The advantage of this method is that the projection of points in space on the rectified left and right images must be in the same row. Therefore, when searching for matching points, only pixels in the same row are considered, and the two-dimensional search space is constrained to one-dimensional pixels in the same row, which greatly reduces the computational complexity. However, the premise for the above constraints to be established is that the binocular images used for matching must undergo strict distortion correction and stereo correction. A common method is to use a calibration plate to calculate the intrinsic and extrinsic parameters of the camera offline to complete the calibration. However, during use, the camera will be affected by external environmental factors such as temperature changes and mechanical vibrations, causing the camera parameters to change over time, resulting in increased calibration errors and failure of epipolar constraints. Existing stereo matching algorithms based on deep learning rarely consider these issues, and usually only use accurately rectified binocular images for training and testing. Therefore, these algorithms are very sensitive to calibration errors, especially to extrinsic calibration errors caused by camera rotation and translation. Even a tiny correction error can cause the accuracy of the algorithm to drop sharply, which severely limits the application of stereo matching algorithms in practical scenarios.

[0003] Published in IEEE Conference on Computer Vision and Pattern Recognition in 2022 <practicalstereomatchingviacascadedrecurrentnetworkwithadaptivecorrelation>(CREStereo) designs a recursive refinement hierarchical stereo matching network for non-ideally rectified binocular images captured by camera modules of consumer devices. In this network, the authors propose an adaptive grouped local correlation layer to mitigate the impact of the correction error on the matching results. However, the deformable search window used in CREStereo has a limited matching range and can only guarantee matching accuracy when the correction error is small. When the correction error increases and the corresponding pixels exceed this range, this method has the risk of matching failure. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a robust stereo matching method and device based on frequency domain adaptive filtering optimization, which can improve the robustness of the stereo matching network model to correction errors.

[0005] The technical solution adopted by the present invention to solve the technical problem is: to provide a robust stereo matching method based on frequency domain adaptive filtering optimization, comprising the following steps:

[0006] Acquire binocular images;

[0007] Input the acquired binocular image into the stereo matching model, complete the pixel matching and obtain the disparity map;

[0008] Wherein, the stereo matching model includes:

[0009] Feature extraction network, used to extract the features of the binocular image and obtain the features of the left image and the right image;

[0010] An adaptive filtering module is used to convert the left image features and the right image features into the frequency domain to obtain the left image frequency domain features and the right image frequency domain features, and adaptively filter the left image features and the right image features based on the left image frequency domain features and the right image frequency domain features, and re-transform the filtered left image features and the right image features into the spatial domain to complete the cost volume construction;

[0011] A frequency compensation module, used for performing frequency compensation on the frequency domain features of the left image, and fusing the compensated frequency domain features of the left image with the aggregated features after aggregation of the cost volume to obtain a fused feature;

[0012] The disparity prediction module is used to perform disparity prediction according to the fusion features and complete pixel matching to obtain a disparity map.

[0013] The adaptive filtering module comprises:

[0014] The frequency domain conversion part is used to convert the left image features and the right image features into the frequency domain, and concatenate the real part and the imaginary part of the converted left image features and the real part and the imaginary part of the right image features in the channel dimension to obtain the left image frequency domain features and the right image frequency domain features;

[0015] The frequency domain mask generation part is used to generate a frequency domain mask according to the frequency domain features of the left image and the frequency domain features of the right image;

[0016] A filtering part, used for filtering the left image features and the right image features using the frequency domain mask;

[0017] A spatial domain conversion part, used for converting the filtered left image features and right image features into the spatial domain;

[0018] The cost volume generation part is used to generate the cost volume according to the left image features and the right image features after conversion into the spatial domain.

[0019] The frequency domain mask generation part first maps the frequency domain features into the embedding space through a convolution module including a BN layer and a ReLU activation function, and then obtains a frequency domain mask with a value range of [0,1] through a convolution layer and a Sigmoid activation function.

[0020] The loss function of the frequency domain mask generation part is: Among them, L con The loss function for the frequency domain mask generation part, H and W represent the height and width of the image features, The standard right image features are filtered by using the frequency domain mask during training and converted to the standard right image features in the spatial domain. During training, the perturbed standard right image is filtered using a frequency domain mask and converted to the perturbed right image features in the spatial domain.

[0021] The frequency compensation module comprises:

[0022] The weight calculation part is used to calculate the frequency compensation weight according to the frequency domain mask during adaptive filtering;

[0023] A guiding part, used to guide the frequency domain features of the left image using frequency compensation weights, and convert them into the spatial domain to obtain compensation features;

[0024] The fusion part is used to splice the compensation feature with the aggregated feature after the cost volume is aggregated in the feature dimension, and input it into the feature fusion network to obtain the fused feature.

[0025] The weight calculation part is Calculate the frequency compensation weights, where is the frequency compensation weight, M is the frequency domain mask during adaptive filtering, represents the frequency compensation weight network based on the encoder-decoder structure; σ(·) represents the Sigmoid function.

[0026] The technical solution adopted by the present invention to solve the technical problem is: to provide a robust stereo matching device based on frequency domain adaptive filtering optimization, comprising:

[0027] An acquisition module, used for acquiring binocular images;

[0028] The matching module is used to input the acquired binocular image into the stereo matching model to complete the pixel matching and obtain the disparity map;

[0029] Wherein, the stereo matching model includes:

[0030] Feature extraction network, used to extract the features of the binocular image and obtain the features of the left image and the right image;

[0031] An adaptive filtering module is used to convert the left image features and the right image features into the frequency domain to obtain the left image frequency domain features and the right image frequency domain features, and adaptively filter the left image features and the right image features based on the left image frequency domain features and the right image frequency domain features, and re-transform the filtered left image features and the right image features into the spatial domain to complete the cost volume construction;

[0032] A frequency compensation module, used for performing frequency compensation on the frequency domain features of the left image, and fusing the compensated frequency domain features of the left image with the aggregated features after aggregation of the cost volume to obtain a fused feature;

[0033] The disparity prediction module is used to perform disparity prediction according to the fusion features and complete pixel matching to obtain a disparity map.

[0034] The technical solution adopted by the present invention to solve its technical problem is: providing a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned robust stereo matching method based on frequency domain adaptive filtering optimization are implemented.

[0035] Beneficial Effects

[0036] Due to the adoption of the above technical solution, the present invention has the following advantages and positive effects compared with the prior art: Based on the stereo matching algorithm framework based on deep learning, the present invention designs a frequency domain adaptive filtering module for the correction error existing in the input image. The module transforms the image features into the frequency domain through the fast Fourier transform (FFT) and uses a learnable filter to complete the filtering, effectively suppressing the frequency components sensitive to the correction error and improving the robustness of the model to the correction error. In addition, considering that feature filtering will cause the loss of some beneficial frequency components to a certain extent, the present invention also constructs a frequency compensation module to supplement the key matching information lost due to feature filtering, so as to eliminate the influence of filtering on the accuracy of the matching network. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flowchart of a robust stereo matching method based on frequency domain adaptive filtering optimization according to a first embodiment of the present invention;

[0038] Figure 2 is a framework diagram of a stereo matching model in a first embodiment of the present invention;

[0039] Figure 3 Schematic diagram of the adaptive filtering module during training in the first embodiment of the present invention. DETAILED DESCRIPTION

[0040] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0041] The first embodiment of the present invention relates to a robust stereo matching method based on frequency domain adaptive filtering optimization, such as Figure 1 As shown, the following steps are included:

[0042] Step 1, obtaining a binocular image;

[0043] Step 2: Input the acquired binocular image into the stereo matching model to complete the pixel matching to obtain the disparity map.

[0044] like Figure 2 As shown, the stereo matching model in this embodiment includes:

[0045] Feature extraction network, used to extract the features of the binocular image and obtain the features of the left image and the right image;

[0046] An adaptive filtering module is used to convert the left image features and the right image features into the frequency domain to obtain the left image frequency domain features and the right image frequency domain features, and adaptively filter the left image features and the right image features based on the left image frequency domain features and the right image frequency domain features, and re-transform the filtered left image features and the right image features into the spatial domain to complete the cost volume construction;

[0047] A frequency compensation module, used for performing frequency compensation on the frequency domain features of the left image, and fusing the compensated frequency domain features of the left image with the aggregated features after aggregation of the cost volume to obtain a fused feature;

[0048] The disparity prediction module is used to perform disparity prediction according to the fusion features and complete pixel matching to obtain a disparity map.

[0049] When training a stereo matching model, first, a data enhancement strategy is used to add disturbances to the right image in the standard binocular image to simulate the scene image captured by the camera in the presence of a correction error. Subsequently, the left image and the perturbed right image are used as inputs to the stereo matching model, and after feature extraction, an adaptive filtering module is used to filter out the frequency components in the features that are susceptible to correction errors. In the adaptive filtering module, the image features are converted to the frequency domain for operation, the frequency domain feature mask obtained by adaptive network prediction is used to complete the filtering, and the filtered frequency domain features are retransformed to the spatial domain through the inverse fast Fourier transform to complete the subsequent matching cost volume construction. When 3D convolution is used for feature aggregation in the subsequent implementation, the frequency domain features of the left image are merged with the aggregated features using a frequency compensation module to restore some of the beneficial features lost in the filtering operation, thereby ensuring the matching accuracy without affecting the robustness of the model to correction errors.

[0050] In the stereo matching problem, the most important constraint is that the corresponding pixels in the left and right images are in the same row. When there is a correction error, this constraint is broken, resulting in the inability to search for corresponding matching pixels for some pixels. Through observation, we found that different structures in the image have different sensitivities to correction errors. For example, the horizontal structure in the image is more sensitive to correction errors, while the vertical structure is more robust. In addition, these structures correspond to different frequency components in the frequency domain features of the image. Therefore, the frequency components with poor robustness can be removed through frequency domain filtering, thereby reducing the impact of correction errors on matching.

[0051] According to the above prior knowledge, the adaptive filtering module designed in this embodiment includes: a frequency domain conversion part, which is used to convert the left image features and the right image features into the frequency domain, and splice the real part and the imaginary part of the converted left image features and the real part and the imaginary part of the right image features in the channel dimension to obtain the left image frequency domain features and the right image frequency domain features; a frequency domain mask generation part, which is used to generate a frequency domain mask according to the left image frequency domain features and the right image frequency domain features; a filtering part, which is used to filter the left image features and the right image features using the frequency domain mask; a spatial domain conversion part, which is used to convert the filtered left image features and the right image features into the spatial domain; a cost body generation part, which is used to generate a cost body according to the left image features and the right image features after conversion to the spatial domain. The adaptive filtering module learns the filter weights of specific frequency components under the constraint of feature consistency, thereby completing the removal of sensitive frequency components.

[0052] When training the stereo matching model, in order to obtain the frequency domain features of the image, the left image feature F l , right image feature F r And the perturbed right image features As input, its size is C×H×W, where C represents the number of feature channels, H represents the height of the feature, and W represents the width of the feature. First, the input features are transformed into frequency domain features using fast Fourier transform (FFT) to obtain the frequency domain features of the left image f l , right image frequency domain features f r And the frequency domain features of the right image after perturbation The FFT calculation process can be described as:

[0053]

[0054] In the formula, (u, v) represents the coordinates in the frequency domain space, and (h, w) represents the coordinates in the spatial domain. The real and imaginary parts of the features after FFT are concatenated in the channel dimension to finally obtain a frequency domain feature with a size of 2C×H×W.

[0055] In order to use the frequency domain features to adaptively obtain the filter weights, the frequency domain features of the left image f l And the frequency domain features of the right image after perturbation The input is sent to the frequency domain mask generation part, which first maps the frequency domain features to the embedding space through a convolution module including a BN layer and a ReLU activation function, and then obtains a frequency domain mask M with a size of 2C×H×W and a value range of [0,1] through a convolution layer and a Sigmoid activation function. The structure of the mask generation part is as follows Figure 3 As shown in . Then the frequency domain mask is used to transform the frequency domain features f of the left image l And the frequency domain features of the right image after perturbation Filter to get the filtered features and The calculation formula is as follows:

[0056]

[0057] In the formula, represents element-wise multiplication, Represents the frequency domain features after filtering. The value of the frequency domain mask M reflects the sensitivity of the frequency component at the corresponding position to the correction error. The closer the value is to 0, the more affected the component is by the correction error and should be filtered out. In order to perform subsequent matching in the spatial domain, the fast inverse Fourier transform IFFT is used to convert the filtered frequency domain features back to the spatial domain to obtain the features. and

[0058] In addition, in order to effectively supervise the mask generation module, this embodiment uses the same filtering method to obtain the frequency domain features of the right image after filtering from the right image features. And transform it into the spatial domain, we get After using the frequency domain mask to completely filter out the sensitive frequency components in the perturbed right image, the obtained features The right image features should be compared with the standard rectified It has consistency, thereby ensuring that the influence of the correction error on subsequent matching is completely eliminated. According to the above-mentioned feature consistency constraint, this embodiment constructs a feature consistency loss, which is used to force the frequency domain mask M output by the frequency domain mask generation module to eliminate the frequency components that are sensitive to the correction error, thereby reducing the image feature differences caused by the correction error. The specific calculation process of the feature consistency loss is shown in the following formula:

[0059]

[0060] Where cos(·) represents the cosine of the angle between the calculated vectors.

[0061] After using the adaptive filtering module to remove the frequency components that are sensitive to the correction error, the robustness of the model to the correction error is significantly enhanced. However, the above operation will inevitably lose some feature information that is beneficial to matching. When the impact of the correction error is smaller than the impact of the feature loss, the lost feature information will limit the matching accuracy of the model.

[0062] In response to the above problems, this embodiment constructs a frequency compensation module, which includes: a weight calculation part, which is used to calculate the frequency compensation weight according to the frequency domain mask during adaptive filtering; a guidance part, which is used to use the frequency compensation weight to guide the frequency domain features of the left image, and convert them to the spatial domain to obtain compensation features; a fusion part, which is used to splice the compensation features with the aggregated features after the cost body is aggregated in the feature dimension, and input them into the feature fusion network to obtain fusion features.

[0063] The frequency compensation module converts the frequency domain feature f of the left image before filtering. l The cost aggregation step is introduced into the stereo matching network and feature fusion is completed under the guidance of the frequency domain mask to restore the lost features.

[0064] Considering that the frequency domain mask M represents the degree of filtering of different frequency component features during the filtering operation, the smaller the mask value, the more thorough the filtering. Therefore, the probability of the frequency domain feature component corresponding to the mask being lost is greater. Based on the above relationship, this embodiment uses the frequency domain mask M to complete the frequency compensation weight The specific calculation formula is as follows:

[0065]

[0066] In the formula, represents a frequency compensation weight network based on an encoder-decoder structure, which is used to adaptively select the compensation frequency according to the frequency domain mask. σ(·) is a Sigmoid function, which is used to limit the network output to [0,1] to indicate the degree of feature filtering. Finally, 1-σ(·) is used to convert the network output into a frequency compensation weight. Used to filter frequency domain feature components for feature fusion.

[0067] In getting the frequency compensation weight After that, first use it to filter the left image frequency domain features f l Guided and transformed to the spatial domain through IFFT to obtain compensation features for fusion with cost aggregation features The calculation process is as follows:

[0068]

[0069] Then, the compensation feature The aggregated feature F obtained from the cost aggregation step of the stereo matching network agg After the feature dimensions are concatenated, they are input into the feature fusion subnetwork to obtain the fused feature F fusion The calculation process is as follows:

[0070]

[0071] In the formula, [·] represents the concatenation operation, and τ(·) represents the feature fusion network. The feature fusion network is used to fuse the compensation features and the aggregation features, which effectively supplements the feature loss problem caused by the pre-stage filtering. While not affecting the robustness of the network, it avoids the decrease in matching accuracy caused by feature loss.

[0072] It is not difficult to find that the present invention is based on the stereo matching algorithm framework based on deep learning. In view of the correction error in the input image, a frequency domain adaptive filtering module is designed. The module transforms the image features into the frequency domain through fast Fourier transform and uses a learnable filter to complete the filtering, which effectively suppresses the frequency components sensitive to the correction error and improves the robustness of the model to the correction error. In addition, considering that feature filtering will cause the loss of some beneficial frequency components to a certain extent, the present invention also constructs a frequency compensation module to supplement the key matching information lost due to feature filtering, so as to eliminate the influence of filtering on the accuracy of the matching network.

[0073] A second embodiment of the present invention relates to a robust stereo matching device based on frequency domain adaptive filtering optimization, comprising:

[0074] An acquisition module, used for acquiring binocular images;

[0075] The matching module is used to input the acquired binocular image into the stereo matching model to complete the pixel matching and obtain the disparity map;

[0076] Wherein, the stereo matching model includes:

[0077] Feature extraction network, used to extract the features of the binocular image and obtain the features of the left image and the right image;

[0078] An adaptive filtering module is used to convert the left image features and the right image features into the frequency domain to obtain the left image frequency domain features and the right image frequency domain features, and adaptively filter the left image features and the right image features based on the left image frequency domain features and the right image frequency domain features, and re-transform the filtered left image features and the right image features into the spatial domain to complete the cost volume construction;

[0079] A frequency compensation module, used for performing frequency compensation on the frequency domain features of the left image, and fusing the compensated frequency domain features of the left image with the aggregated features after aggregation of the cost volume to obtain a fused feature;

[0080] The disparity prediction module is used to perform disparity prediction according to the fusion features and complete pixel matching to obtain a disparity map.

[0081] The adaptive filtering module comprises:

[0082] The frequency domain conversion part is used to convert the left image features and the right image features into the frequency domain, and concatenate the real part and the imaginary part of the converted left image features and the real part and the imaginary part of the right image features in the channel dimension to obtain the left image frequency domain features and the right image frequency domain features;

[0083] The frequency domain mask generation part is used to generate a frequency domain mask according to the frequency domain features of the left image and the frequency domain features of the right image;

[0084] A filtering part, used for filtering the left image features and the right image features using the frequency domain mask;

[0085] A spatial domain conversion part, used for converting the filtered left image features and right image features into the spatial domain;

[0086] The cost volume generation part is used to generate the cost volume according to the left image features and the right image features after conversion into the spatial domain.

[0087] The frequency domain mask generation part first maps the frequency domain features into the embedding space through a convolution module including a BN layer and a ReLU activation function, and then obtains a frequency domain mask with a value range of [0,1] through a convolution layer and a Sigmoid activation function.

[0088] The loss function of the frequency domain mask generation part is: Among them, L con The loss function for the frequency domain mask generation part, H and W represent the height and width of the image features, The standard right image features are filtered by using the frequency domain mask during training and converted to the standard right image features in the spatial domain. During training, the perturbed standard right image is filtered using a frequency domain mask and converted to the perturbed right image features in the spatial domain.

[0089] The frequency compensation module comprises:

[0090] The weight calculation part is used to calculate the frequency compensation weight according to the frequency domain mask during adaptive filtering;

[0091] A guiding part, used to guide the frequency domain features of the left image using frequency compensation weights, and convert them into the spatial domain to obtain compensation features;

[0092] The fusion part is used to splice the compensation feature with the aggregated feature after the cost volume is aggregated in the feature dimension, and input it into the feature fusion network to obtain the fused feature.

[0093] The weight calculation part is Calculate the frequency compensation weights, where is the frequency compensation weight, M is the frequency domain mask during adaptive filtering, represents the frequency compensation weight network based on the encoder-decoder structure; σ(·) represents the Sigmoid function.

[0094] The third embodiment of the present invention relates to a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the robust stereo matching method based on frequency domain adaptive filtering optimization of the first embodiment described above are implemented. It should be understood by those skilled in the art that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program codes.

[0095] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0096] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction method, which is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0097] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0098] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.< / practicalstereomatchingviacascadedrecurrentnetworkwithadaptivecorrelation>

Claims

1. A robust stereo matching method based on frequency domain adaptive filtering optimization, characterized in that: The following steps are involved: Acquire binocular images; Input the acquired binocular image into the stereo matching model, complete the pixel matching and obtain the disparity map; Wherein, the stereo matching model includes: Feature extraction network, used to extract the features of the binocular image and obtain the features of the left image and the right image; An adaptive filtering module is used to convert the left image features and the right image features into the frequency domain to obtain the left image frequency domain features and the right image frequency domain features, and adaptively filter the left image features and the right image features based on the left image frequency domain features and the right image frequency domain features, and re-transform the filtered left image features and the right image features into the spatial domain to complete the cost volume construction; The adaptive filtering module comprises: The frequency domain conversion part is used to convert the left image features and the right image features into the frequency domain, and concatenate the real part and the imaginary part of the converted left image features and the real part and the imaginary part of the right image features in the channel dimension to obtain the left image frequency domain features and the right image frequency domain features; The frequency domain mask generation part is used to generate a frequency domain mask according to the frequency domain features of the left image and the frequency domain features of the right image; A filtering part, used for filtering the left image features and the right image features using the frequency domain mask; A spatial domain conversion part, used for converting the filtered left image features and right image features into the spatial domain; The cost volume generation part is used to generate the cost volume according to the left image features and the right image features after conversion to the spatial domain; A frequency compensation module, used for performing frequency compensation on the frequency domain features of the left image, and fusing the compensated frequency domain features of the left image with the aggregated features after aggregation of the cost volume to obtain a fused feature; The disparity prediction module is used to perform disparity prediction according to the fusion features and complete pixel matching to obtain a disparity map.

2. The robust stereo matching method based on frequency domain adaptive filtering optimization according to claim 1, characterized in that: The frequency domain mask generation part first maps the frequency domain features into the embedding space through a convolution module including a BN layer and a ReLU activation function, and then obtains a frequency domain mask with a value range of [0,1] through a convolution layer and a Sigmoid activation function.

3. The robust stereo matching method based on frequency domain adaptive filtering optimization according to claim 1, characterized in that: The loss function of the frequency domain mask generation part is: Among them, L con The loss function for the frequency domain mask generation part, H and W represent the height and width of the image features, The standard right image features are filtered by using the frequency domain mask during training and converted to the standard right image features in the spatial domain. During training, the perturbed standard right image is filtered using a frequency domain mask and converted to the perturbed right image features in the spatial domain.

4. The robust stereo matching method based on frequency domain adaptive filtering optimization according to claim 1, characterized in that: The frequency compensation module comprises: The weight calculation part is used to calculate the frequency compensation weight according to the frequency domain mask during adaptive filtering; A guiding part, used to guide the frequency domain features of the left image using frequency compensation weights, and convert them into the spatial domain to obtain compensation features; The fusion part is used to splice the compensation feature with the aggregated feature after the cost volume is aggregated in the feature dimension, and input it into the feature fusion network to obtain the fused feature.

5. The robust stereo matching method based on frequency domain adaptive filtering optimization according to claim 4, characterized in that: The weight calculation part is Calculate the frequency compensation weights, where, is the frequency compensation weight, M is the frequency domain mask during adaptive filtering, represents the frequency compensation weight network based on the encoder-decoder structure; σ(·) represents the Sigmoid function.

6. A robust stereo matching device based on frequency domain adaptive filtering optimization, characterized in that: include: An acquisition module, used for acquiring binocular images; The matching module is used to input the acquired binocular image into the stereo matching model to complete the pixel matching and obtain the disparity map; Wherein, the stereo matching model includes: Feature extraction network, used to extract the features of the binocular image and obtain the features of the left image and the right image; An adaptive filtering module is used to convert the left image features and the right image features into the frequency domain to obtain the left image frequency domain features and the right image frequency domain features, and adaptively filter the left image features and the right image features based on the left image frequency domain features and the right image frequency domain features, and re-transform the filtered left image features and the right image features into the spatial domain to complete the cost volume construction; The adaptive filtering module comprises: The frequency domain conversion part is used to convert the left image features and the right image features into the frequency domain, and concatenate the real part and the imaginary part of the converted left image features and the real part and the imaginary part of the right image features in the channel dimension to obtain the left image frequency domain features and the right image frequency domain features; The frequency domain mask generation part is used to generate a frequency domain mask according to the frequency domain features of the left image and the frequency domain features of the right image; A filtering part, used for filtering the left image features and the right image features using the frequency domain mask; A spatial domain conversion part, used for converting the filtered left image features and right image features into the spatial domain; The cost volume generation part is used to generate the cost volume according to the left image features and the right image features after conversion to the spatial domain; A frequency compensation module, used for performing frequency compensation on the frequency domain features of the left image, and fusing the compensated frequency domain features of the left image with the aggregated features after aggregation of the cost volume to obtain a fused feature; The disparity prediction module is used to perform disparity prediction according to the fusion features and complete pixel matching to obtain a disparity map.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the robust stereo matching method based on frequency domain adaptive filtering optimization as claimed in any one of claims 1 to 5 are implemented.

Images