High robustness stereo matching method based on feature pyramid and attention perception
By employing feature pyramid and attention-based methods, the disparity map blurring problem in stereo matching algorithms under complex scenes is solved, achieving efficient and detailed disparity map generation and improving the robustness and accuracy of stereo matching.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING INST OF REMOTE SENSING EQUIP
- Filing Date
- 2023-07-14
- Publication Date
- 2026-07-07
AI Technical Summary
Existing stereo matching algorithms struggle to generate accurate disparity maps when dealing with textureless regions, repeating patterns, thin structures, occlusion, or reflections, resulting in blurred matching and edges, as well as low computational efficiency and high memory consumption.
We employ a feature pyramid and attention-aware approach. By setting a feature extraction pyramid, we obtain multi-scale feature maps, construct a multi-scale cost volume pyramid, and implement feedback interaction for cross-scale cost aggregation between different scales. We use softmax and soft-argmin functions to generate an initial disparity map and design a saliency-aware attention module to fuse feature maps to refine the disparity map.
It effectively corrects erroneous matching points, restores lost image details and sharp edges, improves the robustness and precision of disparity prediction, and generates detailed and accurate disparity maps.
Smart Images
Figure CN117078978B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of binocular stereo matching technology, and in particular to a highly robust stereo matching method based on feature pyramids and attention perception. Background Technology
[0002] Stereo matching aims to establish correspondences in stereo images. It plays a crucial role in depth estimation and is central to many 3D vision applications, including autonomous driving, robot navigation, augmented reality, and industrial manufacturing.
[0003] In recent times, deep convolutional neural networks have achieved remarkable results in a number of challenging real-world applications. Nevertheless, state-of-the-art network architectures still struggle to handle matching ambiguity caused by textureless regions, repeating patterns, thin structures, occlusion, or reflections, which can lead to incorrect parallax predictions.
[0004] Existing algorithms typically rely on regularization to address matching ambiguity. Using feature concatenation and 3D-CNN (Convolutional Neural Networks)-based regularization to generate cost volumes can significantly improve matching quality, but this also suffers from drawbacks such as slow computation and high memory consumption. To improve efficiency, recent algorithms employ a coarse-to-fine stereo matching framework, fusing feature information from multi-scale cost volumes during the matching process. However, due to the lack of high-frequency detail information during upsampling and feature fusion, this type of framework blurs salient edges in the disparity map, which can affect the final prediction results.
[0005] Despite recent progress in 3D cost volume regularization, we observe that existing algorithms still suffer from corresponding point matching errors and blurred image edges in the disparity maps they generate. Since obtaining a completely accurate and unambiguous matching cost volume is generally impossible, an effective back-end thinning scheme is needed to further improve matching accuracy. Summary of the Invention
[0006] To address the aforementioned shortcomings in existing technologies, this invention provides a highly robust stereo matching method based on feature pyramids and attention perception.
[0007] This invention also provides a robust stereo matching method based on feature pyramids and attention perception, comprising:
[0008] A feature extraction pyramid is set up to process the input stereo image and obtain multi-scale feature maps of the left and right view input images respectively.
[0009] A multi-scale cost volume pyramid is constructed using multi-scale feature maps, and feedback interaction is implemented between cost volumes at different scales to aggregate cross-scale costs.
[0010] The cost volume pyramid is converted into a probability value pyramid using the softmax function, and the probability value pyramid is processed using the soft-argmin function to generate an initial disparity map;
[0011] A saliency attention perception module is designed to extract and generate an attention feature map in the shallow layer of the stereo matching network. This feature map is then fused with the initial disparity map to obtain a refined disparity map.
[0012] In some embodiments, a feature extraction pyramid is set to process the input binocular stereo image, and multi-scale feature maps of the left and right view input images are obtained respectively, including:
[0013] The feature extraction pyramid uses two weight-shared residual convolutional modules to process the input stereo image with a resolution of H×W, and uses average pooling to generate feature maps at three scales. The stereo image includes a left-view image and a right-view image.
[0014] In some embodiments, a multi-scale cost volume pyramid is constructed using multi-scale feature maps, including:
[0015] The multi-scale feature maps generated from the left and right input images are connected according to the disparity level, and then a cost volume pyramid containing three scales is constructed, where the disparity level ranges from 0 to 192.
[0016] In some embodiments, feedback-interactive cross-scale cost aggregation is implemented between cost volumes of different scales, including:
[0017] The minimum cost volume is fed into a 3D-CNNEncoder-decoder structure, and contextual information is aggregated along the spatial dimension H×W and the disparity hypothesis plane dimension D.
[0018] Upsample the cost volume at the current scale to the same dimension as the cost volume pyramid at the next scale level, connect it to the initial cost volume at the next scale level, and repeat the Encoder-decoder aggregation operation.
[0019] The above operations are performed iteratively until the initial cost volumes at the largest scale are aggregated; wherein, the proposed feedback interaction cross-scale cost aggregation is expressed by the following formula:
[0020]
[0021] In formula (1) and C represents the aggregation cost volume at the current level and the previous scale level, respectively. i Let μ be the initial cost volume at the current scale level, where i represents the scale level of the cost volume pyramid, i∈[1,A], and μ is the initial cost volume. T It is a trilinear interpolation operation, where Γ represents the 3D-CNN Encoder-decoder aggregation operation. It's a cascading operation, in the formula. C0 = 0.
[0022] In some embodiments, the cost volume pyramid is converted into a probability value pyramid using a softmax function, including:
[0023] For each scale level of the aggregated cost volume pyramid, the softmax function is used to transform the cost volume pyramid into a probability value pyramid. The specific operation definition of the softmax function is as follows:
[0024]
[0025] In the formula, j represents all scale levels of the cost volume pyramid, P(c i The transformed probability value pyramid.
[0026] In some embodiments, the probability pyramid is processed using the soft-argmin function to generate an initial disparity map, including:
[0027] Soft-argmin is applied to each level of the obtained probability value pyramid to regress the disparity value. The formula for this disparity regression operation is as follows:
[0028]
[0029] In equation (3), D pred P(c) represents the disparity prediction value. d ) represents the softmax operation that maps cost volumes to probability values, d represents the disparity hypothesis plane, and D max c represents the maximum predicted disparity. d It is the aggregation matching cost under the disparity plane d, which is a bilinear interpolation upsampling of the regressed disparity map pyramid.
[0030] In some embodiments, a saliency attention perception module is designed to extract and generate an attention feature map in the shallow layer of the stereo matching network, and then fuse this feature map with the initial disparity map to obtain a refined disparity map, including:
[0031] Spatial saliency feature map O is generated using the spatial dimension saliency branch. s ;
[0032] Generate channel saliency feature maps using channel dimension saliency branches. c ;
[0033] The obtained spatial dimension saliency feature map O s Channel dimension saliency feature map O c The original input feature map is fused to generate a salient attention feature map, and this salient attention feature map is then fused with the acquired initial disparity map to obtain a refined disparity map.
[0034] In some embodiments, the spatial saliency feature map O s The mathematical expression for is defined as follows:
[0035]
[0036] In equation (4), ξ e X represents the dimension expansion operation. in Indicates input features, and P represents different shape reshaping operations performed in the C, H, and W dimensions. avg F represents average pooling. s1 and F s2 These represent convolutional layers of size 1×1. This represents matrix multiplication, and P represents the softmax function operation.
[0037] In some embodiments, the channel saliency feature map O c Its mathematical expression is defined as follows:
[0038]
[0039] In equation (4), ξ e This indicates a dimension expansion operation. and F represents different shape reshaping operations performed in the C, H, and W dimensions. c1 and F c2 These represent convolutional layers of size 1×1. This represents matrix multiplication, and P represents the softmax function operation.
[0040] In some embodiments, the acquired spatial dimension saliency feature map O s Channel dimension saliency feature map O c The original input feature map is fused to generate a salient attention feature map, which is then fused with the acquired initial disparity map to obtain a refined disparity map, including:
[0041] Element-wise summation is used to fuse spatial dimensional saliency feature maps O sand channel dimension saliency feature map O c ;
[0042] A 3×3 convolutional layer is used to process the fused output and further fuse it with the original input feature map to generate a saliency attention feature map.
[0043] Bilinear interpolation is used to upsample the saliency attention feature map;
[0044] The feature map is fused with the initial disparity map by element-wise summation to generate a refined disparity map.
[0045] This invention provides a highly robust stereo matching method based on multi-scale cost volume pyramid and saliency attention perception refinement, which can effectively correct erroneous matching points, restore lost image details and sharp object edges, improve the robustness and precision of disparity prediction, and generate a fine and accurate disparity map. Attached Figure Description
[0046] Figure 1 A flowchart illustrating a highly robust stereo matching method based on feature pyramids and attention perception, provided in an embodiment of the present invention;
[0047] Figure 2 This is a network structure diagram of a highly robust stereo matching method based on feature pyramids and attention perception provided in an embodiment of the present invention;
[0048] Figure 3 This is a schematic diagram of the saliency attention perception module provided in an embodiment of the present invention;
[0049] Figure 4 The visualization effect of the saliency attention feature map provided in the embodiments of the present invention;
[0050] Figure 5 This is a schematic diagram comparing the initial parallax and the refined parallax provided in an embodiment of the present invention;
[0051] Figure 6 This is a schematic diagram of test results on the KITTI 2012 dataset provided in an embodiment of the present invention;
[0052] Figure 7 A schematic diagram of test results on the KITTI 2015 dataset provided in an embodiment of the present invention;
[0053] Figure 8 A ranking diagram in the KITTI 2012 global public benchmark provided for embodiments of the present invention (as of June 5, 2022). Detailed Implementation
[0054] Exemplary embodiments will be described more fully below with reference to the accompanying drawings; however, these exemplary embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and that those skilled in the art will fully understand the scope of the invention.
[0055] Where there is no conflict, the embodiments of the present invention and the features thereof can be combined with each other.
[0056] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.
[0057] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a” and “the” used herein are also intended to include the plural forms unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated features, integrals, steps, operations, elements, and / or components is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded.
[0058] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be understood to have the meaning consistent with their meaning in the context of the relevant art and the invention, and will not be understood to have an idealized or overly formal meaning unless expressly so defined herein.
[0059] To enable those skilled in the art to better understand the technical solution, the following describes in detail, with reference to the accompanying drawings, a highly robust stereo matching method based on feature pyramids and attention perception provided by the present invention.
[0060] like Figure 1 , Figure 2 As shown, this embodiment of the invention provides a highly robust stereo matching method based on feature pyramids and attention perception, including:
[0061] Step S1: Set up a feature extraction pyramid to process the input binocular stereo image and obtain multi-scale feature maps of the left and right view input images respectively.
[0062] Step S2: Construct a multi-scale cost volume pyramid using multi-scale feature maps, and implement feedback interaction for cross-scale cost aggregation between cost volumes at different scales;
[0063] Step S3: The cost volume pyramid is converted into a probability value pyramid using the softmax function, and the probability value pyramid is processed using the soft-argmin function to generate an initial disparity map.
[0064] Step S4: Design a saliency attention perception module to extract and generate an attention feature map in the shallow layer of the stereo matching network, and fuse the feature map with the initial disparity map to obtain a refined disparity map.
[0065] This invention provides a highly robust stereo matching method based on multi-scale cost volume pyramid and saliency attention perception refinement, which can effectively correct erroneous matching points, restore lost image details and sharp object edges, improve the robustness and precision of disparity prediction, and generate a fine and accurate disparity map.
[0066] In some embodiments, a feature extraction pyramid is set to process the input binocular stereo image, and multi-scale feature maps of the left and right view input images are obtained respectively, including:
[0067] The feature extraction pyramid uses two weight-shared residual convolutional modules to process a stereo image with an input resolution of H×W. The stereo image includes a left-view image and a right-view image, and average pooling is used to generate feature maps at three scales.
[0068] In a preferred embodiment, the feature extraction pyramid uses two weight-shared residual convolutional modules to process an input stereo image with a resolution of H×W, and uses average pooling to generate feature maps at three scales, with each scale having a resolution of [missing information].
[0069] In some embodiments, step S2 includes S2.1, constructing a multi-scale cost volume pyramid using multi-scale feature maps, including:
[0070] The multi-scale feature maps generated from the left and right input images are connected according to the disparity level, which ranges from 0 to 192, and then a cost volume pyramid containing three scales is constructed.
[0071] In a preferred embodiment, the multi-scale feature maps generated from the left and right input images are concatenated according to disparity levels to construct a multi-scale cost volume pyramid, where each level of the cost volume pyramid has a size of [missing information]. and Where H and W represent the height and width in the spatial dimension, and D represents the parallax hypothesis plane.
[0072] In some embodiments, step S2 includes S2.2, implementing feedback interaction cross-scale cost aggregation between cost volumes of different scales, including:
[0073] The minimum cost volume is fed into a 3D-CNNEncoder-decoder structure, and contextual information is aggregated along the spatial dimension H×W and the disparity hypothesis plane dimension D.
[0074] Upsample the cost volume at the current scale to the same dimension as the cost volume pyramid at the next scale level, connect it to the initial cost volume at the next scale level, and repeat the Encoder-decoder aggregation operation.
[0075] The above operations are performed iteratively until the initial cost volumes at the largest scale are aggregated; wherein, the proposed feedback interaction cross-scale cost aggregation is expressed by the following formula:
[0076]
[0077] In formula (1) and C represents the aggregation cost volume at the current level and the previous scale level, respectively. i Let μ be the initial cost volume at the current scale level, where i represents the scale level of the cost volume pyramid, i∈[1,A], and μ is the initial cost volume. T It is a trilinear interpolation operation, where Γ represents the 3D-CNN Encoder-decoder aggregation operation. It's a cascading operation, in the formula. C0 = 0.
[0078] This invention designs a feedback-interactive cross-scale cost aggregation scheme to allow effective multi-scale cost volume interaction aggregation, generating a cost volume pyramid with rich semantic information.
[0079] In some embodiments, step S3 includes S3.1, converting the cost volume pyramid into a probability value pyramid using a softmax function, including:
[0080] For each scale level of the aggregated cost volume pyramid, the softmax function is used to transform the cost volume pyramid into a probability value pyramid. The specific operation definition of the softmax function is as follows:
[0081]
[0082] In the formula, j represents all scale levels of the cost volume pyramid, P(c i The transformed probability value pyramid.
[0083] In some embodiments, step S3 includes S3.2, processing the probability value pyramid using the soft-argmin function to generate an initial disparity map, including:
[0084] Soft-argmin is applied to each level of the obtained probability value pyramid to regress the disparity value. The formula for this disparity regression operation is as follows:
[0085]
[0086] In equation (3), D pred P(c) represents the disparity prediction value. d ) represents the softmax operation that maps cost volumes to probability values, d represents the disparity hypothesis plane, and D max c represents the maximum predicted disparity. d It is the aggregation matching cost under the disparity plane d, which is a bilinear interpolation upsampling of the regressed disparity map pyramid.
[0087] In this embodiment of the invention, in order to obtain a disparity map with the same resolution as the input image, bilinear interpolation upsampling is performed on the regressed disparity map pyramid.
[0088] In some embodiments, a saliency attention perception module is designed to extract and generate an attention feature map in the shallow layer of the stereo matching network, and then fuse this feature map with the initial disparity map to obtain a refined disparity map, including:
[0089] S4.1, Use the spatial dimension saliency branch to generate the spatial saliency feature map O s ;
[0090] S4.2, use the channel dimension saliency branch to generate the channel saliency feature map O c ;
[0091] S4.3, the obtained spatial dimension saliency feature map O s Channel dimension saliency feature map O c The original input feature map is fused to generate a salient attention feature map, and this salient attention feature map is then fused with the acquired initial disparity map to obtain a refined disparity map.
[0092] The saliency attention perception module, such as Figure 3 As shown, it includes spatial dimension saliency branches and channel dimension saliency branches. The visualization effect of the saliency attention feature map is as follows. Figure 4 As shown.
[0093] In some embodiments, specifically in this embodiment:
[0094] In step 4.2, the spatial dimension saliency branch processes the input feature map F∈R using two parallel stacked 1×1 convolutional layers. C×H×W This generates two new feature maps F. s1 ∈R C×H×W and Fs2 ∈R C×H×W Next, F is further processed using average pooling, which serves as the channel descriptor. s1 Then F s2 Reshape to R C×HW The softmax function is used to compute the value in the last dimension to encode the importance of each pixel in the feature map of each channel. For F after average pooling... s1 Reshape it into R 1×C This is compared with F after the softmax function. s2 Perform matrix multiplication to generate a spatial saliency feature map.
[0095] Spatial saliency feature map O s The mathematical expression for is defined as follows:
[0096]
[0097] In equation (4), ξ e This indicates a dimension expansion operation. and P represents different shape reshaping operations performed in the C, H, and W dimensions. avg F represents average pooling. s1 and F s2 These represent convolutional layers of size 1×1. This represents matrix multiplication, and P represents the softmax function operation.
[0098] In some embodiments, the channel saliency feature map O c Its mathematical expression is defined as follows:
[0099]
[0100] In equation (4), ξ e X represents the dimension expansion operation. in Indicates input features, and F represents different shape reshaping operations performed in the C, H, and W dimensions. c1 and F c2 These represent convolutional layers of size 1×1. This represents matrix multiplication, and P represents the softmax function operation.
[0101] In some embodiments, step 4.3 involves obtaining the spatial dimension saliency feature map O. s Channel dimension saliency feature map O cThe original input feature map is fused to generate a salient attention feature map, which is then fused with the acquired initial disparity map to obtain a refined disparity map, including:
[0102] Element-wise summation is used to fuse spatial dimensional saliency feature maps O. s and channel dimension saliency feature map O c ;
[0103] A 3×3 convolutional layer is used to process the fused output and further fuse it with the original input feature map to generate a saliency attention feature map.
[0104] Bilinear interpolation is used to upsample the saliency attention feature map;
[0105] The feature map is fused with the initial disparity map by element-wise summation to generate a refined disparity map.
[0106] In this invention, an attention-aware disparity refinement scheme is introduced. This scheme utilizes high-frequency spatial detail information obtained from a depth saliency attention detection mechanism to guide the refinement of an initially obtained coarse disparity map. This refinement scheme can correct erroneous matching points, recover lost image details and sharp object edges, and provide a fine, robust disparity map.
[0107] The prediction results obtained by the highly robust stereo matching method provided by this invention are as follows: Figure 5 , Figure 6 and Figure 7 As shown, the present invention can effectively correct erroneous matching points, restore lost image details and sharp object edges, and provide a fine, robust disparity map.
[0108] This invention exhibits strong generalization ability and high transferability. Employing an end-to-end network architecture, it can learn generalization features from datasets across multiple scenarios and transfer these features for application, generating high-quality prediction results. For example... Figure 8 As shown, this invention ranked seventh in the KITTI 2012 global public benchmark as of June 5, 2022.
[0109] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for general illustrative purposes only and should not be construed as limiting. In some embodiments, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of the invention as set forth by the appended claims.
Claims
1. A highly robust stereo matching method based on feature pyramids and attention perception, characterized in that, include: A feature extraction pyramid is set up to process the input stereo image and obtain multi-scale feature maps of the left and right view input images respectively. A multi-scale cost volume pyramid is constructed using multi-scale feature maps, and feedback interaction is implemented between cost volumes at different scales to aggregate cross-scale costs. The cost volume pyramid is converted into a probability value pyramid using the softmax function, and the probability value pyramid is processed using the soft-argmin function to generate an initial disparity map; Design a saliency attention perception module to extract and generate attention feature maps in the shallow layer of the stereo matching network, and fuse these feature maps with the initial disparity map to obtain a refined disparity map; A saliency attention perception module is designed to extract and generate attention feature maps in the shallow layers of the stereo matching network. These feature maps are then fused with the initial disparity map to obtain a refined disparity map, including: Spatial saliency feature map is generated using the spatial dimension saliency branch. ; Generate channel saliency feature maps using channel dimension saliency branches. ; The obtained spatial dimension saliency feature map Channel dimension saliency feature map The salient attention feature map is generated by fusing the multi-scale feature map with the original disparity map, and then the salient attention feature map is fused with the original disparity map to obtain a refined disparity map. Spatial saliency feature map The mathematical expression for is defined as follows: （4） In formula (4) This indicates a dimension expansion operation. Indicates input features, and Indicates in Different shape reshaping operations performed in different dimensions Indicates average pooling. and They represent sizes of Convolutional layers, Represents matrix multiplication. This represents the softmax function operation; Channel saliency feature map Its mathematical expression is defined as follows: （5） In formula (4) This indicates a dimension expansion operation. and Indicates in Different shape reshaping operations performed in different dimensions and They represent sizes of Convolutional layers, Represents matrix multiplication. This represents the softmax function operation; The obtained spatial dimension saliency feature map Channel dimension saliency feature map The salient attention feature map is generated by fusing with the multi-scale feature map, and then fused with the acquired initial disparity map to obtain a refined disparity map, including: Element-wise summation is used to fuse spatial dimension saliency feature maps. and channel dimension saliency feature map ; A 3×3 convolutional layer is used to process the fused output and further fuse it with multi-scale feature maps to generate a salient attention feature map. Bilinear interpolation is used to upsample the saliency attention feature map; The feature map is fused with the initial disparity map by element-wise summation to generate a refined disparity map.
2. The robust stereo matching method based on feature pyramids and attention perception according to claim 1, characterized in that, A feature extraction pyramid is set up to process the input stereo image, and multi-scale feature maps of the left and right view input images are obtained respectively, including: The feature extraction pyramid uses two weight-shared residual convolutional modules to process the input stereo image with a resolution of H×W, and uses average pooling to generate feature maps at three scales. The stereo image includes a left-view image and a right-view image.
3. The robust stereo matching method based on feature pyramids and attention perception according to claim 1, characterized in that, Constructing a multi-scale cost volume pyramid using multi-scale feature maps, including: The multi-scale feature maps generated from the left and right input images are connected according to the disparity level, and then a cost volume pyramid containing three scales is constructed, where the disparity level ranges from 0 to 192.
4. The robust stereo matching method based on feature pyramids and attention perception according to claim 1, characterized in that, Implementing feedback-based cross-scale cost aggregation between cost volumes of different scales includes: The minimum cost volume is fed into a 3D-CNN convolutional neural network encoder-decoder structure, along the spatial dimension. And the plane dimension of the parallax assumption Aggregate contextual information; Upsample the cost volume at the current scale to the same dimension as the cost volume pyramid at the next scale level, connect it to the initial cost volume at the next scale level, and repeat the Encoder-decoder aggregation operation. The above operations are performed iteratively until the initial cost volumes at the largest scale are aggregated; wherein, the proposed feedback interaction cross-scale cost aggregation is expressed by the following formula: （1） In formula (1) and These represent the aggregate cost volumes at the current level and the previous scale level, respectively. Let i represent the initial cost volume at the current scale level, and let i denote the scale level of the cost volume pyramid. [1,A], It is a trilinear interpolation operation. Represents the 3D-CNN Encoder-decoder aggregation operation. It's a cascading operation, in the formula. .
5. The robust stereo matching method based on feature pyramids and attention perception according to claim 4, characterized in that, The cost volume pyramid is transformed into a probability value pyramid using the softmax function, including: For each scale level of the aggregated cost volume pyramid, the softmax function is used to transform the cost volume pyramid into a probability value pyramid. The specific operation definition of the softmax function is as follows: （2） In the formula, j represents all scale levels of the cost volume pyramid. This represents the pyramid of transformed probability values.
6. The robust stereo matching method based on feature pyramids and attention perception according to claim 1, characterized in that, The probability pyramid is processed using the soft-argmin function to generate an initial disparity map, including: Soft-argmin is applied to each level of the obtained probability value pyramid to regress the disparity value. The formula for this disparity regression operation is as follows: （3） In formula (3) This represents the disparity prediction value. This represents the softmax operation that maps cost volumes to probability values. Represents the parallax hypothesis plane. This represents the maximum value of the predicted disparity. It is the parallax plane The aggregation matching cost is used to perform bilinear interpolation upsampling on the disparity map pyramid of the regression.