Progressive parallax optimization method based on context geometric perception

Abstract

The invention relates to a progressive stereo parallax optimization method based on context geometric perception, and belongs to the field of computer vision and photogrammetry. The method comprises the following steps: acquiring initial images of left and right views, preprocessing the images, and extracting image features; generating an initial cost volume according to the splicing cost volume and the grouping related cost volume; carrying out cost aggregation by adopting a plurality of hourglasses, and carrying out convolution processing on different-scale cost volumes of the last two hourglasses; performing parallax regression prediction according to the output content; spatial parallax iterative optimization is carried out through the SAWG module and the SPON module, and matching is completed. According to the method, multi-scale cost volume information is continuously fused during stereo matching, iterative optimization is carried out on the initial disparity map, and finally the problems of edge expansion and texture missing are solved.

Description

Technical Field

[0001] The invention belongs to the field of computer vision and photogrammetry, and relates to a progressive parallax optimization method based on contextual geometry perception. Background Art

[0002] With the continuous development of computer technology, two-dimensional spatial data represented by maps and images can no longer meet people's cognitive needs for complex geographic space. Therefore, accurate and efficient acquisition of three-dimensional information has become a hot research issue in the field of photogrammetry and computer vision. At present, the main methods for obtaining three-dimensional data can be divided into two categories: active sensing and passive sensing. The active sensing method obtains three-dimensional data through LiDAR. Although it has high accuracy, it has the problems of high cost and sparse data. In contrast, passive sensing methods such as stereo matching can obtain high-precision three-dimensional data through matching calculations between images, and have the significant advantage of low cost. As the core of key technologies such as three-dimensional reconstruction, stereo navigation and non-contact ranging, stereo matching is widely used in smart cities, autonomous driving, digital twins, natural disaster monitoring, cultural heritage protection, lunar exploration projects and robot intelligent control.

[0003] At present, stereo matching algorithms are mainly divided into traditional methods and deep learning methods. Traditional stereo matching methods mainly include matching cost calculation, cost aggregation, disparity calculation, and disparity post-processing. Representative methods include SGM and PatchMatch. SGM combines local information with global constraints, uses path accumulation to perform cost aggregation, and gradually finds the optimal disparity estimate; PatchMatch uses random initialization and propagation strategies to quickly search for similar local areas in the image through iterative optimization, and gradually approaches the optimal matching result. Although these methods have low requirements for computing resources and strong logic, they mainly rely on manually designed cost functions and have poor robustness in complex scenes such as lighting changes and texture loss.

[0004] The deep learning-based methods inherit the theoretical basis of traditional methods, including feature extraction, cost volume construction, cost aggregation, disparity regression and other steps. GC-Net is the first end-to-end deep learning stereo matching network, which realizes cost aggregation through 3D convolution; IGEV-Stereo constructs a combined geometric encoder that encodes geometric and contextual information as well as local matching details, and iteratively indexes them to update the disparity map. These methods significantly improve the accuracy of disparity estimation, but there are still some shortcomings: the model usually performs cost aggregation at 1 / 4 scale and then interpolates to the full scale for disparity regression, which easily leads to edge expansion and texture information loss in the prediction results. In addition, many methods use stacked hourglass modules to aggregate cost volume information, and fuse the encoding and decoding cost volumes in a splicing or residual connection manner, which cannot fully utilize the fused cost volume information. This limits the matching accuracy of the model in weak texture areas and reduces its robustness in scenes with a large disparity range. Therefore, how to effectively solve the edge expansion and texture loss of the disparity map and improve the robustness of the model has become a key challenge in obtaining 3D geometric information in urban scenes. Summary of the invention

[0005] In order to solve the problems of edge expansion and texture loss in the existing methods of predicting disparity maps, the present invention proposes a progressive disparity optimization method based on contextual geometry perception to improve the prediction accuracy of the model in edge areas and texture areas and the overall robustness.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A progressive disparity optimization method based on context geometry perception, the method comprising the following steps:

[0008] S1, obtaining the initial images of the left and right views, preprocessing the images, and extracting image features;

[0009] S2. Constructing a splicing cost volume and a grouping related cost volume according to the splicing features and the correlation features to generate an initial cost volume;

[0010] S3, use three hourglasses for cost aggregation, and convolve the different scale cost volumes of the last two hourglasses into a single channel, and then interpolate to the full scale as the input of the disparity optimization stage;

[0011] S4, interpolate the cost volume output by the last hourglass to the full scale, and then perform disparity regression to predict the initial disparity map;

[0012] S5. Perform disparity iterative optimization through the SAWG module and the SPON module. In the SAWG module, the spatial attention weight is used to guide the cost volume of each scale, and in the SPON module, the cost volume information, the spatial consistency of the RGB image and the initial disparity map are used to update the disparity map to complete the matching.

[0013] Further, in step S1, the preprocessing includes: randomly cropping the input images of the left and right views into RGB formats of a preset size and performing standardization processing;

[0014] Then the features are extracted through the resnet network, where the feature extraction process includes:

[0015] The input image is first downsampled to extract 32-channel feature information at a scale of 1 / 2; then, after further feature extraction through two convolutional layers and three residual blocks, it is downsampled again and, after 16 residual blocks, 64-channel feature information l1 at a scale of 1 / 4 is obtained; then, l1 is extracted into 128-channel feature information l2 through three residual blocks, and further 128-channel feature information l3 is obtained through another three residual blocks; finally, l1, l2, and l3 are concatenated in the channel dimension to obtain 320-channel feature information l4, which is used to construct the correlation cost body; at the same time, l4 is convolved to extract 12 channels of feature information for constructing the concatenated cost body.

[0016] Further, in step S2, the features are convolved to 12 channels to construct the concatenated cost volume, which is expressed as:

[0017] C concat (d,x,y,·)=Concat{f l (x,y),f r (xd,y)}

[0018] In the formula, C concat (d,x,y,·) is the stitching cost volume, d,x,y are disparity, pixel horizontal coordinate, pixel vertical coordinate, "·" represents a placeholder, f l (·) represents the pixel feature vector of the left image, f r (·) represents the pixel feature vector of the right image, and Concat{·} represents the concatenation method;

[0019] The channels are divided into several groups, and the correlation cost volume is constructed for each group, which is expressed as:

[0020]

[0021] In the formula, C gwc (d,x,y,g) is the relevant cost volume, g is the index of the feature channel grouping, N c,N g are the total number of feature channels and the number of groups, respectively. are the g-th group of left image eigenvectors and the g-th group of right image eigenvectors, respectively, <·> represents the vector inner product;

[0022] The concatenated cost volume and the group-related cost volume are concatenated and fused as the input of the cost aggregation.

[0023] Furthermore, in step S3, multiple 3D hourglasses are used to perform cost volume aggregation, and in each 3D hourglass, it includes:

[0024] Encoding stage: First, the 1 / 4 scale cost volume V4 of 32 channels is downsampled to obtain the 1 / 8 scale cost volume of 64 channels, and then a 3D convolution is performed to obtain V8, which is then downsampled again and a 3D convolution is performed to obtain the 1 / 16 scale cost volume V of 128 channels. 16 ;

[0025] Decoding stage: cost volume V 16 Perform 3D deconvolution to get V′8, add V′8 and V8 after 1*1*1 convolution kernel, and then activate the result with relu function to get V″8; V″8 gets V″4 in the same way; V″4 is used as the input of subsequent hourglass;

[0026] Take the 1 / 16, 1 / 8, and 1 / 4 scale cost volumes of the last two hourglasses, convolve them into a single channel, and interpolate them to the full scale for input of disparity optimization:

[0027]

[0028] In the formula, res represents the scale, V res Represents the cost volume V of the res scale res ∈R (D×res)×(H×res)×(W×res) , f conv (.) represents 3D convolution, Interp(.) represents trilinear interpolation, Represents the full-scale cost volume obtained by interpolating the res-scale cost volume

[0029] Further, in step S4, for each output module, two 3D convolutions are used to generate a 1-channel cost volume and upsample it; then, a softmax function is applied on the disparity dimension to convert the cost volume into a probability volume, and finally, the predicted disparity of each pixel is calculated by weighted summation, as follows:

[0030]

[0031] In the formula, is the predicted disparity value, D maxis the maximum disparity, k is the discrete disparity candidate value, p k Represents the probability value for k.

[0032] Furthermore, in the SAWG module of step S5, a lightweight U-net network is used to extract fine features from the RGB image, and then the cost volume is activated by generating spatial attention weights through the Sigmiod activation function, and the weights are generated and acted on the cost volume. The process is expressed as follows:

[0033] V'(d,x,y)=V(d,x,y)⊙(σ(f U-Net (I)))

[0034] Where V(.) is the cost volume V(.)∈R D×H×W , I is the RGB image I∈R 3×H×W , f U-Net (.) is a lightweight U-Net network, ⊙ represents element-by-element multiplication, and σ(.) is the sigmoid activation function.

[0035] Further, in the SPON module in step S5, the following operations are included:

[0036] S511, expanding the RGB image and the initial disparity map to the C channel and then splicing and fusing them, then interpolating the full-scale cost volume to the C channel through convolution, and splicing and fusing the two again;

[0037] S512, then, a U-Net network is used to learn the spatial consistency of the disparity map, features, and cost volume, generate the weights of B*9*H*W, and generate the biases (B*16*H*W) in the x and y directions of the surrounding 8 pixels;

[0038] S513, then insert the center point into the bias, and the generated bias is used to locate the corresponding point through the sampler, and finally matrix multiplication is performed with the decentralized weight to obtain the residual, and the disparity map is updated.

[0039] The beneficial effects of the present invention are:

[0040] The present invention utilizes the spatial consistency between cost volume information, disparity map and RGB image, generates a correlation weight with surrounding pixels for each pixel, and performs spatial optimization on the full-scale disparity map, which can greatly enhance the geometric consistency of the disparity map to solve the discontinuity of disparity information and edge expansion problems.

[0041] The present invention makes full use of the multi-scale cost volume information of two hourglasses and uses cost volumes of different scales to iteratively update the disparity map, thereby enhancing the prediction effect of the predicted disparity map in weak texture areas and the robustness of scenes with a large disparity range.

[0042] The present invention adopts a lightweight U-Net to extract delicate feature information to generate spatial attention weights for guiding cost volume information, effectively alleviating the problems of excessive smoothing of cost volume and insufficient texture information.

[0043] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0045] Figure 1 It is a schematic diagram of the overall architecture of the progressive disparity optimization method based on contextual geometry perception of the present invention;

[0046] Figure 2 It is a schematic diagram of the feature extraction process of the present invention;

[0047] Figure 3 It is a schematic diagram of the cost volume polymerization process of the present invention;

[0048] Figure 4 It is a structural schematic diagram of the SAWG module of the present invention;

[0049] Figure 5 It is a structural schematic diagram of the SPON module of the present invention. DETAILED DESCRIPTION

[0050] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0051] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0052] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0053] See also Figure 1 to Figure 5 , which is a progressive disparity optimization method based on contextual geometry perception.

[0054] Example

[0055] This embodiment provides a detailed implementation process of a progressive disparity optimization method based on contextual geometry perception, which specifically adopts a multi-scale spatial optimization network model architecture, such as Figure 1 As shown, it enhances the prediction effect of the predicted disparity map in weak texture areas and the robustness of scenes with large disparity ranges through a multi-scale cost volume optimization strategy. In this model, a framework based on GWC-net is used, and the last two hourglass modules of GWC-Net are taken. The cost volumes of 1 / 16, 1 / 8, and 1 / 4 scales of each hourglass are taken, and the spatial attention feature information is supplemented by the SAWG (Spatial Attention Weight Generation) module in sequence. The obtained cost volume and the disparity map to be updated are input into the SPON (Spatial Optimization Network) module to generate residuals for updating the disparity map. The model of the present invention continuously incorporates multi-scale cost volume information, updates the disparity from coarse to fine, and ultimately solves the problems of edge expansion and disparity inconsistency. Specifically, the following steps are included:

[0056] S1, obtaining the initial images of the left and right views, preprocessing the images, and extracting image features;

[0057] S2, generating an initial cost volume according to the spliced ​​cost volume and the group-related cost volume;

[0058] S3, use three hourglasses for cost aggregation, and convolve the different scale cost volumes of the last two hourglasses into a single channel, and then interpolate to the full scale as the input of the disparity optimization stage;

[0059] S4, interpolate the cost volume output by the last hourglass to the full scale, and then perform disparity regression to predict the initial disparity map;

[0060] S5. Perform spatial disparity iterative optimization through the SAWG module and the SPON module. In the SAWG module, the spatial attention weight is used to guide the cost volume of each scale, and in the SPON module, the cost volume information, the spatial consistency of the RGB image and the initial disparity map are used to update the disparity map to complete the matching.

[0061] In step S1 of this embodiment, the left and right view images are first preprocessed, the input images of the left and right views are randomly cropped into RGB format with a size of 512×256, and the data is standardized. Then, features are extracted through the resnet network, where Figure 2 As shown, the feature extraction process includes:

[0062] The input image is first downsampled to extract 32-channel feature information at a scale of 1 / 2; then, after further feature extraction through two convolutional layers and three residual blocks, it is downsampled again and, after 16 residual blocks, 64-channel feature information l1 at a scale of 1 / 4 is obtained; then, l1 is extracted into 128-channel feature information l2 through three residual blocks, and further 128-channel feature information l3 is obtained through another three residual blocks; finally, l1, l2, and l3 are concatenated in the channel dimension to obtain 320-channel feature information l4, which is used to construct the correlation cost body; at the same time, l4 is convolved to extract 12 channels of feature information for constructing the concatenated cost body.

[0063] In step S2 of this embodiment, the cost volume consists of two parts, the splicing cost volume and the group-related cost volume. Splicing volume: Convolve the features into 12 channels for the construction of the splicing cost volume. Grouping cost volume: Divide the channels into 40 groups, and construct the related cost volume for each group to obtain the grouping cost volume. Finally, the splicing cost volume and the group-related cost volume are spliced ​​and fused as the input of the cost aggregation. Among them, the splicing cost volume formula is:

[0064] C concat (d,x,y,·)=Concat{f l (x,y),f r (xd,y)}

[0065] In the formula, C concat (d,x,y,·) is the stitching cost volume, d,x,y are disparity, pixel horizontal coordinate, pixel vertical coordinate, "·" represents a placeholder, f l (·) represents the feature vector of the left pixel, f r(·) represents the feature vector of the pixels in the right image, and Concat{·} represents the concatenation method;

[0066] The relevant cost volume formula is:

[0067]

[0068] In the formula, C gwc (d,x,y,g) is the relevant cost volume, g is the index of the feature channel grouping, N c ,N g are the total number of feature channels and the number of groups, respectively. are the g-th group of left image eigenvectors and the g-th group of right image eigenvectors, respectively, <·> represents the vector inner product;

[0069] In step S3 of this embodiment, multiple 3D hourglasses are used to aggregate the cost volume, where Figure 3 As shown, this embodiment uses three 3D hourglasses to aggregate the cost volume, and each 3D hourglass includes a coding stage and a decoding stage.

[0070] Encoding stage: First, the 32-channel 1 / 4-scale cost volume V4 is downsampled to obtain a 64-channel 1 / 8-scale cost volume, then a 3D convolution is performed to obtain V8, and then downsampled again, and a 3D convolution is performed to obtain a 128-channel 1 / 16-scale cost volume V16.

[0071] Decoding stage: The cost volume V16 is subjected to 3D deconvolution to obtain V8_1, V8_1 is added to V8 after the 1*1*1 convolution kernel, and then the result is activated by the relu function to obtain V8_2. V8_2 is obtained in the same way to obtain V4_2. V4_2 is used as the input of the subsequent hourglass.

[0072] Take the 1 / 16, 1 / 8, and 1 / 4 scale cost volumes of the last two hourglasses, convolve them into a single channel, and interpolate them to the full scale for input of disparity optimization.

[0073]

[0074] In the formula, res represents the scale, V res Represents the cost volume V of the res scale res ∈R (D×res)×(H×res)×(W×res) , f conv (.) represents 3D convolution, Interp(.) represents trilinear interpolation, Represents the full-scale cost volume obtained by interpolating the res-scale cost volume

[0075] In step S4 of this embodiment, the process of disparity regression prediction is as follows: for each output module, two 3D convolutions are used to generate a 1-channel cost volume and upsample it. Subsequently, a softmax function is applied on the disparity dimension to convert the cost volume into a probability volume. Finally, the predicted disparity of each pixel is calculated by weighted summation (i.e., the probability value of each disparity is multiplied by the disparity value and added together), and the formula is as follows:

[0076]

[0077] In the formula, is the predicted disparity value, D max is the maximum disparity, k is the discrete disparity candidate value, p k represents the probability value for k;

[0078] In step S5 of the embodiment, each cost volume is processed using the SAWG module using the multi-scale cost volume information to alleviate the problem of over-smoothing of the cost volume and insufficient texture information. By inputting the optimized cost volume and RGB image, the SPON module is used to learn the spatial consistency between pixels and generate a residual for updating the full-scale disparity map.

[0079] In order to alleviate the problem of over-smoothing of cost volumes and insufficient texture information, the present invention proposes a SAWG module, which uses RGB images to generate delicate spatial attention weights to guide cost volumes of various scales. Figure 4 As shown, this embodiment mainly uses a lightweight U-net to extract delicate features from the RGB image, and then generates spatial attention weights through the Sigmiod activation function to activate the cost volume, generate weights and act on the cost volume. The process is expressed as follows:

[0080] V'(d,x,y)=V(d,x,y)⊙(σ(f U-Net (I)))

[0081] Where V(.) is the cost volume V(.)∈R D×H×W , I is the RGB image I∈R 3×H×W , f U-Net (.) is a lightweight U-Net, ⊙ represents element-by-element multiplication, and σ(.) is the sigmoid activation function.

[0082] In order to solve the problems of edge expansion and discontinuity caused by interpolation, the present invention proposes a SPON module to optimize the full-scale disparity map. The cost volume information, RGB image and initial disparity map are mainly used to use deformable convolution to obtain weights and biases, which are used to obtain the spatial consistency points between pixels and surrounding pixels, and the residuals are generated through these consistency points to update the disparity map.

[0083] like Figure 5 As shown in the figure, first, the RGB image and the initial disparity map are expanded to the C channel and then spliced ​​and fused. Then, the cost volume interpolated to the full scale is processed to the C channel through convolution, and the two are spliced ​​and fused:

[0084] F in =Concat(f conv (I,C),f conv (D init ,C))

[0085] F out =f conv (Concat(f conv (F in ,C),f conv (V init ,C)),C)

[0086] f conv (.,C) represents convolution to C channel, I represents the input RGB image I∈R 3×H×W , D init Represents the input disparity map D init ∈R 1×H×W , V init Represents the full-scale cost volume V of the input init ∈R 192×H×W , Concat(.) represents splicing along channel C.

[0087] Subsequently, a U-Net network is used to learn the spatial consistency of the disparity map, features, and cost volume: the obtained consistency information is divided into two functions: the first is used to generate weights (B*9*H*W), and the second is used to generate biases of surrounding pixels (B*16*H*W):

[0088] F corr =f U-Net (F out )

[0089] weight(C,H,W)=f conv (F corr ,9)

[0090]

[0091] offset(C,H,W)=f conv (F corr ,16)

[0092] f conv (.,C) represents convolution to C channels, f U-Net (.) is a lightweight U-Net, C iRepresents the value of the i-th channel.

[0093] Then the x and y biases of the center point are inserted into the bias. The generated bias is used to locate the corresponding point through the sampler, and finally the matrix multiplication with the weight is performed to obtain the residual, which is used to update the disparity map. The process is expressed as:

[0094]

[0095] coord = f gridgenerate (k)+offset(b*hw,r,r,2)

[0096] coord=(coord / k×2)-1

[0097] depth res =f sample (f unfold (depth,k),coord)

[0098] In the formula, f Pixelinsert (.) represents the offset insertion of the pixel center point, f gridgenerate (.) represents a uniformly generated grid, f unfold (.) means selecting the pixel value within the range of k*k for each pixel, f sample (.) indicates selecting disparity pixels based on grid coordinates.

[0099] The cost volumes obtained by the last two hourglasses are interpolated to the full-scale regression to obtain the initial full-scale disparity map according to the conventional method. The spatial attention weight is introduced to guide the interpolated cost volumes of each scale. Then the disparity map, RGB left image, and guided cost volume are input into SPON to generate residuals, and the initial disparity map is iteratively updated. Taking GWC-net as the baseline model, this embodiment updates the disparity map 6 times.

[0100] Output and result verification:

[0101] In this embodiment, the Sceneflow dataset is used to verify the performance of the present invention. The size of the final disparity map generated is the same as the original input Figure 1 Finally, the comparison results between the present invention and other existing methods are shown in Table 1:

[0102] Table 1

[0103]

[0104] Experiments show that the most advanced results are achieved on the Sceneflow dataset. Compared with Gwc-net, the EPE is significantly improved from 0.61 to 0.43 under the same training conditions. The strategy of the present invention achieves the most advanced results on the sceneflow dataset.

[0105] It is worth mentioning that the SAWG module and SPON module of the present invention can be seamlessly embedded in other stereo matching networks using hourglass modules for optimization and updating of full-scale disparity maps. Models such as PSM-Net, GC-Net and CF-Net can integrate the above modules.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A progressive disparity optimization method based on contextual geometry perception, characterized in that: The method comprises the following steps: S1, obtaining the initial images of the left and right views, preprocessing the images, and extracting image features; S2. Constructing a splicing cost volume and a grouping related cost volume according to the splicing features and the correlation features to generate an initial cost volume; S3, use three hourglasses for cost aggregation, and convolve the different scale cost volumes of the last two hourglasses into a single channel, and then interpolate to the full scale as the input of the disparity optimization stage; S4, interpolate the cost volume output by the last hourglass to the full scale, and then perform disparity regression to predict the initial disparity map; S5. Perform disparity iterative optimization through the SAWG module and the SPON module. In the SAWG module, the spatial attention weight is used to guide the cost volume of each scale, and in the SPON module, the spatial consistency of the cost volume information, the RGB image and the initial disparity map is used to update the disparity map to complete the matching.

2. The context-based geometry perception based progressive disparity optimization method according to claim 1, characterized in that: In step S1, the preprocessing includes: randomly cropping the input images of the left and right views into RGB formats of a preset size and performing standardization processing; Then the features are extracted through the resnet network, where the feature extraction process includes: The input image is first downsampled to extract 32-channel feature information at a scale of 1 / 2; then, after further feature extraction through two convolutional layers and three residual blocks, it is downsampled again and, after 16 residual blocks, 64-channel feature information l1 at a scale of 1 / 4 is obtained; then, l1 is extracted into 128-channel feature information l2 through three residual blocks, and further 128-channel feature information l3 is obtained through another three residual blocks; finally, l1, l2, and l3 are concatenated in the channel dimension to obtain 320-channel feature information l4, which is used to construct the correlation cost body; at the same time, l4 is convolved to extract 12 channels of feature information for constructing the concatenated cost body.

3. The context-based geometry perception based progressive disparity optimization method according to claim 2, characterized in that: In step S2, the features are convolved to 12 channels to construct the concatenated cost volume, which is expressed as: C concat (d,x,y,·)=Concat{f l (x,y),f r (x-d,y)} In the formula, C concat (d,x,y,·) is the stitching cost volume, d,x,y are disparity, pixel horizontal coordinate, pixel vertical coordinate, "·" represents a placeholder, f l (·) represents the pixel feature vector of the left image, f r (·) represents the pixel feature vector of the right image, and Concat{·} represents the concatenation method; The channels are divided into several groups, and the correlation cost volume is constructed for each group, which is expressed as: In the formula, C gwc (d,x,y,g) is the relevant cost volume, g is the index of the feature channel grouping, N c ,N g are the total number of feature channels and the number of groups, respectively. are the g-th group of left image eigenvectors and the g-th group of right image eigenvectors, respectively, <·> represents the vector inner product; The concatenated cost volume and the group-related cost volume are concatenated and fused as the input of the cost aggregation.

4. The context-based geometry perception based progressive disparity optimization method according to claim 3, characterized in that: In step S3, multiple 3D hourglasses are used to perform cost volume aggregation. In each 3D hourglass, the following are included: Encoding stage: First, the 1 / 4 scale cost volume V4 of 32 channels is downsampled to obtain the 1 / 8 scale cost volume of 64 channels, and then a 3D convolution is performed to obtain V8, which is then downsampled again and a 3D convolution is performed to obtain the 1 / 16 scale cost volume V of 128 channels. 16 ; Decoding stage: cost volume V 16 Perform 3D deconvolution to get V8′, add V8′ and V8 after 1*1*1 convolution kernel, and then activate the result with relu function to get V″8; V″8 gets V″4 in the same way; V″4 is used as the input of subsequent hourglass; Take the 1 / 16, 1 / 8, and 1 / 4 scale cost volumes of the last two hourglasses, convolve them into a single channel, and interpolate them to the full scale for input of disparity optimization: In the formula, res represents the scale, V res Represents the cost volume V of the res scale res ∈R (D×res)×(H×res)×(W×res) , f conv (.) represents 3D convolution, Interp(.) represents trilinear interpolation, Represents the full-scale cost volume obtained by interpolating the res-scale cost volume 5. A context-based geometry perception based progressive disparity optimization method according to claim 4, characterized in that: In step S4, for each output module, two 3D convolutions are used to generate a 1-channel cost volume and upsample it; then, a softmax function is applied on the disparity dimension to convert the cost volume into a probability volume. Finally, the predicted disparity of each pixel is calculated by weighted summation, as follows: In the formula, is the predicted disparity value, D max is the maximum disparity, k is the discrete disparity candidate value, p k Represents the probability value for k.

6. The context-based geometry perception-based progressive disparity optimization method according to claim 5, characterized in that: In the SAWG module of step S5, a lightweight U-net network is used to extract fine features from the RGB image, and then the cost volume is activated by generating spatial attention weights through the Sigmiod activation function. The weights are generated and acted on the cost volume. The process is expressed as follows: V'(d,x,y)=V(d,x,y)⊙(σ(f U-Net (I))) Where V(.) is the cost volume V(.)∈R D×H×W , I is the RGB image I∈R 3×H×W , f U-Net (.) is a lightweight U-Net network, ⊙ represents element-by-element multiplication, and σ(.) is the sigmoid activation function.

7. The context-based geometry perception-based progressive disparity optimization method according to claim 6, characterized in that: In the SPON module in step S5, the following operations are included: S511, expanding the RGB image and the initial disparity map to the C channel and then splicing and fusing them, then interpolating the full-scale cost volume to the C channel through convolution, and splicing and fusing the two again; S512, then, a U-Net network is used to learn the spatial consistency of the disparity map, features, and cost volume, generate the weights of B*9*H*W, and generate the biases (B*16*H*W) in the x and y directions of the surrounding 8 pixels; S513, then insert the center point into the bias, and the generated bias is used to locate the corresponding point through the sampler, and finally matrix multiplication is performed with the decentralized weight to obtain the residual, and the disparity map is updated.

Images