Multi-view three-dimensional reconstruction method based on depth perception

Abstract

The invention discloses a multi-view three-dimensional reconstruction method based on depth perception, and relates to multi-view three-dimensional reconstruction. Through a multi-scale feature extraction framework, in combination with a feature pyramid and a double-branch deep fusion network, high-discrimination-force features are extracted from images of multiple visual angles. Then, utilizing microhomography transformation to map the two-dimensional features to a three-dimensional depth hypothesis plane, constructing a cost body, and emphasizing feature relevance between views through a grouping aggregation strategy; and the cost body is regularized by adopting 3D U-Net in combination with a deep collaborative attention mechanism so as to reduce mismatching and redundancy. Depth map estimation is regarded as a classification task, a depth hypothesis probability is calculated by using Softmax, an optimal depth value is selected through Argmax, and a multi-view depth map is generated. The whole process is trained by adopting a cross entropy loss supervision model, and high-precision three-dimensional reconstruction is realized. The method effectively improves the precision and efficiency of three-dimensional reconstruction, and shows higher precision and robustness in the three-dimensional reconstruction of a complex scene.

Description

Technical Field

[0001] The present invention relates to multi-view Figure 3 3D reconstruction, and more particularly, to a multi-view stereo 3D reconstruction method based on depth perception. Background Art

[0002] 3D reconstruction technology is a core research direction in the field of computer vision, and its goal is to restore the 3D structure of the scene. With the rapid development of artificial intelligence and computer vision technology, 3D reconstruction has been widely used in many fields such as virtual reality, autonomous driving, game production, precision agriculture, and medical diagnosis. However, although 3D reconstruction technology has made important progress in many applications, it still faces many challenges when dealing with complex scenes.

[0003] Multi-view Stereo (MVS) is an important method for 3D reconstruction, which aims to recover the 3D structure and texture information of a scene from 2D images of multiple viewpoints. Traditional 3D reconstruction methods mainly rely on manually designed similarity metrics and engineering regularization methods to restore the 3D point cloud of the scene through multi-view feature matching. Typical reconstruction algorithms such as Colmap, Gipuma and OpenMVS often perform poorly when dealing with complex scenes such as weak textures, repeated textures and reflections, resulting in incomplete or insufficiently accurate reconstruction results.

[0004] With the introduction of deep learning technology, the field of 3D reconstruction has ushered in new breakthroughs. In 2018, Yao Yao et al. proposed the first end-to-end multi-view stereo reconstruction framework MVSNet. The network first uses convolutional neural network (CNN) for feature extraction to more effectively obtain high-level image features. Then, the features are encoded through camera parameters, and the features are projected into 3D space using microprojection transformation to form a cost volume. Finally, it uses a variance-based multi-view aggregation method to adapt to the input of any view, and performs noise filtering and depth map generation based on a 3D convolutional neural network structure. Although this method can extract more discriminative features from large-scale data, it is difficult to apply to the reconstruction of actual scenes due to the large memory usage and high computational cost of the 3D CNN structure. Subsequent research work has made many improvements to MVSNet. For example, CasMVSNet, CVP-MVSNet, etc. proposed a coarse-to-fine depth inference structure that generates depth maps at multiple resolutions, gradually reducing the number of hypothesized planes, thereby improving accuracy and efficiency. There are also some methods, such as R-MVSNet and AA-RMVSNet, which use recurrent neural networks (RNNs) to split the three-dimensional cost volume into a series of cost maps along the depth dimension and then process them, thereby avoiding the use of 3D CNN to reduce memory usage.

[0005] Although multi-view reconstruction methods based on deep learning have made some progress in accuracy and efficiency, many coarse-to-fine methods currently only use the depth map in the coarse stage to guide the depth hypothesis, and fail to fully utilize the depth information from the coarse depth map itself, resulting in insufficient accuracy and completeness of depth estimation. At the same time, during the cost volume regularization process, since features at different spatial locations share visual similarities, this will lead to similar confidence scores being assigned to the same positions at different depth levels, causing ambiguity in depth estimation, thereby affecting the final reconstruction quality. Therefore, there is still much room for improvement in multi-view stereo 3D reconstruction methods based on learning. Summary of the invention

[0006] The purpose of the present invention is to propose a multi-view stereo network 3D reconstruction method based on depth perception to address the problems that the multi-view stereo 3D reconstruction method fails to fully utilize the depth information in the coarse depth map and the feature mismatch in the cost volume regularization stage, resulting in insufficient reconstruction accuracy and completeness. The method adopts a dual-branch deep fusion network and a deep collaborative attention module to fully utilize the depth information in the multi-view feature extraction and cost volume regularization process, thereby improving the accuracy of depth estimation and the quality of 3D reconstruction.

[0007] This paper constructs a new multi-view stereoscopic 3D reconstruction framework based on depth perception, providing an innovative solution for 3D reconstruction technology. Compared with most existing methods, this framework shows significant advantages in improving the quality of 3D reconstruction, especially when reconstructing complex scenes, it can effectively improve the reconstruction accuracy and robustness. It provides strong technical support for a wide range of application scenarios, especially 3D reconstruction in large-scale and complex environments.

[0008] The present invention comprises the following steps:

[0009] 1) Multi-view feature extraction: Based on a multi-scale coarse-to-fine feature extraction framework, combined with feature pyramid and dual-branch deep fusion network, highly discriminative image features are extracted from images of N+1 different perspectives; feature maps of four scales are obtained, with sizes of 1, 1 / 2, 1 / 4, and 1 / 8 of the original image respectively; the dual-branch deep fusion network includes a feature extraction branch and a deep information fusion branch, both of which adopt an encoder-decoder structure and enhance feature fusion through jump connections;

[0010] 2) Cost volume construction: Based on differentiable homography transformation, the two-dimensional source feature maps of different views are mapped to the three-dimensional discrete depth hypothesis plane to obtain N source feature volumes; the similarity between the source feature volume and the reference feature is calculated by using a grouping aggregation strategy to aggregate the source feature volume and finally construct the cost volume C; the grouping aggregation strategy includes grouping similarity calculation and cross-attention mechanism to emphasize the correlation between features of different views;

[0011] 3) Cost body regularization: 3D U-Net combined with deep collaborative attention mechanism is used to regularize the cost body to reduce mismatches and redundant information in the cost body; the deep collaborative attention mechanism includes one-dimensional sequence segmentation, one-dimensional convolution processing of different sizes, group normalization and function activation, and self-attention calculation to enhance the feature representation ability of the cost body;

[0012] 4) Depth map inference: The depth map estimation task is regarded as a classification task. The probability of each depth hypothesis is calculated through the Softmax function, and the depth value with the largest probability is selected using the Argmax function. Finally, a multi-view depth map of the scene is obtained, and the cross-entropy loss is used to supervise the training of the model.

[0013] Furthermore, the method also includes the following steps: after obtaining the multi-view depth map of the scene, combining the corresponding RGB image and the camera's projection matrix, using the open3D library to calculate the consistency between the depth maps; and aligning the depth information of multiple viewpoints to generate a point cloud model of the scene; through this process, an accurate three-dimensional model of the scene is reconstructed from the two-dimensional images of multiple viewpoints.

[0014] In step 1), the encoder part of the feature extraction branch includes a convolution block, two basic residual blocks and three downsampling layers; the input of the depth information fusion branch is composed of the fused depth map and the coarse depth map.

[0015] In step 2), in the grouping aggregation strategy, the reference feature map is stacked along the depth hypothesis plane to obtain a reference feature body, and the reference feature body and the source feature body are grouped to reduce the number of channels of the feature body, and then the similarity and attention weight are calculated to obtain the final cost body.

[0016] In step 3), the encoder part of the 3D U-Net is realized by stacking convolution blocks and downsampling layers, and the cost body is downsampled multiple times; the decoder part is formed by stacking convolution blocks and upsampling layers, and the same-scale cost bodies enhanced by the deep collaborative attention module are added through jump connections.

[0017] In step 3), in the deep collaborative attention mechanism, the input is average pooled along the width, height and depth dimensions to obtain three one-dimensional sequences, and then one-dimensional convolution processing of different sizes, group normalization and function activation are used to obtain the attention corresponding to each channel, and finally the weighted cost body is calculated.

[0018] In step 4), the cross entropy loss calculation method considers the total number of pixels in the image, the difference between the true distribution and the model predicted distribution, and the total loss function of the four stages of the model is obtained by weighting with different weights.

[0019] Compared with the prior art, the advantages of the present invention are:

[0020] 1) In the multi-view feature extraction stage, the present invention combines feature pyramids and dual-branch deep fusion networks for feature extraction, making full use of the depth prior information from the coarse stage, so that the network can learn the implicit rich scene edge and geometric information. This method can perform more accurate and robust reconstruction for challenging areas such as scene reflections, repeated textures, and weak textures. Conventional coarse-to-fine deep learning methods only use the depth map of the coarse stage to guide the range of depth assumptions, losing the rich geometric information contained in the depth map itself, resulting in low reconstruction accuracy.

[0021] 2) In the cost body construction stage, the present invention introduces a group similarity aggregation strategy. Compared with the traditional variance-based or learning-based aggregation strategy, this strategy can aggregate the source feature body more efficiently and accurately, significantly improving the robustness and accuracy of the model.

[0022] 3) In the cost volume regularization stage, due to the existence of non-Lambertian surfaces and object occlusions, the original cost volume calculated from the image features may be contaminated by noise. This noise may cause the features of different key points to be incorrectly matched, resulting in similar confidence at different depths of the cost volume, affecting the accuracy of depth estimation. Since the similarity confidence problem occurs in different depth hypothesis layers at the same pixel position, some current attention methods only focus on width, height, and channel dimensions, which has certain limitations. The present invention designs a deep collaborative attention module that uses the information of the entire cost volume to calculate attention weights to enhance or weaken similarity confidence at different depths, thereby alleviating the problem of feature mismatch. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a multi-view stereoscopic 3D reconstruction framework diagram based on depth perception of the present invention;

[0024] Figure 2 This is a diagram of the dual-branch deep fusion network structure of the present invention;

[0025] Figure 3 This is a regularized network structure diagram of the present invention;

[0026] Figure 4 This is a structural diagram of the deep collaborative attention module of the present invention;

[0027] Figure 5 This is a partial reconstruction result of the present invention on the DTU dataset;

[0028] Figure 6 Comparison of the reconstruction results of the present invention with those of other methods. DETAILED DESCRIPTION

[0029] In order to make the technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in the following embodiments in conjunction with the accompanying drawings.

[0030] This paper proposes a multi-view stereo 3D reconstruction method based on depth perception. The overall network architecture is detailed in Figure 1 The specific embodiment includes the following steps:

[0031] S1. Multi-view feature extraction: This method combines feature pyramid and dual-branch deep fusion network to extract The image features with high discriminability are extracted from images of different perspectives. Based on the multi-scale coarse-to-fine feature extraction framework, feature maps of four scales are obtained, with sizes of 1, 1 / 2, 1 / 4, and 1 / 8 of the original image respectively.

[0032] S2. Cost volume construction: Based on the differentiable homography transformation, the two-dimensional source feature maps of different views are mapped to the three-dimensional discrete depth hypothesis plane to obtain N source feature volumes, and the grouping aggregation strategy is used to calculate the similarity between the source feature volume and the reference feature to aggregate the source feature volume, and finally construct the cost volume C.

[0033] S3. Cost body regularization: 3D U-Net combined with deep collaborative attention mechanism is used to regularize the cost body to reduce mismatches and redundant information in the cost body.

[0034] S4. Depth map inference: The depth map estimation task is regarded as a classification task. The probability of each depth hypothesis is calculated by the Softmax function, and the depth value with the largest probability is selected using the Argmax function to finally obtain an accurate depth map. The cross entropy loss is used to supervise the training of the model.

[0035] Step S1 is specifically as follows:

[0036] Given a scenario Multi-view images ,in As a reference image, As source images, these images are input into the feature extraction network for feature extraction. The overall network structure of the present invention includes four stages from coarse to fine. The feature extraction network uses a feature pyramid to extract multi-scale features in the coarsest stage (stage 0), and a dual-branch deep fusion network is used in the subsequent stages. The basic structure of the dual-branch deep fusion network is as follows Figure 2 As shown in Figure 1, it includes two main branches: feature extraction branch and depth information fusion branch. This design can simultaneously utilize the color information of the image and the spatial information of the depth image, effectively promoting the full fusion of image features and depth information, and providing stronger feature representation capabilities for depth estimation tasks.

[0037] The feature extraction branch adopts an encoder-decoder network structure with symmetric jump connections. The encoder part includes a convolution block, two basic residual blocks and three downsampling layers. The RGB image is spliced ​​with the normalized depth map, and after the initial convolution layer, the image features are extracted through multiple layers of downsampling and basic residual blocks. Then, the decoder gradually upsamples the feature map through a series of deconvolution layers, and finally generates a feature map of the same size as the input image. The specific calculation method is shown in formula (1):

[0038] (1)

[0039] in, Indicates The depth map after stage upsampling, Indicates The reference image of the stage, Represents a splicing operation, Represents the feature extraction operation of the feature extraction branch.

[0040] The structure of the depth information fusion branch is similar to that of the feature extraction branch, and also adopts an encoder-decoder structure. The input of this branch is the fused depth map and a rough depth map The encoder fuses the features of the corresponding levels in the feature extraction branch through skip connections, thereby enhancing the information interaction between branches and improving the network's perception of image texture and depth information. At the end of the network, the output of the depth information fusion branch is combined with the original features extracted by FPN from the coarsest stage. The sum is performed and finally a refined depth map is obtained through a convolutional layer. The calculation method is shown in formula (2).

[0041] (2)

[0042] in, represents the convolution operation, represents the element addition operation, Represents the feature extraction operation of the deep information fusion branch; The final input image After the feature extraction network, four sets of feature maps of different scales are obtained , , , , the feature map sizes are , , , .

[0043] Step S2 is specifically as follows:

[0044] After obtaining the multi-scale feature maps from the multi-view feature extraction step, according to the depth hypothesis The position of the corresponding pixel in the source image in the reference image is calculated by the differentiable homography transformation based on the input camera parameters, so as to achieve feature matching between the reference image and the source image. and in The corresponding pixel in the source perspective , in Depth Hypothesis Below, the feature map is defined as:

[0045] (3)

[0046] in, and Represent the reference view and The camera intrinsic parameter matrix of the source view, and From the reference perspective to the The rotation matrix and translation matrix of the source view. Through this process, The source image is transformed into Source feature .

[0047] Then, the group similarity aggregation strategy is used to aggregate the feature-mapped The source feature volumes are aggregated into a 3D cost volume, thereby combining the geometric information of the multi-view images into a unified 3D structure, where each position contains a cost value under a certain depth assumption. Stack along the depth assumption plane to obtain the reference feature body , and reference feature With the source feature Using the grouping strategy, the number of channels of the feature body is reduced, and we get ,in, Indicates the number of groups, and Represents the reference feature body and source feature body after grouping. Calculated according to formula (4) and , and average according to the number of channels:

[0048] (4)

[0049] in, Indicates The reference feature of the group, Indicates Group in The source feature of the view, Represents element-wise dot product.

[0050] Under the same depth assumption, the pixels corresponding to different views contribute differently to the final cost volume. Therefore, the attention weights of different source feature volumes are calculated through the cross-attention mechanism according to formula (5):

[0051] (5)

[0052] in, Indicates The view in Attention scores under the layer depth assumption plane, represents the similarity between the reference feature map and the source feature map under the layer depth hypothesis plane, is the temperature coefficient.

[0053] Finally, according to the aggregation weight corresponding to each source feature body Calculate the final cost volume , the calculation formula is as follows:

[0054] (6)

[0055] The group similarity aggregation strategy effectively improves computational efficiency, generates attention based on the contribution of each source view under different depth assumptions, emphasizes the correlation between features of different views, generates the final cost body, and lays the foundation for subsequent cost body regularization.

[0056] Step S3 is as follows:

[0057] Due to the presence of non-Lambertian surfaces and object occlusions, the original cost volume calculated from image features It may be contaminated by noise. The presence of this noise can lead to incorrect depth information during deep reasoning. Therefore, the regularization step aims to refine the cost volume to generate a probability volume for deep reasoning. .

[0058] The regularized network structure adopted by the present invention is as follows Figure 3 As shown in the figure, the whole is a 3D U-Net with an encoder-decoder structure. The encoder part is implemented by stacking convolution blocks and downsampling layers, which downsamples the cost volume three times to generate the original scale , , The decoder is composed of stacked convolutional blocks and upsampling layers, and the same-scale cost bodies enhanced by the deep collaborative attention module are added through skip connections.

[0059] Figure 4For the deep collaborative attention structure diagram, given an input , average pooling is performed along the width, height, and depth dimensions, resulting in three one-dimensional sequences: , each sequence is divided into four sub-features, using It is defined as shown in the following formula:

[0060] (7)

[0061] (8)

[0062] (9)

[0063] in, Then, one-dimensional convolutions of different sizes are used to process these sub-features, and we get , and these sub-features are concatenated and normalized by grouping and After the function is activated, the attention corresponding to each channel is obtained:

[0064] (10)

[0065] (11)

[0066] (12)

[0067] in, , and Represent the weights in the width, height and depth dimensions respectively, express function, represents the group normalization operation, Represents the concatenation operation. The weighted cost volume can be calculated by formula (13): :

[0068] (13)

[0069] Next, Apply 7×7×7 average pooling to reduce the cost volume size and perform group normalization to obtain , and then use grouped convolution techniques to generate query, key, and value matrices to calculate self-attention.

[0070] (14)

[0071] (15)

[0072] (16)

[0073] (17)

[0074] in, , , Represents the mapping functions that generate queries, keys, and values, respectively, Indicates size

[0075] 1×1×1 grouped convolution.

[0076] Finally, the attention score of the channel dimension is calculated according to formula (19): , and with Weighted to get the cost body after attention enhancement :

[0077] (18)

[0078] (19)

[0079] Price Body After the cost body regularization network combined with deep collaborative attention in this step, the refined cost body after filtering out noise is obtained. .

[0080] Step S4 is specifically as follows:

[0081] The present invention regards the depth estimation task as a classification task and adopts a “winner takes all” strategy to obtain the final depth map. First, the fine cost volume is calculated according to formula (20). Apply the Softmax function along the depth dimension to obtain the probability body , the probability volume contains the corresponding weights of all depth hypotheses:

[0082] (20)

[0083] in, The cost body corresponds to The value of the position, Indicates the number of depth hypothesis layers.

[0084] Then, the depth hypothesis value with the highest probability is calculated according to formula (21) as the final estimated depth value of the pixel:

[0085] (twenty one)

[0086] Specifically, at the pixel location From all possible depth assumptions Select a depth value from , which makes the probability The final result is is the most likely depth value for the pixel.

[0087] Finally, the present invention adopts the training of the cross entropy loss supervision model, and the calculation method is shown in formula (22):

[0088] (twenty two)

[0089] in, represents the total number of pixels in the image, represents the true distribution, In the case of one-hot encoding, only the true depth corresponds to The item is 1, and the others are 0. The total loss function of the four stages of the model is obtained by weighting with different weights. The specific calculation method is shown in formula (23):

[0090] (twenty three)

[0091] in, is the total number of stages, represents the weight coefficient of the first stage, represents the cross entropy loss of the first stage. In the present invention, Set to 4, Both are set to 1.

[0092] Furthermore, after obtaining the multi-view depth map of the scene, the corresponding RGB image and the camera's projection matrix are combined, and the open3D library is used to calculate the consistency between the depth maps and align the depth information of multiple viewpoints to generate a point cloud model of the scene.

[0093] The multi-view stereo 3D reconstruction method based on depth perception designed in the present invention is trained and tested on the public indoor dataset DTU. Figure 5 The reconstruction results of the reconstruction method of the present invention on the DTU test set are demonstrated. The multi-view stereo 3D reconstruction method based on depth perception of the present invention can still achieve excellent 3D reconstruction effects when facing challenging scenes, such as areas with unclear features such as low texture, reflection and occlusion.

[0094] Figure 6 The comparison results of the point cloud models generated by the present invention and some existing 3D reconstruction methods are shown. By comparing the point cloud models of different methods, the advantages of the present invention in reconstruction effect can be intuitively presented.

[0095] Table 1 shows the performance differences between the 3D reconstruction method of the present invention (Ours) and various existing 3D reconstruction methods (Gipuma, COLMAP, MVSNet, etc.) in terms of accuracy, completeness, and overall error. Boldface indicates the best ranking, and underlined indicates the second best. The evaluation indicators include accuracy, completeness, and overall error. The lower the value, the better the model performance.

[0096] Table 1 Quantitative comparison of 3D reconstruction results

[0097]

[0098] In the comparison process, accuracy examines the average distance between the three-dimensional point cloud reconstructed by the method of the present invention and the real point cloud, integrity evaluates the average distance between the real point cloud and the point cloud reconstructed by the present invention, and the overall error is the mean of accuracy and integrity, which is an important indicator for comprehensively evaluating the model effect. The accuracy of the present invention is 0.327mm, which is at a relatively good level in the table, indicating that the present invention can capture the shape and position of the object more accurately with a small error during the reconstruction process. The integrity of the present invention is 0.301mm, ranking first in Table 1, which is significantly better than all other methods, indicating that the present invention can retain the information in the real scene more completely and reduce information loss during the reconstruction process. The overall error of the present invention is 0.314mm, which also ranks first in Table 1, better than all other methods, indicating that the present invention has high comprehensive performance in three-dimensional reconstruction.

[0099] Although some methods are not much different from the present invention in terms of accuracy (such as CasMVSNet), the present invention has more obvious advantages in terms of completeness and overall error. Figure 5 and 6 From the demonstration, it can be inferred that the present invention can still achieve excellent 3D reconstruction effect when facing challenging scenes (such as areas with low texture, reflection, occlusion and other unclear features). This shows that the present invention has high applicability and robustness.

[0100] Experimental data show that compared with the traditional framework, the reconstruction method of the present invention can not only better capture the details of the target object, but also maintain a high reconstruction quality in complex scenes, achieve more complete and accurate three-dimensional reconstruction, and surpass the existing technology in multiple indicators, with better performance.

[0101] The above embodiments are only preferred embodiments of the present invention and cannot be considered to limit the scope of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the present invention.

Claims

1. A multi-view stereoscopic 3D reconstruction method based on depth perception, characterized in that The following steps are involved: 1) Multi-view feature extraction: Based on a multi-scale coarse-to-fine feature extraction framework, combined with feature pyramid and dual-branch deep fusion network, highly discriminative image features are extracted from images of N+1 different perspectives; Four scale feature maps are obtained, the sizes of which are 1, 1 / 2, 1 / 4, and 1 / 8 of the original image respectively; the dual-branch deep fusion network includes a feature extraction branch and a deep information fusion branch, both of which adopt an encoder-decoder structure and enhance feature fusion through jump connections; 2) Cost volume construction: Based on the differentiable homography transformation, the two-dimensional source feature maps of different views are mapped to the three-dimensional discrete depth hypothesis plane to obtain N source feature volumes; A grouping aggregation strategy is used to calculate the similarity between the source feature body and the reference feature, so as to aggregate the source feature body and finally construct the cost body C; the grouping aggregation strategy includes grouping similarity calculation and cross attention mechanism to emphasize the correlation between features of different views; 3) Cost body regularization: 3D U-Net combined with deep collaborative attention mechanism is used to regularize the cost body to reduce mismatches and redundant information in the cost body; the deep collaborative attention mechanism includes one-dimensional sequence segmentation, one-dimensional convolution processing of different sizes, group normalization and function activation, and self-attention calculation to enhance the feature representation ability of the cost body; 4) Depth map inference: The depth map estimation task is regarded as a classification task. The probability of each depth hypothesis is calculated through the Softmax function, and the depth value with the largest probability is selected using the Argmax function. Finally, a multi-view depth map of the scene is obtained, and the cross-entropy loss is used to supervise the training of the model.

2. The multi-view stereoscopic 3D reconstruction method based on depth perception as claimed in claim 1, characterized in that The method further includes the following steps: after obtaining the multi-view depth map of the scene, the consistency between the depth maps is calculated using the open3D library in combination with the corresponding RGB image and the projection matrix of the camera; And align the depth information of multiple perspectives to generate a point cloud model of the scene; Through this process, an accurate 3D model of the scene is reconstructed from 2D images from multiple viewpoints.

3. The multi-view stereoscopic 3D reconstruction method based on depth perception as claimed in claim 1, characterized in that In step 1), the encoder part of the feature extraction branch includes a convolution block, two basic residual blocks and three downsampling layers; the input of the depth information fusion branch is composed of the fused depth map and the coarse depth map.

4. The multi-view stereoscopic 3D reconstruction method based on depth perception as claimed in claim 1, characterized in that In step 2), in the grouping aggregation strategy, the reference feature map is stacked along the depth hypothesis plane to obtain a reference feature body, and the reference feature body and the source feature body are grouped to reduce the number of channels of the feature body, and then the similarity and attention weight are calculated to obtain the final cost body.

5. The multi-view stereoscopic 3D reconstruction method based on depth perception as claimed in claim 1, characterized in that In step 2), the specific steps of constructing the cost body are: According to the depth assumption and the input camera parameters, the position of the corresponding pixel in the source image in the reference image is calculated through the differentiable homography transformation to achieve feature matching between the reference image and the source image; for a given pixel in the reference view and in The corresponding pixel in the source perspective , in Deep Hypothesis Below, the feature map is defined as: in, and Represent the reference view and The camera intrinsic parameter matrix of the source view, and From the reference perspective to the The rotation matrix and translation matrix of the source view; through this process, The source image is transformed into Source feature ; The group similarity aggregation strategy is used to aggregate the feature maps The source feature volumes are aggregated into a three-dimensional cost volume, thereby combining the geometric information of the multi-view images into a unified three-dimensional structure, where each position contains a cost value under a certain depth assumption; specifically, the reference feature map Stack along the depth assumption plane to obtain the reference feature body , and reference feature With the source feature Using the grouping strategy, the number of channels of the feature body is reduced, and we get ,in, Indicates the number of groups, and Represents the reference feature body and the source feature body after grouping; calculated according to the following formula and , and average according to the number of channels: in, Indicates The reference feature of the group, Indicates Group in The source feature of the view, Represents element-wise multiplication; Under the same depth assumption, the pixels corresponding to different views contribute differently to the final cost volume. Therefore, the attention weights of different source feature volumes are calculated through the cross-attention mechanism according to the following formula: in, Indicates The view in Attention scores under the layer depth assumption plane, represents the similarity between the reference feature map and the source feature map under the layer depth hypothesis plane, is the temperature coefficient; Finally, according to the aggregation weight corresponding to each source feature body Calculate the final cost volume , the calculation formula is as follows: The group similarity aggregation strategy effectively improves computational efficiency and generates attention based on the contribution of each source view under different depth assumptions, emphasizing the correlation between features of different views and generating the final cost volume.

6. The multi-view stereoscopic 3D reconstruction method based on depth perception as claimed in claim 1, characterized in that In step 3), the encoder part of the 3D U-Net is realized by stacking convolution blocks and downsampling layers, and the cost body is downsampled multiple times; the decoder part is formed by stacking convolution blocks and upsampling layers, and the same-scale cost bodies enhanced by the deep collaborative attention module are added through jump connections.

7. The multi-view stereoscopic 3D reconstruction method based on depth perception as claimed in claim 1, characterized in that In step 3), in the deep collaborative attention mechanism, the input is average pooled along the width, height and depth dimensions to obtain three one-dimensional sequences, and then one-dimensional convolution processing of different sizes, group normalization and function activation are used to obtain the attention corresponding to each channel, and finally the weighted cost body is calculated.

8. The multi-view stereoscopic 3D reconstruction method based on depth perception as claimed in claim 1, characterized in that In step 4), the cross entropy loss calculation method considers the total number of pixels in the image, the difference between the true distribution and the model predicted distribution, and the total loss function of the model is obtained by weighting with different weights.

9. The multi-view stereoscopic 3D reconstruction method based on depth perception as claimed in claim 1, characterized in that In step 4), the specific steps of the depth map inference are: The depth estimation task is regarded as a classification task, and the fine cost volume is calculated according to the following formula: Apply the Softmax function along the depth dimension to obtain the probability body , the probability volume contains the corresponding weights of all depth hypotheses: in, The cost body corresponds to The value of the position, Indicates the number of depth hypothesis layers; Then, the depth hypothesis value with the highest probability is calculated according to the following formula as the final estimated depth value of the pixel: Specifically, at the pixel location From all possible depth assumptions Select a depth value from , which makes the probability The maximum final result is the most likely depth value of the pixel; Finally, the cross entropy loss is used to supervise the training of the model, as shown in the following formula: in, represents the total number of pixels in the image, represents the true distribution, In the case of one-hot encoding, only the true depth corresponds to The item is 1, and the others are 0; the total loss function of the four stages of the model is obtained by weighting with different weights, as shown in the following formula: in, is the total number of stages, represents the weight coefficient of the first stage, represents the cross entropy loss at the stage.

Images