3D Reconstruction Method Based on Large Kernel Attention Mechanism

By employing feature extraction and cost volume regularization through a large kernel attention mechanism, the problem of poor reconstruction in information-fuzzy regions by existing methods is solved, achieving higher quality and more efficient 3D reconstruction.

CN117576336BActive Publication Date: 2026-11-17XIAMEN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311520459.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-11-15
Publication Date
2026-11-17
Estimated Expiration
2043-11-15

AI Technical Summary

Technical Problem

Existing multi-view 3D reconstruction methods based on convolutional neural networks suffer from insufficient reconstruction completeness and accuracy when dealing with areas with blurred information such as low texture, repetitive texture, and specular reflection.

Method used

A 3D reconstruction method based on the big kernel attention mechanism is adopted. Feature extraction is performed through the big kernel attention mechanism, and cost volume is constructed and regularized by combining differentiable homography transformation, adaptive aggregation strategy and global adaptive mean pooling attention. Finally, depth map inference is performed.

Benefits of technology

It improves reconstruction quality and efficiency, especially in complex scenarios where it can effectively extract global semantic information, thereby improving the integrity and accuracy of reconstruction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117576336B_ABST
    Figure CN117576336B_ABST
Patent Text Reader

Abstract

A 3D reconstruction method based on the large kernel attention mechanism. 1. Feature extraction: Based on a multi-scale feature extraction strategy, feature extraction is performed on images from N viewpoints of a reconstructed scene using the large kernel attention (LKA) mechanism; 2. Differentiable homography transformation: Spatial transformation between multiple views is achieved through homography transformation to obtain the pixel correspondence between the source image and multiple reference images; 3. Correlation body construction and regularization: Feature information at corresponding locations is extracted, and N-1 matching correlation bodies are constructed using feature correlation. An adaptive aggregation strategy (AAS) is used to construct a unified matching correlation body, and global adaptive mean pooling attention (GAAPA) is used for regularization; 4. Depth map inference: Depth map inference is treated as a classification task. The regularized cost body is used with the Softmax function in the depth dimension D to obtain the probability body. The optimal depth value of each pixel is inferred along the D dimension based on the probability distribution to form the final depth map.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to multi-view Figure 3 3D reconstruction, especially a 3D reconstruction method based on a large kernel attention mechanism. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence and computer vision, 3D reconstruction technology has received increasing attention. With the widespread adoption of smart terminals and mobile internet, 3D reconstruction technology has been applied on a large scale in many fields, such as virtual reality, smart cities, environmental monitoring, and medical diagnosis. As 3D reconstruction technology becomes more widely used, the technical requirements for reconstruction quality and accuracy are also increasing.

[0003] Multi-view stereo (MVS) reconstruction aims to recover textured 3D geometry from a series of pre-calibrated and overlapping 2D images. Before the maturity of deep learning, traditional methods used hand-crafted similarity metrics and engineered regularization to match features between multiple views to recover the 3D point cloud of a scene. Reconstruction frameworks based on traditional methods, such as Colmap, Gipuma, and OpenMVS, while achieving good reconstruction results in some cases, have limitations when dealing with challenging scenes. For example, low-texture and repetitive texture regions, specular reflections, and object occlusion make dense matching difficult, leading to poor reconstruction performance. Furthermore, while traditional reconstruction methods like Colmap perform very well in terms of reconstruction accuracy, there is still significant room for improvement in reconstruction completeness and efficiency.

[0004] With the increasing development and maturity of deep learning technology centered on neural networks, this technology has been introduced into the field of 3D reconstruction. In 2018, Yao Yao et al. from the Hong Kong University of Science and Technology (Yao Yao, Zixin Luo, Shiwei Li, et al. MVSNet: Depth inference for unstructured multi-view stereo. In Proceedings of the European Conference on Computer Vision, pages 767–783, 2018.) proposed the first end-to-end multi-view stereo reconstruction framework, MVSNet. MVSNet was the first attempt to use a deep learning-based depth map inference method for multi-view reconstruction. It uses differentiable homography transformation to encode camera parameters and extracts image features to construct a unified cost volume to accommodate multiple view inputs. It uses a 3D CNN to regularize the cost volume and infers the final depth map from the cost volume. Due to the structural characteristics of 3D CNNs, the computational cost of the cost volume regularization module is quite high. To alleviate this problem, subsequent research has made some improvements. CasMVSNet, UCS-Net, Vis-MVSNet, and others employ a coarse-to-fine strategy, first vaguely inferring early, low-resolution depth maps, and then gradually reducing the depth interval in later, higher-resolution stages to refine the inference of the early, coarse depth maps. Meanwhile, another RNN-based solution is also widely used. R-MVSNe, D2HC-RMVSNet, and others divide the constructed 3D cost volume along the depth dimension into a series of 2D cost maps, and sequentially regularize the cost maps using RNN-based methods to avoid using memory-intensive 3D CNNs.

[0005] Compared to traditional methods, deep learning-based 3D reconstruction has made some progress and breakthroughs. However, most existing learning-based multi-view reconstruction frameworks use conventional convolutional neural networks (CNNs) as their basic network backbone. These frameworks lack a global receptive field, thus failing to efficiently utilize the global semantic information of the image. They also perform poorly in areas with low texture or specular reflections, resulting in less than ideal reconstruction results. Therefore, learning-based 3D reconstruction still has significant room for improvement. Summary of the Invention

[0006] The purpose of this invention is to address multi-view Figure 3In 3D reconstruction, most existing methods based on convolutional neural networks perform poorly when dealing with areas with blurred information, such as low-texture, repetitive textures, and specular reflections, resulting in low reconstruction integrity and accuracy. This paper proposes a 3D reconstruction method based on Large Kernel Attention (LKA). By using a large kernel attention model for reconstruction, its global receptive field and robust dynamic kernel effectively improve the model's information generalization ability, thereby improving reconstruction quality and efficiency.

[0007] This invention establishes a complete 3D reconstruction framework based on the big kernel attention mechanism, providing a novel solution for 3D reconstruction technology. Compared with most existing methods, it improves the overall reconstruction quality and efficiency of 3D reconstruction, and can also achieve 3D reconstruction of scenes with some ambiguous information.

[0008] This invention includes the following steps:

[0009] S1. Feature Extraction: Based on the multi-scale feature extraction strategy, the feature extraction of N multi-view images is performed through the large kernel attention (LKA) mechanism to form feature maps at three scales, with resolutions of 1 / 1, 1 / 4, and 1 / 8 of the original input image, respectively.

[0010] S2. Differentiable homography transformation: Under a series of assumed depth values, spatial transformation between multiple views is achieved through homography transformation to obtain the pixel correspondence between the source image and multiple reference images;

[0011] S3. Cost Body Construction and Cost Body Regularization: After obtaining the positions of the source image pixels in N-1 reference images, the feature information of the corresponding positions is extracted respectively. N-1 matching cost bodies are constructed using feature correlation. Then, an adaptive aggregation strategy is used to construct a unified matching cost body. Finally, Global Adaptive Average-pooling Attention (GAAPA) is used for regularization.

[0012] S4. Depth Map Inference: Treating depth map inference as a classification task, the regularized cost volume is used with the Softmax function in the depth dimension D to obtain the probability volume. Based on the probability distribution, the optimal depth value of each pixel is inferred along the D dimension to form the final depth map.

[0013] Furthermore, a visibility-based fusion algorithm is used for deep fusion, combining the depth maps of the multi-view images inferred by the network with the RGB information of the multi-view images to write the final 3D point cloud output; the 3D representation of the scene is reconstructed from the 2D images of the scene from multiple perspectives.

[0014] Compared with the prior art, the advantages of the present invention are as follows:

[0015] 1) In the feature extraction stage, this invention uses a feature extraction method based on the Large Kernel Attention (LKA) mechanism. Compared with existing methods based on conventional CNNs, it possesses a local receptive field, a global receptive field, and a more robust dynamic kernel, resulting in stronger information generalization capabilities. It still achieves good reconstruction results even in scenes with blurred information, such as low-texture and repetitive texture regions, and specular reflection regions. Conventional CNNs, on the other hand, cannot efficiently extract long-range contextual information from images and therefore perform less well. Furthermore, the LKA mechanism also exhibits spatial and channel adaptability, demonstrating strong adaptability to complex scenes.

[0016] 2) In the cost body construction and regularization stage, this invention introduces an adaptive aggregation strategy (AAS). This invention abandons the variance-based aggregation method used in existing methods and instead uses adaptive aggregation based on convolutional CNNs to construct a unified cost body. Secondly, it employs Global Adaptive Mean Pooling Attention (GAAPA) for cost body regularization. Adaptive aggregation helps the network to model data information more flexibly, improving the model's robustness and effectiveness. Attached Figure Description

[0017] Figure 1 This invention provides an overall framework for 3D reconstruction based on an attention mechanism.

[0018] Figure 2 This is the single-stage attention network structure of the present invention;

[0019] Figure 3 This is a schematic diagram illustrating the LKA (Large Kernel Attention) mechanism of this invention.

[0020] Figure 4 This is a schematic diagram illustrating the principle of differentiable monography transformation in this invention.

[0021] Figure 5 This is the processing flow of the regularization module in this invention;

[0022] Figure 6 This is a flowchart of the Global Adaptive Mean Pooling Attention GAAPA structure of the present invention;

[0023] Figure 7 This is a partial reconstruction result of the DTU dataset of this invention;

[0024] Figure 8 This is a comparison of the 3D point cloud output results of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the following embodiments will be used in conjunction with the accompanying drawings to further illustrate the invention. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0026] A 3D reconstruction method based on an attention mechanism, the overall network framework of which is as follows: Figure 1 As shown, the specific implementation method includes the following steps:

[0027] S1. Feature Extraction. Based on a multi-scale feature extraction strategy, features are first extracted from N multi-view images using a Large Kernel Attention (LKA) mechanism to form feature maps at three scales, with resolutions of 1 / 1, 1 / 4, and 1 / 8 of the original input image, respectively.

[0028] S2. Differentiable Homography Transformation. Under a series of assumed depth values, spatial transformation between multiple views is achieved through homography transformation, obtaining the pixel correspondence between the source image and N-1 reference images.

[0029] S3. Correlation Matrix Construction and Regularization. After obtaining the positions of the source image pixels in N-1 reference images, feature information of the corresponding positions is extracted, and N-1 correlation matrices are constructed using feature correlation. Then, an adaptive aggregation strategy is used to construct a unified correlation matrix, and finally, Global Adaptive Average-pooling Attention (GAAPA) is used for regularization.

[0030] S4. Depth Map Inference. Treating depth map inference as a classification task, the regularized cost volume is processed using the Softmax function along the depth dimension D to obtain the probability volume. Based on the probability distribution, the optimal depth value of each pixel is inferred along the D dimension to form the final depth map.

[0031] Step 1 is as follows:

[0032] Let the images of multiple views of a scene be... Select an image as the source image I0, and determine N-1 reference images of the source image I0 from M views using the scoring function shown in formula (1). The higher the score, the more common pixels there are between the images. Select the top N-1 images with higher scores to facilitate feature matching between views and improve the training efficiency of the model.

[0033]

[0034] Where S is the common trajectory between view j and view k, and p is the number of pixels within it. θjk (p) is the angle of pixel p offset from the baseline, as shown in formula (2). G is a Gaussian piecewise function, as shown in formula (3).

[0035]

[0036]

[0037] Among them, c j With c k Let σ1, σ2, and θ0 be the camera centers of views j and k, respectively, and let σ1, σ2, and θ0 be settable threshold parameters.

[0038] After selecting a good source image I0 and N-1 reference images, the N images are fed into the feature extraction module of the network framework for feature extraction. The feature extraction module of this invention consists of a cascaded three-stage attention network. Figure 2 The structure of the single-stage attention network is shown in Table 1. The parameters of the attention network in the three stages are shown in Table 1.

[0039] Table 1. Parameter settings for the three-stage attention network

[0040] stage Number of channels C Stacking times L Output size 1 8 3 H×W×8 2 32 3 H / 4×W / 4×32 3 64 6 H / 8×W / 8×64

[0041] Among them, the Large Kernel Attention (LKA) mechanism is the core component of the attention network. Figure 3 This describes the underlying principle of the LKA (Large Kernel Attention) mechanism. The LKA mechanism consists of grouped convolutions (DW-Conv), deep dilated convolutions (DW-D-Conv), and 1×1 point convolutions (1×1-Conv). This combination of three convolution types allows the model to balance local receptive fields, long-range dependencies, and robust dynamic kernels, significantly enhancing the network's information modeling capabilities. Furthermore, the LKA mechanism boasts linear computational complexity, greatly improving the computational efficiency of 3D reconstruction networks. The computational process of the LKA mechanism is as follows:

[0042] Attention = Conv 1×1 (DW-D-Conv(DW-Conv(F))) (4)

[0043]

[0044] Table 2. Parameter settings for the three-stage LKA mechanism

[0045]

[0046] The parameter settings for the LKA mechanism in the three-stage attention network are shown in Table 2. Finally, the input image... After the feature extraction module, feature maps {A1, A2, A3} at three scales are formed, with sizes and shapes of H×W×8, H / 4×W / 4×32, and H / 8×W / 8×64, respectively.

[0047] Step 2 is as follows:

[0048] After extracting image feature data, spatial transformation between multiple views is performed to achieve feature matching. The network assumes a depth of {d1, d2, ..., d...}. k Under the following conditions, the source image I0 and its N-1 neighboring images are realized through differentiable homography transformation. Spatial transformations between them. The homography transformation matrix is ​​as follows:

[0049]

[0050] Where K i R i , t i Let n be the camera intrinsic parameters, rotation matrix, and translation vector for the i-th viewpoint, respectively. i This refers to the principal axis of the reference camera, H. i (d) refers to the homography transformation between the source image and the reference image at the depth hypothesis value d.

[0051] Figure 4 This is a schematic diagram of the principle of differentiable homography transformation. Under the depth assumption {d1,d2,d3}, the coordinate position of each pixel in the source image in the reference images 1 and 2 can be obtained by using the differentiable homography transformation shown in formula (6).

[0052] Step 3 is as follows:

[0053] The source image I0 is divided into depths d∈{d1,d2,...,d...} k After mapping to N-1 reference image views, the reference image features at the corresponding coordinate positions are extracted to construct the cost body for feature matching. This invention abandons variance-based metrics and instead uses pixel-level feature correlation as a metric to evaluate feature matching. The correlation body construction equation is as follows:

[0054]

[0055] Among them, F i (d) ∈R C×H×W This refers to the matching feature volume of the i-th reference image obtained through homography at depth d, where F0∈R C×H×W It is the feature body of the source image, C i ∈R C×D×H×WLet C be the feature association body between the source image and the i-th reference image, where p is a pixel in the image. To reduce the computational cost of subsequent regularization operations, the association body C is... i Calculate the mean along the channel dimension, i.e., C. i ∈R D×H×W .

[0056] To enable any number of image inputs to the network, this invention employs an adaptive aggregation strategy (AAS) to aggregate N-1 related entities C. i Adaptive aggregation is performed to construct a unified associative body C. Specifically, the equations are constructed using the adaptive aggregation strategy AAS as follows:

[0057]

[0058]

[0059] Among them, W i It is the adaptive aggregation weight of the associated entity, which is learned through a convolutional neural network. Max is the maximum value function, Sigmoid is the activation function, and Conv3d is the 3D convolution function.

[0060] Related body C∈R D×H×W Cost volumes often contain noise, and to remove outliers caused by noise, regularization of the cost volume is necessary. This invention employs Global Adaptive Mean Pooling Attention (GAAPA), following an encoder-decoder structure, to construct a multi-scale regularization module for cost volume regularization.

[0061] Figure 5 This describes the processing flow of the regularization module. The multi-scale regularization module consists of an encoder and a decoder. The encoder uses GAAPA to process the correlation body C∈R. D×H×W The data is gradually compressed along the three dimensions D, H, and W to form correlation volumes at three scales: 1 / 1, 1 / 2, and 1 / 4. The decoder then performs a conventional upsampling operation on the smallest 1 / 4 scale correlation volume C∈R. D / 4×H / 4×W / 4 Recovery is performed sequentially along dimensions D, H, and W, and correlation volumes of the same scale as the encoder are added through skip connections, ultimately forming a noise-free correlation volume S∈R. D×H×W .

[0062] Figure 6 The flowchart for the Global Adaptive Mean Pooling (GAAPA) structure is shown below. First, the related entity C ∈ R... D×H×W By using convolution mapping, transforming it from 3D to 2D, and then changing the dimensional order, it forms... Then, three linear mappings with different parameters are used to form the Q, K, and V matrices, while keeping the Q matrix unchanged. Adaptive mean pooling is used to pool the K and V matrices to compress the space size. Finally, channel grouping, dot product, channel aggregation, and linear mapping are performed. The GAAPA formula is as follows:

[0063]

[0064]

[0065]

[0066]

[0067] Where x is the input These represent the height, width, and number of channels of the input token, respectively. N i d represents the number of groups. head The number of channels in each group is equivalent to p h p w This refers to the spatial size after adaptive pooling. The weights are linearly mapped. Attention is the dot product operation paradigm of the attention mechanism, Conv2d is a 2D convolution function, Pooling2d is a 2D adaptive mean pooling function, and ReLU is the activation function.

[0068] Table 3. Three-Scale GAAPA Parameter Settings

[0069]

[0070] Table 3 shows the specific parameter settings for the Global Adaptive Mean Pooling Attention (GAAPA) at three scales in the multi-scale regularization module. The regularization module, consisting of the 1 / 1, 1 / 2, and 1 / 4 scales of GAAPA, can effectively filter out outliers in the cost volume, and the adaptive mean pooling does not cause excessive smoothing of the cost volume, which is beneficial for depth map inference in step 4.

[0071] Step 4 is as follows:

[0072] In the cost body C∈R D×H×W After regularization, a noise-free cost volume S is formed. ` ∈R D×H×W The Softmax function is applied in the depth dimension D to obtain the probability volume P∈R. D×H×WThen, the depth value of each pixel in the image is inferred based on the probability distribution of the depth dimension. This invention adopts a winner-take-all strategy, that is, the depth with the highest probability value is used as the value in the final inferred depth map.

[0073] In step 1, this invention forms three-stage feature maps {A1, A2, A3}, mainly performing the final depth map inference in three stages from coarse to fine; when A3∈R H / 8×W / 8×64 After steps 2, 3, and 4, a rough depth map D3∈R is obtained. H / 8×W / 8 D3∈R H / 8×W / 8 The depth is restored to H / 4×W / 4 using bilinear interpolation. Each depth value in the depth map inferred from D3 is used as the new hypothetical center. Smaller depth intervals are then set, and the calculation of the differentiable homography transformation is redefined.

[0074]

[0075] in, It is the depth predicted by the m-th pixel in the source image at the k-th stage. This represents the remaining depth of the m-th pixel that the network needs to learn in the (k+1)-th stage, and is used to set parameters. After steps 3 and 4, a relatively detailed depth map D2∈R is obtained. H / 4×W / 4 Similarly, the final depth map D1∈R, which has the same size as the source image, is... H×W The same applies to the inference. In this way, through the three-stage depth map inference process from low resolution to high resolution, the quality of the final depth map is greatly improved.

[0076] Furthermore, after obtaining the depth maps of the scene from multiple views through the network, the depth maps are fused using a visibility depth fusion strategy, combining the relevant RGB information of the multiple view images to form the final 3D point cloud output.

[0077] The attention-based 3D reconstruction network framework built in this invention was trained on the public dataset DTU. Figure 7 The 3D reconstruction results of the attention-based 3D reconstruction framework of the present invention on the DTU test set (photographed from a fixed viewpoint) are presented. Figure 8 The 3D reconstruction framework of this invention is compared with the 3D point cloud output results of some existing frameworks.

[0078] The quantitative comparison results of the three-dimensional reconstruction results of this invention with other common existing methods are shown in Table 4.

[0079] Table 4 Quantitative Comparison of 3D Reconstruction Results

[0080]

[0081] Table 4 presents a quantitative comparison of the reconstruction results of the 3D reconstruction framework of this invention with some existing frameworks. Smaller errors indicate better framework performance. Experiments show that the 3D reconstruction network framework of this invention can perform efficient and high-quality 3D scene reconstruction, especially in complex reconstruction scenes, such as low-texture and repetitive texture areas, specular reflection areas, and other scenes with information ambiguity. The network framework of this invention can also effectively complete the 3D scene reconstruction. Compared with some previous reconstruction frameworks, the attention-based 3D reconstruction framework of this invention improves reconstruction quality and efficiency. Whether from the final output 3D point cloud representation or from the quantitative evaluation results, the 3D reconstruction framework of this invention has highly competitive performance, especially in terms of reconstruction completeness, where it achieves optimal performance.

[0082] The above embodiments are merely preferred embodiments of the present invention and should not be considered as limiting the scope of the present invention. All equivalent variations and improvements made within the scope of the present invention should still fall within the patent coverage of the present invention.

Claims

1. A three-dimensional reconstruction method based on a large kernel attention mechanism, characterized in that Includes the following steps: S1. Feature Extraction: Based on the multi-scale feature extraction strategy, the feature extraction of N multi-view images is performed through the large kernel attention (LKA) mechanism to form feature maps at three scales, with resolutions of 1 / 1, 1 / 4, and 1 / 8 of the original input image, respectively. S2. Differentiable homography transformation: Under a series of assumed depth values, spatial transformation between multiple views is achieved through homography transformation to obtain the pixel correspondence between the source image and multiple reference images; S3. Correlation Entity Construction and Correlation Entity Regularization: After obtaining the positions of the source image pixels in N-1 reference images, the feature information of the corresponding positions is extracted respectively, and N-1 correlation entities are constructed using feature correlation. Then, an adaptive aggregation strategy (AAS) based on convolutional neural network (CNN) is used to construct a unified correlation entity. Finally, global adaptive mean pooling (GAAPA) is used for regularization. The correlation body regularization, used to remove outliers caused by noise, specifically involves the following steps: A multi-scale regularization module is constructed using Global Adaptive Mean Pooling Attention (GAAPA) following an encoder-decoder structure for correlation body regularization. The specific process for correlation body regularization is as follows: The encoder uses GAAPA to correlate the body The correlation volume is gradually compressed along the three dimensions D, H, and W to form correlation volumes at three scales: 1 / 1, 1 / 2, and 1 / 4. The decoder then performs a conventional upsampling operation on the smallest 1 / 4 scale correlation volume. The data is recovered sequentially along dimensions D, H, and W, and correlation volumes of the same scale from the encoder are added via skip connections to ultimately form a noise-free correlation volume. ; GAAPA specifically refers to: Related entities By using convolution mapping, transforming it from 3D to 2D, and then changing the dimensional order, it forms... Then, three linear mappings with different parameters are used to form Q, K, and V matrices, while keeping the Q matrix unchanged. Adaptive mean pooling is used to pool the K and V matrices to compress the space size. Finally, channel grouping, dot product, channel aggregation, and linear mapping are performed. S4. Depth map inference: Treating depth map inference as a classification task, the regularized association body is used to obtain the probability body in the depth dimension D using the Softmax function. Based on the probability distribution, the optimal depth value of each pixel is inferred along the D dimension to form the final depth map. S5. 3D Reconstruction: A visibility-based fusion algorithm is used for deep fusion. The depth maps of the multiple views inferred by the network are combined with the RGB information of the multiple view images to write the final 3D point cloud output. The 3D representation of the scene is reconstructed from the 2D images of the scene from multiple perspectives.

2. The 3D reconstruction method based on the big kernel attention mechanism as described in claim 1, characterized in that... In step 1, the specific steps of feature extraction are as follows: Let the images of multiple views of a scene be... Randomly select an image as the source image. The source image is determined from the remaining M views using the scoring function shown in formula (1). N-1 reference images A higher score indicates more common pixels between images; the top N-1 images with higher scores are selected to facilitate feature matching between views and improve model training efficiency; the entire reconstruction process uses a randomly determined source image at a time. Based on the information from its N-1 reference images, the source image is inferred. Depth map, then from the remaining multi-view images Select a new image as the source image. And redetermine its N-1 reference images, and continue to infer the source image. The depth map, until exhaustion All images in the image are used as source images; (1) Where S is the common trajectory between view j and view k, and p is the pixel point in it; is the angle of pixel p offset from the baseline, as shown in formula (2); G is the Gaussian piecewise function, as shown in formula (3); (2) (3) in, and Let j be the camera center of view and k. , and This is a configurable threshold parameter; Select a good source image After N-1 reference images, the N images are fed into the feature extraction module in the network framework for feature extraction; The feature extraction module consists of three cascaded attention networks. Each attention network comprises a normalization layer, an attention layer, and a feedforward network layer, with layer skipping connections to enhance the network's robustness. The normalization layer normalizes the encoded data to accelerate the convergence of the network model. The attention layer performs attention encoding on the data and consists of a 1×1 point convolution, a GELU activation function, an LKA large kernel attention mechanism, and another 1×1 point convolution. The feedforward network layer further encodes the data using convolution and consists of a 1×1 point convolution, a 3×3 grouped convolution, a GELU activation function, and another 1×1 point convolution.

3. The 3D reconstruction method based on the big kernel attention mechanism as described in claim 2, characterized in that... The LKA large kernel attention mechanism is the core of the attention network, consisting of grouped convolutions (DW-Conv), deep dilated convolutions (DW-D-Conv), and 1×1 point convolutions (1×1-Conv). This combination of three convolution types allows the model to balance local receptive fields, long-range dependencies, and robust dynamic kernels, significantly enhancing the network's information modeling capabilities. Simultaneously, the LKA mechanism possesses linear computational complexity, improving the computational efficiency of the 3D reconstruction network. The computation process of the LKA large kernel attention mechanism is as follows: (4) (5) Input image After the feature extraction module, feature maps at three scales are generated. .

4. The 3D reconstruction method based on the large kernel attention mechanism as described in claim 1, characterized in that... In step S2, the specific steps of the differentiable homography transformation are as follows: After extracting image feature data, spatial transformation between multiple views is performed to achieve feature matching; the network operates on a series of hypothetical depths. Below, the source image is realized through differentiable homography transformation. and Neighborhood images Spatial transformations between them; the homography transformation matrix is ​​as follows: (6) in , , Let be the camera intrinsic parameters, rotation matrix, and translation vector for the i-th viewpoint, respectively. This refers to the main axis of the reference camera. This refers to the homography transformation between the source image and the reference image at the depth hypothesis value d.

5. The 3D reconstruction method based on the large kernel attention mechanism as described in claim 1, characterized in that... In step S3, the construction of the associated entity is as follows: Source image At depth After mapping to N-1 reference image views, the features of the reference images at the corresponding coordinate positions are extracted to construct a feature matching association body. Pixel-level feature correlation is used as a metric to evaluate feature matching. The association body construction equation is as follows: (7) in, This refers to the matching feature body of the i-th reference image obtained through homography at depth d. It is the feature body of the source image. Let p be a pixel in the source image and the i-th reference image, representing the feature association between them. To reduce the computational cost of subsequent regularization operations, the feature association is... Calculate the mean along the channel dimension, i.e. ; To enable the network to accept any number of image inputs, an adaptive aggregation strategy (AAS) is used to aggregate N-1 related entities. Adaptive aggregation is performed to construct a unified associative body C; the adaptive aggregation strategy AAS construction equation is as follows: (8) (9) in, These are the adaptive aggregation weights of the associated entities, learned through a convolutional neural network. Max is the maximum value function, Sigmoid is the activation function, and Conv3d is the 3D convolution function.

6. The 3D reconstruction method based on the large kernel attention mechanism as described in claim 1, characterized in that... In step S4, the specific steps of the depth map inference are as follows: Related entities After regularization, a noise-free correlated body is formed. Then, the Softmax function is applied in the depth dimension D to obtain the probability volume. The depth values ​​of each pixel in the image are inferred based on the probability distribution of the depth dimension; a winner-take-all strategy is adopted, that is, the depth with the highest probability value is used as the value in the final inferred depth map; feature maps in three stages. The final depth map inference is performed in three stages, from coarse to fine.