Method for carrying out multi-view feature extraction by utilizing lightweight Transform-CNN (Convolutional Neural Network) fusion network

Abstract

The invention provides a method for performing multi-view feature extraction by using a lightweight Transform-CNN (Convolutional Neural Network) fusion network, and the method comprises the following steps: constructing the lightweight Transform-CNN fusion network which comprises a Transform encoder, a feature pyramid network (FPN) and a feature fusion module; the method comprises the following steps: extracting global features of an input image through a Transform encoder; multi-scale local features of the image are extracted through FPN; and fusing the global features and the local features through a feature fusion module to generate a fused feature map. The method provided by the invention has remarkable technical effects in the aspects of global and local feature extraction, feature fusion, consistency of multi-view feature extraction, three-dimensional reconstruction quality, robustness, generalization ability and the like. The method provides an efficient, accurate and robust solution for application scenes such as three-dimensional reconstruction, and has important practical value and research significance.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and deep learning technology, and specifically to a method for multi-view feature extraction using a lightweight Transformer-CNN fusion network. The method achieves comprehensive feature capture of an image from global to local by fusing the global feature representation capability of a Transformer and the local feature extraction capability of a Feature Pyramid Network (FPN). The method is particularly suitable for three-dimensional reconstruction tasks in complex scenes, and significantly improves reconstruction effect and computational efficiency. Background Art

[0002] In the field of computer vision and graphics, 3D reconstruction is an important research topic, which aims to restore the 3D shape and structure of an object from a 2D image. With the rapid development of deep learning technology, especially the emergence of advanced models such as convolutional neural networks (CNN) and Transformer, 3D reconstruction technology has made significant progress.

[0003] However, traditional 3D reconstruction methods often face many challenges. On the one hand, the extraction of global features is crucial to understanding the overall structure and shape in an image, but traditional CNN models have limitations in processing global information and are unable to efficiently capture long-range dependencies in images. On the other hand, the extraction of local features is equally important for restoring detailed information in an image, but how to achieve the extraction of multi-scale local features while maintaining computational efficiency has always been a difficult problem.

[0004] In order to overcome these challenges, methods have been explored to combine Transformer with CNN. The Transformer model is well known for its powerful global feature extraction capabilities and self-attention mechanism, while CNN is good at extracting local features and capturing image detail information. Therefore, combining the two is expected to achieve efficient extraction and accurate fusion of global and local features.

[0005] Although there have been some attempts to combine Transformer with CNN for 3D reconstruction, these methods often have problems such as large amount of computation, high model complexity, and inaccurate feature fusion. Especially when processing multi-view images, how to ensure the consistency of features in different views and achieve high-quality 3D reconstruction is still an urgent problem to be solved.

[0006] To this end, a method for multi-view feature extraction using a lightweight Transformer-CNN fusion network is proposed. Summary of the invention

[0007] The present invention aims to solve the problems raised in the background technology and provides a method for multi-view feature extraction using a lightweight Transformer-CNN fusion network. The method combines the global feature representation capability of Transformer and the local feature extraction capability of FPN to achieve comprehensive feature capture of the image from global to local, thereby improving the accuracy and efficiency of 3D reconstruction.

[0008] The specific technical solutions are as follows:

[0009] A method for multi-view feature extraction using a lightweight Transformer-CNN fusion network includes the following steps:

[0010] Construct a lightweight Transformer-CNN fusion network, which includes a Transformer encoder, a feature pyramid network (FPN), and a feature fusion module;

[0011] Extract global features of the input image through the Transformer encoder;

[0012] Extract multi-scale local features of images through FPN;

[0013] The global features are fused with the local features through the feature fusion module to generate a fused feature map.

[0014] The above-mentioned method for multi-view feature extraction using a lightweight Transformer-CNN fusion network, wherein: the Transformer encoder includes multiple self-attention layers and feedforward neural network layers to capture long-range dependencies in the image.

[0015] The above-mentioned method for multi-view feature extraction using a lightweight Transformer-CNN fusion network, wherein: the FPN includes multiple upsampling layers, downsampling layers and feature fusion layers, which are used to construct a feature pyramid and realize multi-scale feature fusion.

[0016] The above-mentioned method for multi-view feature extraction using a lightweight Transformer-CNN fusion network, wherein: the feature fusion module fuses global features with local features by weighted summation or splicing.

[0017] The above-mentioned method for multi-view feature extraction using a lightweight Transformer-CNN fusion network, wherein: it also includes the step of applying the lightweight Transformer-CNN fusion network to the input multi-view images for feature extraction.

[0018] The above-mentioned method for multi-view feature extraction using a lightweight Transformer-CNN fusion network, wherein: it also includes the step of optimizing the inter-view consistency constraints of the extracted multi-view feature maps.

[0019] The above-mentioned method for multi-view feature extraction using a lightweight Transformer-CNN fusion network, wherein: it also includes the step of using the extracted multi-view feature map to perform three-dimensional reconstruction, wherein the three-dimensional reconstruction uses an optimization algorithm such as volume rendering or point cloud fusion.

[0020] The present invention also provides a lightweight Transformer-CNN fusion network, comprising: a Transformer encoder, a feature pyramid network (FPN) and a feature fusion module, for implementing any of the above-mentioned methods for multi-view feature extraction using a lightweight Transformer-CNN fusion network.

[0021] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned methods for extracting multi-view features using a lightweight Transformer-CNN fusion network.

[0022] The present invention also provides a computer device, including a processor and a memory, wherein a computer program is stored in the memory, and when the processor executes the computer program, any one of the above-mentioned methods for multi-view feature extraction using a lightweight Transformer-CNN fusion network is implemented.

[0023] The present invention has the following beneficial effects:

[0024] The method for multi-view feature extraction and 3D reconstruction using a lightweight Transformer-CNN fusion network proposed in the present invention has the following technical effects:

[0025] Efficient global and local feature extraction:

[0026] Through the Transformer encoder, this method can efficiently capture long-distance dependencies in images and extract global features.

[0027] By using the Feature Pyramid Network (FPN), multi-scale local feature extraction is achieved, which enhances the model's ability to capture image detail information.

[0028] Accurate feature fusion:

[0029] The feature fusion module effectively fuses global features with local features by weighted summation or splicing, thereby improving the accuracy and robustness of the features.

[0030] The introduction of attention mechanisms, such as channel attention or spatial attention, further enhances the expressiveness of features.

[0031] Consistency of multi-view feature extraction:

[0032] A lightweight Transformer-CNN fusion network is applied to the input multi-view images for feature extraction, ensuring the consistency of features in different views.

[0033] Through the optimization of inter-view consistency constraints, the consistency of multi-view features in the feature space is further improved, providing a reliable foundation for 3D reconstruction.

[0034] High-quality 3D reconstruction:

[0035] The extracted multi-view feature maps are used for 3D reconstruction, and optimization algorithms such as volume rendering or point cloud fusion are adopted to achieve high-quality 3D model generation.

[0036] The volume rendering algorithm generates realistic 3D model rendering images by calculating the propagation path of light in 3D space and its interaction with objects.

[0037] The point cloud fusion algorithm matches and fuses the feature points of multiple views to generate accurate three-dimensional point cloud data, and then constructs a high-quality three-dimensional model.

[0038] Good robustness and generalization ability:

[0039] This method is highly robust to changes in illumination and perspective, and can handle 3D reconstruction problems in complex scenes.

[0040] By introducing sparse self-attention, window self-attention mechanism and regularization methods, the generalization ability and training stability of the model are improved.

[0041] Efficient training and evaluation:

[0042] Use deep learning frameworks such as PyTorch to implement and train models, and use GPU acceleration to improve training efficiency.

[0043] Reasonable hyperparameters and training strategies are set to ensure the stability and convergence of the model.

[0044] The performance of the model was comprehensively evaluated by combining quantitative and qualitative evaluation.

[0045] In summary, the method for multi-view feature extraction using a lightweight Transformer-CNN fusion network proposed in the present invention aims to achieve efficient extraction and accurate fusion of global and local features, while improving the consistency of multi-view features, and providing a reliable foundation for three-dimensional reconstruction. This method realizes the extraction and fusion of global and local features of the input image by constructing a lightweight Transformer encoder, a feature pyramid network (FPN) and a feature fusion module, and then uses the extracted multi-view feature map to perform three-dimensional reconstruction. This method not only improves the quality and efficiency of three-dimensional reconstruction, but also enhances the robustness and generalization ability of the model, and has important practical value and research significance. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of the architecture of a lightweight Transformer-CNN fusion network provided in an embodiment of the present invention;

[0047] Figure 2 A flowchart of a method for multi-view feature extraction using a lightweight Transformer-CNN fusion network provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The technical solution of the present invention is further described below with reference to the accompanying drawings and through specific implementation methods.

[0049] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on this patent; in order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0050] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "inner", "outer", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as a limitation on this patent. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0051] In the description of the present invention, unless otherwise clearly specified and limited, if the term "connection" or the like appears to indicate the connection relationship between components, the term should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two components or the interaction relationship between two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0052] Example

[0053] This embodiment provides a method for extracting multi-view features using a lightweight Transformer-CNN fusion network. Figure 1-2 As shown, the specific steps include:

[0054] S1: Construct a lightweight Transformer-CNN fusion network, which includes a Transformer encoder, a feature pyramid network (FPN), and a feature fusion module, where:

[0055] Transformer Encoder

[0056] The Transformer encoder consists of multiple self-attention layers and feed-forward neural network layers to capture long-range dependencies in the image. The self-attention mechanism effectively captures global information by calculating the correlation scores between different positions in the input sequence.

[0057] The calculation formula of the self-attention mechanism is as follows:

[0058]

[0059] Where Q, K, and V represent query, key, and value matrices respectively. k Indicates the dimension of the key.

[0060] A feedforward neural network layer usually consists of two linear transformations and a nonlinear activation function, which is calculated as follows:

[0061] textFFN(x)=max(0,xW1+b1)W2+b2

[0062] Among them, W1, b1, W2, and b2 are the weights and biases of the linear transformation, respectively, and max(0,·) represents the ReLU activation function.

[0063] Among them, the Transformer encoder can introduce position encoding or relative position embedding to enhance the Transformer encoder's ability to capture spatial position information in the image.

[0064] Consider using sparse self-attention or window self-attention mechanisms to reduce the amount of computation while maintaining the ability to capture long-distance dependencies;

[0065] Regularization methods such as LayerNorm and Dropout, as well as residual connections, can be added to improve the generalization ability and training stability of the model.

[0066] Feature Pyramid Network (FPN)

[0067] FPN constructs a feature pyramid and realizes multi-scale feature fusion through multiple upsampling layers, downsampling layers and feature fusion layers. The upsampling layer enlarges the feature map to a higher resolution through the interpolation algorithm, and the downsampling layer reduces the feature map to a lower resolution through the convolution operation. The feature fusion layer fuses feature maps of different scales by splicing or weighted summation.

[0068] The calculation formula for feature fusion is as follows (taking the splicing method as an example):

[0069] F fused =Concat(F1,F2,...,F N )

[0070] Among them, F1, F2, …, F N Represents feature maps of different scales, and Concat represents a concatenation operation.

[0071] Among them, an adaptive weight allocation mechanism can be introduced in FPN to dynamically adjust the fusion weights of features of different scales according to the importance of the features; in addition to the traditional top-down and bottom-up paths, lateral connections can be added to better fuse feature information of different scales.

[0072] Feature fusion module

[0073] The feature fusion module fuses global features with local features by weighted summation or splicing. The weighted summation method calculates the weights of global features and local features and sums them according to the weights; the splicing method directly splices global features and local features together.

[0074] The calculation formula for weighted sum is as follows:

[0075] F fused =αF global +βF local

[0076] Among them, Fglobal represents the global feature, F local represents the local features, α and β are the weights of the global features and local features respectively.

[0077] Among them, attention mechanisms, such as channel attention or spatial attention, can be used in the feature fusion module to enhance the expressiveness of features.

[0078] If the input needs to contain multiple modal information (such as depth, optical flow, etc.), a multimodal fusion strategy can be designed to effectively integrate the features of different modalities.

[0079] S2: Extracting global features

[0080] The global feature extraction of the input image is performed through the Transformer encoder. The input image is converted into a sequence form and input into the Transformer encoder. After being processed by multiple self-attention layers and feedforward neural network layers, a global feature map is obtained.

[0081] The function definition of global feature extraction is as follows:

[0082] F global = TransformerEncoder(I)

[0083] Among them, I represents the input image and TransformerEncoder represents the function of the Transformer encoder.

[0084] S3: Extracting multi-scale local features

[0085] The input image is input into FPN and processed by multiple upsampling layers, downsampling layers and feature fusion layers to obtain local feature maps of different scales.

[0086] The function definition of multi-scale local feature extraction is as follows:

[0087] {F_{\text{local}}^1,F_{\text{local}}^2,\ldots,F_{\text{local}}^N\}=\text{FPN}(I)

[0088] Where I represents the input image, FPN represents the function of the feature pyramid network, Represents local feature maps of different scales.

[0089] S4: Feature Fusion

[0090] The global features are fused with the local features through the feature fusion module. The global features are fused with the local features of different scales by weighted summation or splicing to obtain a fused feature map.

[0091] The function definition of feature fusion is as follows (taking weighted summation as an example):

[0092]

[0093] Among them, F global Represents global features represents the local features of the i-th scale, α i and β i are the weights of the global features and the i-th scale local features respectively.

[0094] S5: Multi-view feature extraction and consistency constraint optimization

[0095] A lightweight Transformer-CNN fusion network is applied to the input multi-view images for feature extraction. The image of each view is input into the network to obtain the fused feature map of each view.

[0096] The function definition of multi-view feature extraction is as follows:

[0097] {F_{\text{fused}}^1,F_{\text{fused}}^2,\ldots,F_{\text{fused}}^M\}=

[0098] \{\text{LightweightTransformerCNN}(I^1),

[0099] \text{LightweightTransformerCNN}(I^2),\ldots,

[0100] \text{LightweightTransformerCNN}(I^M)\}

[0101] Among them, I 1 ,I 2 ,…,I M represents a multi-view image, LightweightTransformerCNN represents the function of the lightweight Transformer-CNN fusion network, Represents the fused feature map of each view.

[0102] In order to optimize the consistency of multi-view features, the inter-view consistency constraint is introduced. By calculating the similarity or distance between feature maps of different views, the feature maps are optimized so that the feature maps of different views are closer in the feature space.

[0103] The objective function of inter-view consistency constraint optimization is defined as follows:

[0104]

[0105] Among them, θ represents the parameters of the network, Distance represents the function of calculating the distance between two feature maps,

[0106] and Represent the fused feature maps of the i-th and j-th views respectively.

[0107] Among them, a multi-view processing strategy can be added

[0108] a. View selection and preprocessing

[0109] View filtering: Based on factors such as image quality and perspective diversity, the input multi-view images are filtered to reduce redundant information.

[0110] View enhancement: Enhance the view, such as data enhancement and image restoration, to improve the robustness of features.

[0111] b. Consistency constraints between views

[0112] Loss function design: Design reasonable loss functions, such as contrast loss and triplet loss, to constrain the consistency of features between different views.

[0113] Consistency Regularization: A consistency regularization term is introduced to encourage features extracted from different views to be semantically consistent.

[0114] S6: 3D reconstruction

[0115] The extracted multi-view feature map is used for 3D reconstruction. Optimization algorithms such as volume rendering or point cloud fusion are used to generate a 3D model based on the multi-view feature map.

[0116] The volume rendering algorithm generates a rendered image of the 3D model by calculating the propagation path of light in 3D space and its interaction with objects. The point cloud fusion algorithm matches and fuses the feature points of multiple views to generate 3D point cloud data and then construct a 3D model.

[0117] The function definition of 3D reconstruction is as follows (taking volume rendering as an example):

[0118]

[0119] in, Represents the multi-view fusion feature map, C represents the internal and external parameters of the camera, VolumeRendering represents the volume rendering function, and 3DModel represents the generated 3D model.

[0120] Among them, 3D reconstruction optimization can also be performed

[0121] a. Volume rendering improvements

[0122] Sampling strategy optimization: Adopt more efficient sampling strategies, such as adaptive sampling or importance sampling, to improve the accuracy and efficiency of volume rendering.

[0123] Material and lighting models: Introducing more complex material and lighting models to simulate the appearance of objects more realistically.

[0124] b. Point cloud fusion strategy

[0125] Point cloud registration: Use ICP (Iterative Closest Point) or other registration algorithms to accurately align point clouds generated by different views.

[0126] Point cloud denoising and simplification: denoise the point cloud and simplify the number of point clouds to reduce the amount of calculation and storage space.

[0127] Training and evaluation

[0128] a. Training strategy

[0129] Step-by-step training: Pre-train the Transformer encoder and FPN first, and then jointly train the entire fusion network to improve training efficiency and effect.

[0130] Transfer learning: Use pre-trained models for transfer learning to speed up the training process of new tasks.

[0131] b. Evaluation Metrics

[0132] Quantitative evaluation: Use quantitative indicators such as 3D reconstruction accuracy and feature matching rate to evaluate model performance.

[0133] Qualitative evaluation: Qualitative evaluation is performed by visualizing reconstruction results, feature maps, etc. to intuitively demonstrate the effect of the model.

[0134] Through the above-mentioned refined and improved scheme, the accuracy and efficiency of multi-view feature extraction using the lightweight Transformer-CNN fusion network can be further improved, while enhancing its practicality and robustness in application scenarios such as 3D reconstruction.

[0135] S7: Experimental Results and Analysis

[0136] In order to verify the effectiveness of this method, a large number of experiments were conducted. In this embodiment, multiple public multi-view datasets were used for training and testing in the experiment. These datasets cover different scenes and object categories, with high diversity and challenges. In the experiment, this embodiment uses deep learning frameworks such as PyTorch to implement and train the model, and uses GPU acceleration to improve training efficiency. At the same time, this embodiment also sets reasonable hyperparameters and training strategies to ensure the stability and convergence of the model. The experimental dataset includes multiple image sequences with different viewing angles and lighting conditions. Through comparative experiments, the performance of this method in terms of three-dimensional reconstruction accuracy, computational efficiency and robustness is evaluated.

[0137] Experimental results show that this method has achieved remarkable results in multi-view feature extraction and 3D reconstruction. Compared with traditional 3D reconstruction methods, this method has higher reconstruction accuracy and faster calculation speed. At the same time, this method is highly robust to changes in illumination and perspective, and can handle 3D reconstruction problems in complex scenes.

[0138] S8: Conclusion

[0139] This embodiment proposes a method for multi-view feature extraction and 3D reconstruction using a lightweight Transformer-CNN fusion network. This method combines the global feature extraction capability of Transformer and the multi-scale local feature extraction capability of CNN, and effectively combines the two through a feature fusion module, thereby improving the accuracy and robustness of feature extraction. Furthermore, the extracted multi-view feature map is used for 3D reconstruction, and optimization algorithms such as volume rendering or point cloud fusion are used to achieve high-quality 3D model generation.

[0140] In summary: The method for multi-view feature extraction and 3D reconstruction using a lightweight Transformer-CNN fusion network proposed in the present invention has the following technical effects:

[0141] Efficient global and local feature extraction:

[0142] Through the Transformer encoder, this method can efficiently capture long-distance dependencies in images and extract global features.

[0143] By using the Feature Pyramid Network (FPN), multi-scale local feature extraction is achieved, which enhances the model's ability to capture image detail information.

[0144] Accurate feature fusion:

[0145] The feature fusion module effectively fuses global features with local features by weighted summation or splicing, thereby improving the accuracy and robustness of the features.

[0146] The introduction of attention mechanisms, such as channel attention or spatial attention, further enhances the expressiveness of features.

[0147] Consistency of multi-view feature extraction:

[0148] A lightweight Transformer-CNN fusion network is applied to the input multi-view images for feature extraction, ensuring the consistency of features in different views.

[0149] Through the optimization of inter-view consistency constraints, the consistency of multi-view features in the feature space is further improved, providing a reliable foundation for 3D reconstruction.

[0150] High-quality 3D reconstruction:

[0151] The extracted multi-view feature maps are used for 3D reconstruction, and optimization algorithms such as volume rendering or point cloud fusion are adopted to achieve high-quality 3D model generation.

[0152] The volume rendering algorithm generates realistic 3D model rendering images by calculating the propagation path of light in 3D space and its interaction with objects.

[0153] The point cloud fusion algorithm matches and fuses the feature points of multiple views to generate accurate three-dimensional point cloud data, and then constructs a high-quality three-dimensional model.

[0154] Good robustness and generalization ability:

[0155] This method is highly robust to changes in illumination and perspective, and can handle 3D reconstruction problems in complex scenes.

[0156] By introducing sparse self-attention, window self-attention mechanism and regularization methods, the generalization ability and training stability of the model are improved.

[0157] Efficient training and evaluation:

[0158] Use deep learning frameworks such as PyTorch to implement and train models, and use GPU acceleration to improve training efficiency.

[0159] Reasonable hyperparameters and training strategies are set to ensure the stability and convergence of the model.

[0160] The performance of the model was comprehensively evaluated by combining quantitative and qualitative evaluation.

[0161] Therefore, the method proposed in the present invention has achieved remarkable technical effects in global and local feature extraction, feature fusion, consistency of multi-view feature extraction, 3D reconstruction quality, robustness and generalization ability. This method provides an efficient, accurate and robust solution for application scenarios such as 3D reconstruction, and has important practical value and research significance.

[0162] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for multi-view feature extraction using a lightweight Transformer-CNN fusion network, characterized in that: The following steps are involved: Construct a lightweight Transformer-CNN fusion network, which includes a Transformer encoder, a feature pyramid network FPN, and a feature fusion module; Extract global features of the input image through the Transformer encoder; Extract multi-scale local features of the image through the feature pyramid network FPN; The global features are fused with the local features through the feature fusion module to generate a fused feature map.

2. The method for multi-view feature extraction using a lightweight Transformer-CNN fusion network according to claim 1, characterized in that: The Transformer encoder consists of multiple self-attention layers and feed-forward neural network layers to capture long-range dependencies in images.

3. The method for multi-view feature extraction using a lightweight Transformer-CNN fusion network according to claim 2, characterized in that: The FPN includes multiple upsampling layers, downsampling layers and feature fusion layers, which are used to construct a feature pyramid and realize multi-scale feature fusion.

4. The method for multi-view feature extraction using a lightweight Transformer-CNN fusion network according to claim 3, characterized in that: The feature fusion module fuses the global features with the local features by weighted summation or concatenation.

5. The method for multi-view feature extraction using a lightweight Transformer-CNN fusion network according to claim 4, characterized in that: The method also includes the step of applying the lightweight Transformer-CNN fusion network to the input multi-view images to perform feature extraction.

6. The method for multi-view feature extraction using a lightweight Transformer-CNN fusion network according to claim 5, characterized in that: The method also includes a step of optimizing the inter-view consistency constraint of the extracted multi-view feature map.

7. The method for multi-view feature extraction using a lightweight Transformer-CNN fusion network according to claim 6, characterized in that: The method also includes a step of performing three-dimensional reconstruction using the extracted multi-view feature map, wherein the three-dimensional reconstruction adopts volume rendering or a point cloud fusion optimization algorithm.

8. A lightweight Transformer-CNN fusion network, characterized in that: include: The Transformer encoder, feature pyramid network FPN and feature fusion module are used to implement the method for multi-view feature extraction using a lightweight Transformer-CNN fusion network as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the program is executed by a processor, the method for extracting multi-view features using a lightweight Transformer-CNN fusion network as described in any one of claims 1 to 7 is implemented.

10. A computer device, characterized in that: The invention comprises a processor and a memory, wherein a computer program is stored in the memory, and when the processor executes the computer program, the method for extracting multi-view features using a lightweight Transformer-CNN fusion network as described in any one of claims 1 to 7 is implemented.

Images