A two-stage monocular view reconstruction method based on adaptive multi-scale feature fusion

Abstract

This invention discloses a two-stage single-view reconstruction method based on adaptive multi-scale feature fusion, belonging to the field of computer vision technology. The method includes the following steps: S1. Input a two-dimensional image and extract multi-scale, multi-level two-dimensional image features through a ResNet encoder; S2. Gradually fuse the two-dimensional image features from shallow to deep layers in pairs, and input them into an adaptive multi-scale feature fusion module to generate fused features; S3. Input the fused features generated in step S2 and a uniformly sampled spherical point cloud into a first-stage point cloud deformation module to generate a first-stage reconstructed point cloud; S4. Input the first-stage reconstructed point cloud and the fused features generated in step S2 into a second-stage point cloud deformation module to generate the final single-view point cloud reconstruction result. This invention solves the technical problem that existing methods rely on single-scale semantic features, resulting in a single semantic style and a lack of diversity in the reconstruction results.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a two-stage single-view reconstruction method based on adaptive multi-scale feature fusion. Background Technology

[0002] 3D reconstruction is a crucial task in computer vision, with wide applications in virtual reality, autonomous driving, and other fields. Single-view 3D reconstruction aims to solve the problem of reconstructing 3D models from a single 2D image. Traditional 3D reconstruction typically relies on multi-view images or depth sensors, while single-view methods require only a single image, significantly lowering the data acquisition threshold. Early methods largely depended on handcrafted features and geometric priors, resulting in limited generalization ability. In recent years, with the development of deep learning, especially the introduction of techniques such as convolutional neural networks and generative adversarial networks, single-view 3D reconstruction has made significant progress in shape prediction.

[0003] Because single-view input inherently suffers from severe loss of depth information and occlusion ambiguity, existing reconstruction methods that map 2D features to 3D geometry mostly focus on optimizing the overall shape of the reconstructed model, striving to achieve a close fit between the model outline and the real object. However, they generally neglect the accurate modeling of local fine structures, resulting in significant limitations in the perception and accurate reconstruction of local details. At the same time, existing technical solutions all adopt preset fixed feature extraction strategies. This mode tends to make the reconstruction process overly dependent on feature information at a single semantic level. The detailed texture information contained in shallow features is often missed. Although deep features have rich category semantic information, their spatial detail representation is relatively coarse. This directly leads to the reconstruction model's inability to achieve an adaptive balance between the overall structural accuracy guaranteed by deep semantics and the local style realism carried by shallow details. Ultimately, the output 3D model tends to be the average style of similar objects, lacking personalization and diversity, and failing to meet the practical application requirements of high-precision and diverse 3D reconstruction. Summary of the Invention

[0004] To overcome the shortcomings of the prior art, the present invention aims to provide a two-stage single-view reconstruction method based on adaptive multi-scale feature fusion. This method addresses the technical problem of insufficient geometric detail reconstruction accuracy in existing single-view 3D reconstruction methods by using two-stage reconstruction and feature fusion to supplement details. Furthermore, by extracting multi-scale feature information and generating diverse style features to guide reconstruction, this method solves the technical problem of existing methods relying on single-scale semantic features, resulting in a single semantic style and lack of diversity in reconstruction results.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows: A two-stage single-view reconstruction method based on adaptive multi-scale feature fusion includes the following steps: S1: Input a two-dimensional image and extract multi-scale, multi-level two-dimensional image features through a ResNet encoder; S2: The two-dimensional image features are progressively fused from shallow to deep layers in a pairwise manner, and then input into the adaptive multi-scale feature fusion module to generate fused features; S3: Input the fusion features generated in step S2 and the uniformly sampled spherical point cloud into the first-stage point cloud deformation module to generate the first-stage reconstructed point cloud; S4: Input the fusion features generated in step S2 and the first-stage reconstructed point cloud into the second-stage point cloud deformation module to generate the final single-view point cloud reconstruction result.

[0006] Furthermore, the processing procedure of the adaptive multi-scale feature fusion module in step S2 is as follows: adaptive fusion of high-level semantic features and low-level detail features is achieved through adaptive weight generation and gating mechanisms, as detailed below: S2.1, low-level detail features are aligned with high-level semantic features in terms of size and channel number by using a 3×3 convolution with a stride of 2; S2.2, calculate the channel weights of high-level semantic features using the channel attention mechanism, and then use these high-level semantic channel weights as a guide to perform weighted processing on low-level detailed features; S2.3, the weighted low-level detail features and high-level semantic features are processed by multiple convolutions respectively; S2.4, the convolutional feature maps are concatenated along the channel dimension, and then the number of channels of the concatenated feature map is compressed to 1 through a convolution operation. Finally, the adaptive weights are calculated using the Sigmoid function. S2.5, Based on the adaptive weights obtained in step 2.4, the high-level semantic features and the weighted low-level detail features are fused through a gating mechanism; S2.6, The fused features from step S2.5 are post-processed and optimized using the EMA attention mechanism to enhance feature representation capabilities and output the optimized fused features.

[0007] Furthermore, the processing procedure of the first-stage point cloud deformation module in step S3 is as follows: S3.1, extract the feature feat1 of the spherical point cloud based on the graph attention mechanism, and set the extracted point cloud feature channels to 32; S3.2, Input the fused features and feature feat1 described in step S2 into the first style generation module to adaptively generate style code style1; S3.3, Instance normalization is performed on the extracted feature feat1 to eliminate statistical differences between batches; S3.4, adjust the number of channels of style code style1 through 1D convolution and divide it into two parts: scaling parameter γ1 and offset parameter β1; S3.5, adjust the feature feat1 according to the scaling parameter γ1 and the offset parameter β1. The process is: feat1' = γ1×feat1+β1, to obtain the adjusted point cloud feature feat1'. S3.6, Input the adjusted point cloud feature feat1', extract the point cloud feature feat2 based on the graph attention mechanism, and set the extracted point cloud feature channels to 64; S3.7, Input the fused features and feature feat2 described in step S2 into the second style generation module to adaptively generate style code style2; S3.8, Instance normalization is performed on the extracted feature feat2 to eliminate statistical differences between batches; S3.9, adjust the number of channels of style code style2 through 1D convolution and divide it into two parts: scaling parameter γ2 and offset parameter β2; S3.10, adjust the feature feat2 according to the scaling parameter γ2 and the offset parameter β2. The process is: feat2'=γ2×feat2+β2, to obtain the adjusted point cloud feature feat2'. S3.11, Input the adjusted feature feat2', extract the feature feat3 of the point cloud based on the graph attention mechanism, and set the extracted point cloud feature channels to 128; S3.12, Input the fused features and feature feat3 described in step S2 into the third style generation module to adaptively generate style code style3; S3.13, Instance normalization is performed on the extracted feature feat3 to eliminate statistical differences between batches; S3.14, adjust the number of channels of style code style3 through 1D convolution and divide it into two parts: scaling parameter γ3 and offset parameter β3; S3.15, adjust the feature feat3 according to the scaling parameter γ3 and the offset parameter β3. The process is: feat3'=γ3×feat3+β3, to obtain the adjusted point cloud feature feat3'. S3.16, input the final point cloud features into the MLP multilayer perceptron for processing to obtain the first offset, and add the uniformly sampled spherical point cloud to the first offset to obtain the first stage reconstructed point cloud.

[0008] Furthermore, the processing procedure of the second-stage point cloud deformation module in step S4 is as follows: S4.1, Based on the graph attention mechanism, extract the features of the reconstructed point cloud in the first stage and perform dimension transpose, transpose it into point cloud features pc_feat of dimension (24, 2048, 128); S4.2, Perform spatial dimension flattening, dimension transpose and fully connected layer linear transformation operations on the fused features obtained in step S2 to convert the original image features with dimensions (24, 512, 7, 7) into a feature sequence image_feat with dimensions (24, 49, 128); S4.3, a multi-head attention mechanism is used to interact the feature sequence image_feat with the point cloud feature pc_feat, specifically as follows: Query: pc_feat [B, 2048, 128] Key: image_feat [B, 49, 128] Value: image_feat [B, 49, 128] The enhanced point cloud feature impc_feat is obtained by calculating attention weights and weighted feature fusion. S4.4, perform residual connection and layer normalization on the enhanced point cloud feature impc_feat and the point cloud feature pc_feat described in step S4.1, and perform dimension transpose on the normalized feature to obtain the final point cloud feature pc_featend. S4.5, the final point cloud feature pc_featend is input into the MLP multilayer perceptron for processing to obtain the second offset, and the first-stage reconstructed point cloud is added to the second offset to generate the final single-view point cloud reconstruction result.

[0009] Further, in step S3.2, the first style generation module receives two inputs: feature feat1 and fused feature, and performs feature alignment and transformation. The processing procedure is as follows: S3.2.1, use a 2D convolution with dilation=2 and padding=2 and max pooling to process the fused features, reduce the number of feature channels from 512 to 128, and generate a new feature z1; S3.2.2, the feature z1 dimension is compressed to 18 through a fully connected layer; S3.2.3, using the learnable subspace basis matrix and bias, project the compressed features from step S3.2.2 onto the point cloud feature space to obtain the projected feature vector; S3.2.4, through the dimension reshaping operation, transforms the projected feature vector into the same spatial dimension as feature feat1, generating style code style1.

[0010] Further, in step S3.7, the second style generation module receives two inputs: feature feat2 and fused feature, and performs feature alignment and transformation. The processing procedure is as follows: S3.7.1, use a 2D convolution with dilation=1 and padding=1 and max pooling to process the fused features, reduce the number of feature channels from 512 to 256, and generate a new feature z2; S3.7.2, the feature z2 dimension is compressed to 18 through a fully connected layer; S3.7.3, using the learnable subspace basis matrix and bias, project the compressed features from step S3.7.2 onto the point cloud feature space to obtain the projected feature vector; S3.7.4 uses a dimension reshaping operation to convert the projected feature vector into the same spatial dimension as feature feat2, generating style code style2.

[0011] Further, in step S3.12, the third style generation module receives two inputs: feature feat3 and fused feature, and performs feature alignment and transformation. The processing procedure is as follows: S3.12.1 uses a 2D convolution with dilation=1 and padding=1 and max pooling to fuse features, keeping the number of feature channels unchanged, and generates a new feature z3; S3.12.2, compresses the feature z3 dimension to 18 through a fully connected layer; S3.12.3, using the learnable subspace basis matrix and bias, project the compressed features from step S3.12.2 onto the point cloud feature space to obtain the projected feature vector; S3.12.4 converts the projected feature vector into the same spatial dimension as the feature feat3 through a dimension reshaping operation, generating the style code style3.

[0012] Furthermore, a dynamic weighted two-stage loss function design is adopted, which adaptively adjusts the weight ratio of the first-stage point cloud reconstruction loss and the second-stage point cloud reconstruction loss through training rounds to achieve a progressive optimization strategy. The loss function is expressed as: Loss=(1-w)×L CD1 +w×L CD2 ; Among them, L CD1 For the point cloud reconstruction loss in the first stage, L CD2 For the second stage of point cloud reconstruction loss, w is a dynamic weight coefficient that changes with the training epoch. The designed weight is w = 0.2 for the first 80 epochs, w = 0.4 for 80-120 epochs, and w = 0.5 for subsequent training epochs. The Chamfer Distance loss function is used to calculate the loss between the first-stage and second-stage point cloud reconstruction. Specifically: Among them, for the first or second stage of reconstructing the point cloud P o Each point in the data is represented in the real point cloud P. t Find its nearest corresponding point in the middle and calculate the sum of the L1 distances between them. For the first or second stage reconstructed point cloud P t For each point in the predicted point cloud P o Find the nearest point in the cloud, divide both values ​​by the number of points in the cloud, sum them up and take the average, so that the loss value is not affected by the absolute size of the cloud.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: The adaptive multi-scale feature fusion module of this invention intelligently fuses image features of different scales through an attention mechanism and a dynamic weighting mechanism guided by high-level semantic features, effectively solving the problems of feature loss and style monotony in single-view reconstruction. Through multiple independent style generation modules, multiple style codes are adaptively generated from the multi-scale features of the single-view image to modulate the generation of point cloud features in a hierarchical manner, overcoming the problem of style monotony caused by coarse global style vector control in traditional methods. By designing a two-stage point cloud reconstruction network, the second stage uses multi-head cross-attention to enable deep interaction between point cloud features and image features, realizing targeted feature supplementation and structural refinement under semantic guidance, effectively improving the geometric accuracy and detail richness of single-view reconstruction. Attached Figure Description

[0014] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a schematic diagram illustrating the progressive fusion using a multi-scale feature fusion module in this invention. Detailed Implementation

[0015] The present invention will be further described in detail below with reference to specific embodiments. These descriptions are for explanation purposes only and are not intended to limit the scope of the invention.

[0016] like Figure 1 As shown, a two-stage single-view reconstruction method based on adaptive multi-scale feature fusion includes the following steps: S1: Input a two-dimensional image and extract multi-scale, multi-level two-dimensional image features through a ResNet encoder; S2: The two-dimensional image features are progressively fused from shallow to deep layers in a pairwise manner, and then input into the adaptive multi-scale feature fusion module to generate fused features; S3: Input the fusion features generated in step S2 and the uniformly sampled spherical point cloud into the first-stage point cloud deformation module to generate the first-stage reconstructed point cloud; S4: Input the fusion features generated in step S2 and the first-stage reconstructed point cloud into the second-stage point cloud deformation module to generate the final single-view point cloud reconstruction result.

[0017] The specific processing flow for each module is as follows: For a detailed schematic diagram of the multi-scale feature fusion module, please refer to [link / reference]. Figure 2 Features are fused in pairs from low to high levels, and the feature fusion module is a simplified version of the adaptive multi-scale feature fusion module.

[0018] In step S2, the adaptive multi-scale feature fusion module achieves adaptive fusion of high-level semantic features and low-level detailed features through adaptive weight generation and gating mechanisms, as detailed below: S2.1, low-level detail features are aligned with high-level semantic features in terms of size and channel number by using a 3×3 convolution with a stride of 2; S2.2, calculate the channel weights of high-level semantic features through the channel attention mechanism, and use the calculated weights as high-level guidance to perform weighted calculations on low-level detailed features; S2.3, the weighted low-level detail features and high-level semantic features are processed by multiple convolutions respectively; S2.4, the convolutional feature maps are concatenated along the channel dimension, and then the number of channels of the concatenated feature map is compressed to 1 through a convolution operation. Then, the adaptive weights are calculated using the Sigmoid function. S2.5, based on the adaptive weights obtained in step 2.4, the high-level semantic features and the weighted low-level detail features are fused through a gating mechanism. S2.6, The fused features from step S2.5 are post-processed and optimized using the EMA attention mechanism to enhance feature representation capabilities and output the optimized fused features (feat).

[0019] The flowchart for the first-stage point cloud deformation module processing is as follows: S3.1, The starting point cloud is a spherical point cloud with uniform sampling of the point cloud. The feature feat1 of the spherical point cloud is extracted based on the graph attention mechanism. The extracted point cloud feature channels are set to 32. S3.2, Input the fused features and feature feat1 described in step S2 into the first style generation module to adaptively generate style code style1; S3.3, Instance normalization is performed on the extracted feature feat1 to eliminate statistical differences between batches; S3.4, adjust the number of channels of style code style1 through 1D convolution and divide it into two parts: scaling parameter γ1 and offset parameter β1; S3.5, adjust the feature feat1 according to the scaling parameter γ1 and the offset parameter β1. The process is: feat1' = γ1×feat1+β1, to obtain the adjusted point cloud feature feat1'. S3.6, Input the adjusted point cloud feature feat1', extract the point cloud feature feat2 based on the graph attention mechanism, and set the extracted point cloud feature channels to 64; S3.7, Input the fused features and feature feat2 described in step S2 into the second style generation module to adaptively generate style code style2; S3.8, Instance normalization is performed on the extracted feature feat2 to eliminate statistical differences between batches; S3.9, adjust the number of channels of style code style2 through 1D convolution and divide it into two parts: scaling parameter γ2 and offset parameter β2; S3.10, adjust the feature feat2 according to the scaling parameter γ2 and the offset parameter β2. The process is: feat2'=γ2×feat2+β2, to obtain the adjusted point cloud feature feat2'. S3.11, Input the adjusted feature feat2', extract the feature feat3 of the point cloud based on the graph attention mechanism, and set the extracted point cloud feature channels to 128; S3.12, Input the fused features and feature feat3 described in step S2 into the third style generation module to adaptively generate style code style3; S3.13, Instance normalization is performed on the extracted feature feat3 to eliminate statistical differences between batches; S3.14, adjust the number of channels of style code style3 through 1D convolution and divide it into two parts: scaling parameter γ3 and offset parameter β3; S3.15, adjust the feature feat3 according to the scaling parameter γ3 and the offset parameter β3. The process is: feat3'=γ3×feat3+β3, to obtain the adjusted point cloud feature feat3'. S3.16, input the final point cloud features into the MLP multilayer perceptron for processing to obtain the first offset, and add the uniformly sampled spherical point cloud to the first offset to obtain the first stage reconstructed point cloud.

[0020] Further, in step S3.2, the first style generation module receives two inputs: feature feat1 and fused feature, and performs feature alignment and transformation. The processing procedure is as follows: S3.2.1, use a 2D convolution with dilation=2 and padding=2 and max pooling to process the fused features, reduce the number of feature channels from 512 to 128, and generate a new feature z1; S3.2.2, the feature z1 dimension is compressed to 18 through a fully connected layer; S3.2.3, using the learnable subspace basis matrix and bias, project the compressed features from step S3.2.2 onto the point cloud feature space to obtain the projected feature vector; S3.2.4, through the dimension reshaping operation, transforms the projected feature vector into the same spatial dimension as feature feat1, generating style code style1.

[0021] Further, in step S3.7, the second style generation module receives two inputs: feature feat2 and fused feature, and performs feature alignment and transformation. The processing procedure is as follows: S3.7.1, use a 2D convolution with dilation=1 and padding=1 and max pooling to process the fused features, reduce the number of feature channels from 512 to 256, and generate a new feature z2; S3.7.2, the feature z2 dimension is compressed to 18 through a fully connected layer; S3.7.3, using the learnable subspace basis matrix and bias, project the compressed features from step S3.7.2 onto the point cloud feature space to obtain the projected feature vector; S3.7.4 uses a dimension reshaping operation to convert the projected feature vector into the same spatial dimension as feature feat2, generating style code style2.

[0022] Further, in step S3.12, the third style generation module receives two inputs: feature feat3 and fused feature, and performs feature alignment and transformation. The processing procedure is as follows: S3.12.1 uses a 2D convolution with dilation=1 and padding=1 and max pooling to fuse features, keeping the number of feature channels unchanged, and generates a new feature z3; S3.12.2, compresses the feature z3 dimension to 18 through a fully connected layer; S3.12.3, using the learnable subspace basis matrix and bias, project the compressed features from step S3.12.2 onto the point cloud feature space to obtain the projected feature vector; S3.12.4 converts the projected feature vector into the same spatial dimension as the feature feat3 through a dimension reshaping operation, generating the style code style3.

[0023] The flowchart for the second-stage point cloud deformation module is as follows: S4.1, Based on the graph attention mechanism, extract the features of the reconstructed point cloud in the first stage and perform dimension transpose, transpose it into point cloud features pc_feat of dimension (24, 2048, 128); S4.2, Perform spatial dimension flattening, dimension transpose and fully connected layer linear transformation operations on the fused features obtained in step S2 to convert the original image features with dimensions (24, 512, 7, 7) into a feature sequence image_feat with dimensions (24, 49, 128); S4.3, a multi-head attention mechanism is used to interact the feature sequence image_feat with the point cloud feature pc_feat, specifically as follows: Query: pc_feat [B, 2048, 128] Key: image_feat [B, 49, 128] Value: image_feat [B, 49, 128] The enhanced point cloud feature impc_feat is obtained by calculating attention weights and weighted feature fusion. S4.4, perform residual connection and layer normalization on the enhanced point cloud feature impc_feat and the point cloud feature pc_feat described in step S4.1, and perform dimension transpose on the normalized feature to obtain the final point cloud feature pc_featend. S4.5, the final point cloud feature pc_featend is input into the MLP multilayer perceptron for processing to obtain the second offset, and the first-stage reconstructed point cloud is added to the second offset to generate the final single-view point cloud reconstruction result.

[0024] Loss function calculation: This invention employs a dynamically weighted two-stage loss function design, which adaptively adjusts the weight ratio of the first and second stage losses through training rounds to achieve a progressive optimization strategy.

[0025] The loss function is expressed as: Loss=(1-w)×L CD1 +w×L CD2 ; Among them, L CD1 For the point cloud reconstruction loss in the first stage, L CD2 The second stage point cloud refinement loss is w, which is a dynamic weight coefficient that changes with the training epoch. The designed weights are w = 0.2 for the first 80 epochs, w = 0.4 for 80-120 epochs, and w = 0.5 for subsequent training epochs.

[0026] The specific loss calculation for the two-stage process uses the Chamfer Distance loss function: Specifically: Among them, for the predicted point cloud P o Each point in the data is represented in the real point cloud P. t Find its nearest corresponding point in the data and calculate the sum of their L1 distances. For the real point cloud P... t For each point in the predicted point cloud P o Find the nearest point in the point cloud. Divide each of the two terms by the number of points in the point cloud, sum them up, and take the average to ensure that the loss value is not affected by the absolute size of the point cloud.

[0027] This invention uses a two-stage reconstruction method to further enrich the reconstruction details based on the already accurate shape prediction.

[0028] The adaptive multi-scale feature fusion module effectively solves the technical problem that single-scale semantic features lead to a monotonous semantic style and lack of diversity in the reconstruction results, while also improving the accuracy of reconstruction. By generating diverse style codes, it overcomes the coarseness problem of traditional global style control, increases the richness of reconstruction semantics, improves the integrity of reconstruction at each scale, and reduces the discontinuity of reconstruction connections.

[0029] Table 1 The reconstruction performance of this invention on the ShapeNet dataset is measured by Chamfer Distance (hereinafter referred to as CD). For ease of comparison, the results in the table are the original values ​​multiplied by 100. A smaller CD value indicates a better reconstruction effect. As shown in Table 1, the final results for testing on all datasets are as follows: the final CD value of our method is 2.84, which is optimal compared to other single-view reconstruction methods. Among the reconstruction results for 13 categories, the best results are achieved in ten categories: plane, cabin, car, chair, display, lamp, loud, sofa, and table.

Claims

1. A two-stage single-view reconstruction method based on adaptive multi-scale feature fusion, characterized in that, Includes the following steps: S1: Input a two-dimensional image and extract multi-scale, multi-level two-dimensional image features through a ResNet encoder; S2: The two-dimensional image features are progressively fused from shallow to deep layers in a pairwise manner, and then input into the adaptive multi-scale feature fusion module to generate fused features; S3: Input the fusion features generated in step S2 and the uniformly sampled spherical point cloud into the first-stage point cloud deformation module to generate the first-stage reconstructed point cloud; S4: Input the fusion features generated in step S2 and the first-stage reconstructed point cloud into the second-stage point cloud deformation module to generate the final single-view point cloud reconstruction result.

2. The two-stage single-view reconstruction method based on adaptive multi-scale feature fusion according to claim 1, characterized in that, The adaptive multi-scale feature fusion module in step S2 processes high-level semantic features and low-level detail features through adaptive weight generation and gating mechanisms, as follows: S2.1, low-level detail features are aligned with high-level semantic features in terms of size and channel number by using a 3×3 convolution with a stride of 2; S2.2, calculate the channel weights of high-level semantic features using the channel attention mechanism, and then use these high-level semantic channel weights as a guide to perform weighted processing on low-level detailed features; S2.3, the weighted low-level detail features and high-level semantic features are processed by multiple convolutions respectively; S2.4, the convolutional feature maps are concatenated along the channel dimension, and then the number of channels of the concatenated feature map is compressed to 1 through a convolution operation. Finally, the adaptive weights are calculated using the Sigmoid function. S2.5, Based on the adaptive weights obtained in step 2.4, the high-level semantic features and the weighted low-level detail features are fused through a gating mechanism; S2.6, The fused features from step S2.5 are post-processed and optimized using the EMA attention mechanism to enhance feature representation capabilities and output the optimized fused features.

3. The two-stage single-view reconstruction method based on adaptive multi-scale feature fusion according to claim 1, characterized in that, The processing procedure of the first-stage point cloud deformation module in step S3 is as follows: S3.1, extract the feature feat1 of the spherical point cloud based on the graph attention mechanism, and set the extracted point cloud feature channels to 32; S3.2, Input the fused features and feature feat1 described in step S2 into the first style generation module to adaptively generate style code style1; S3.3, Instance normalization is performed on the extracted feature feat1 to eliminate statistical differences between batches; S3.4, adjust the number of channels of style code style1 through 1D convolution and divide it into two parts: scaling parameter γ1 and offset parameter β1; S3.5, adjust the feature feat1 according to the scaling parameter γ1 and the offset parameter β1. The process is: feat1'=γ1×feat1+β1, to obtain the adjusted point cloud feature feat1'. S3.6, Input the adjusted point cloud feature feat1', extract the point cloud feature feat2 based on the graph attention mechanism, and set the extracted point cloud feature channels to 64; S3.7, Input the fused features and feature feat2 described in step S2 into the second style generation module to adaptively generate style code style2; S3.8, Instance normalization is performed on the extracted feature feat2 to eliminate statistical differences between batches; S3.9, adjust the number of channels of style code style2 through 1D convolution and divide it into two parts: scaling parameter γ2 and offset parameter β2; S3.10, adjust the feature feat2 according to the scaling parameter γ2 and the offset parameter β2. The process is: feat2'=γ2×feat2+β2, to obtain the adjusted point cloud feature feat2'. S3.11, Input the adjusted feature feat2', extract the feature feat3 of the point cloud based on the graph attention mechanism, and set the extracted point cloud feature channels to 128; S3.12, Input the fused features and feature feat3 described in step S2 into the third style generation module to adaptively generate style code style3; S3.13, Instance normalization is performed on the extracted feature feat3 to eliminate statistical differences between batches; S3.14, adjust the number of channels of style code style3 through 1D convolution and divide it into two parts: scaling parameter γ3 and offset parameter β3; S3.15, adjust the feature feat3 according to the scaling parameter γ3 and the offset parameter β3. The process is: feat3'=γ3×feat3+β3, to obtain the adjusted point cloud feature feat3'. S3.16, input the final point cloud features into the MLP multilayer perceptron for processing to obtain the first offset, and add the uniformly sampled spherical point cloud to the first offset to obtain the first stage reconstructed point cloud.

4. The two-stage single-view reconstruction method based on adaptive multi-scale feature fusion according to claim 1, characterized in that, The processing procedure of the second-stage point cloud deformation module in step S4 is as follows: S4.1, Based on the graph attention mechanism, extract the features of the reconstructed point cloud in the first stage and perform dimension transpose, transpose it into point cloud features pc_feat of dimension (24, 2048, 128); S4.2, Perform spatial dimension flattening, dimension transpose and fully connected layer linear transformation operations on the fused features obtained in step S2 to convert the original image features with dimensions (24, 512, 7, 7) into a feature sequence image_feat with dimensions (24, 49, 128); S4.3, a multi-head attention mechanism is used to interact the feature sequence image_feat with the point cloud feature pc_feat, specifically as follows: Query: pc_feat [B, 2048, 128] Key: image_feat [B, 49, 128] Value: image_feat [B, 49, 128] The enhanced point cloud feature impc_feat is obtained by calculating attention weights and fusing features with weights. S4.4, perform residual connection and layer normalization on the enhanced point cloud feature impc_feat and the point cloud feature pc_feat described in step S4.1, and perform dimension transpose on the normalized feature to obtain the final point cloud feature pc_featend. S4.5, the final point cloud feature pc_featend is input into the MLP multilayer perceptron for processing to obtain the second offset, and the first-stage reconstructed point cloud is added to the second offset to generate the final single-view point cloud reconstruction result.

5. The two-stage single-view reconstruction method based on adaptive multi-scale feature fusion according to claim 3, characterized in that, In step S3.2, the first style generation module receives two inputs: feature feat1 and fused feature, and performs feature alignment and transformation. The processing procedure is as follows: S3.2.1, use a 2D convolution with dilation=2 and padding=2 and max pooling to process the fused features, reduce the number of feature channels from 512 to 128, and generate a new feature z1; S3.2.2, the feature z1 dimension is compressed to 18 through a fully connected layer; S3.2.3, using the learnable subspace basis matrix and bias, project the compressed features from step S3.2.2 onto the point cloud feature space to obtain the projected feature vector; S3.2.4, through the dimension reshaping operation, transforms the projected feature vector into the same spatial dimension as feature feat1, generating style code style1.

6. The two-stage single-view reconstruction method based on adaptive multi-scale feature fusion according to claim 3, characterized in that, In step S3.7, the second style generation module receives two inputs: feature feat2 and fused feature, and performs feature alignment and transformation. The processing procedure is as follows: S3.7.1, use a 2D convolution with dilation=1 and padding=1 and max pooling to process the fused features, reduce the number of feature channels from 512 to 256, and generate a new feature z2; S3.7.2, the feature z2 dimension is compressed to 18 through a fully connected layer; S3.7.3, using the learnable subspace basis matrix and bias, project the compressed features from step S3.7.2 onto the point cloud feature space to obtain the projected feature vector; S3.7.4 uses a dimension reshaping operation to convert the projected feature vector into the same spatial dimension as feature feat2, generating style code style2.

7. The two-stage single-view reconstruction method based on adaptive multi-scale feature fusion according to claim 3, characterized in that, In step S3.12, the third style generation module receives two inputs: feature feat3 and fused feature. It then performs feature alignment and transformation. The processing procedure is as follows: S3.12.1 uses a 2D convolution with dilation=1 and padding=1 and max pooling to fuse features, keeping the number of feature channels unchanged, and generates a new feature z3; S3.12.2, compresses the feature z3 dimension to 18 through a fully connected layer; S3.12.3, using the learnable subspace basis matrix and bias, project the compressed features from step S3.12.2 onto the point cloud feature space to obtain the projected feature vector; S3.12.4 converts the projected feature vector into the same spatial dimension as the feature feat3 through a dimension reshaping operation, generating the style code style3.

8. The two-stage single-view reconstruction method based on adaptive multi-scale feature fusion according to claim 1, characterized in that, A dynamic weighted two-stage loss function is adopted, and the weight ratio of the point cloud reconstruction loss in the first stage and the point cloud reconstruction loss in the second stage is adaptively adjusted by the training rounds to achieve a progressive optimization strategy. The loss function is expressed as: Loss=(1-w)×L CD1 +w×L CD2 ; Among them, L CD1 For the point cloud reconstruction loss in the first stage, L CD2 For the second stage of point cloud reconstruction loss, w is a dynamic weight coefficient that changes with the training epoch. The designed weight is w = 0.2 for the first 80 epochs, w = 0.4 for 80-120 epochs, and w = 0.5 for subsequent training epochs. The Chamfer Distance loss function is used to calculate the loss between the first-stage and second-stage point cloud reconstruction. Specifically: Among them, for the first or second stage of reconstructing the point cloud P o Each point in the data is represented in the real point cloud P. t Find its nearest corresponding point in the middle and calculate the sum of the L1 distances between them. For the first or second stage reconstructed point cloud P t For each point in the predicted point cloud P o Find the nearest point in the cloud, divide both values ​​by the number of points in the cloud, sum them up and take the average, so that the loss value is not affected by the absolute size of the cloud.

Images