Single-view point cloud reconstruction method and system based on occlusion recovery and skeleton generation
By using occlusion recovery and skeleton generation methods, the problems of occlusion and complex structures in single-view 3D reconstruction are solved, generating high-quality 3D point clouds and achieving effective recovery of occluded areas and complete structural reconstruction of objects.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-02
- Publication Date
- 2026-03-13
AI Technical Summary
Existing single-view 3D reconstruction methods produce poor reconstruction results when dealing with image occlusion and complex structures, resulting in missing details and an inability to accurately generate complete and smooth 3D point clouds.
A coarse point cloud is generated by a point cloud generation network, the occlusion recovery network ORnet is used to recover the information of the occluded parts, and the skeleton point cloud is generated by combining the skeleton generation network SRNet. Finally, the point cloud is upsampled and feature fused by a point cloud refiner to generate a final point cloud with complete structure and smoothness.
It significantly improves reconstruction accuracy and visual realism, generates high-quality 3D point clouds, and can effectively restore occluded areas while maintaining the complete structure and smooth surface of objects.
Smart Images
Figure CN121661246A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to monocular vision. Figure 3 In the field of dimensional reconstruction technology, this invention relates in particular to a single-view point cloud reconstruction method based on occlusion recovery and skeleton generation. Background Technology
[0002] Current single view Figure 3 The mainstream strategy for 3D reconstruction methods is to utilize shape generation and shape thinning strategies to reconstruct the 3D shape of objects by extracting features from the image. Therefore, shape generation and shape thinning are crucial in multi-view... Figure 3 It plays an important role in the reconstruction.
[0003] Early monocular vision Figure 3 3D reconstruction methods use convolutional neural networks (CNNs) to encode features of the input image and decode these features into 3D shapes. For example, LEPC first uses a CNN to extract high-dimensional features from the image, decodes these features into 3D space using a structure generator, and then introduces a pseudo-renderer to refine and visualize the 3D representation of the 2D image. Wang et al. proposed a single-view reconstruction method called Pix2Mesh. This architecture first uses a CNN network to capture deep features of the image and generates an initial mesh. Shape generation is achieved by guiding the deformation of the initial mesh through feature and mesh concatenation. Finally, the image features are decoded into a mesh representation through the mesh deformation of high-dimensional features. PCDNet extracts single-view features through a convolutional neural network and randomly generates an initial point cloud. It uses projection and an AdaIN optimizer to guide the deformation of the initial point cloud to the corresponding 3D point cloud shape of the image. However, since convolutional neural networks can only extract existing information from the image, they cannot model occluded parts and cannot accurately generate shapes.
[0004] In addition, some works achieve shape refinement through interpolation algorithms. For example, CRA-PCN uses a seed generator to generate a seed point cloud for the object, and then uses an interpolation algorithm to refine the point cloud. PCDNetv2 uses adaptive pooling to aggregate global and local features of the point cloud, and then uses an interpolation algorithm to transform the initial point cloud into the final refined point cloud. The Pix2Mesh++ method first generates a coarse mesh using Pix2Mesh, and then uses a bilinear interpolation algorithm to refine the mesh representation. Although the above methods can achieve shape refinement, they cannot simultaneously refine the overall structure and local details of the object, resulting in relatively poor reconstruction performance.
[0005] Therefore, this paper proposes a single-view point cloud reconstruction method based on occlusion recovery and skeleton generation (SPOS), which can gradually recover a fine point cloud representation of a given object using an RGB image or grayscale image as input. Summary of the Invention
[0006] Therefore, the purpose of this invention is to provide a single-view point cloud reconstruction method based on occlusion recovery and skeleton generation, to solve the problems of poor reconstruction effect and lack of detail in existing technologies when dealing with image occlusion and complex structures. This method recovers the occluded region through symmetry learning and utilizes a skeleton-guided point cloud refinement strategy to generate high-quality 3D point clouds with complete structure and smooth surface from a single RGB or grayscale image, significantly improving reconstruction accuracy and visual realism.
[0007] To achieve the above objectives, this invention provides a single-view point cloud reconstruction method based on occlusion recovery and skeleton generation, comprising the following steps:
[0008] S1. Generate a coarse point cloud from the input single-view image using a point cloud generation network;
[0009] S2. The ORnet occlusion recovery network is used to divide the coarse point cloud into regions and extract features. The symmetric relationship between regions is learned through self-attention and cross-attention mechanisms to recover the information of the occluded parts and generate a point cloud with complete information.
[0010] S3. The generated complete point cloud is downsampled and feature-enhanced using the skeleton generation network SRNet to generate a skeleton point cloud;
[0011] S4. Use a point cloud refiner to upsample and fuse the skeleton point cloud to generate a final point cloud with complete structure and high smoothness.
[0012] More preferably, the step of generating a coarse point cloud from the input single-view image using a point cloud generation network includes the following steps:
[0013] The point cloud generation network uses PCDNet, which includes a convolutional neural network and a random initial point cloud. The convolutional neural network performs convolution operations on the input single image to increase the feature dimension of the image and obtain the shape features of the image. The random initial point cloud uses the shape features extracted from multiple fused images to deform the point cloud, optimize the point cloud and integrate the shape feature information to generate a coarse point cloud.
[0014] More preferably, in S2, the occlusion recovery network ORnet uses a combination of the farthest point sampling method and the K-nearest neighbor method to divide the coarse point cloud generated in S1 into regions according to a fixed number of points in each region; the fixed number of points includes 1 center point and 15 neighboring points; and extracts the features of each region point.
[0015] More preferably, the extraction of features for each region point includes the following steps:
[0016] S201. Calculate the coordinate characteristics of the center point and neighboring points within the region;
[0017] S202. Use a multilayer perceptron to extract high-dimensional features of the center point and neighboring points respectively, and use the coordinate features of all points to describe the spatial information of all points.
[0018] Further preferably, in S3, the symmetric relationship between regions is learned through self-attention and cross-attention mechanisms to recover the information of the occluded parts and generate a point cloud with complete information, including:
[0019] A fixed number of points in each region are input into the occlusion recovery network ORnet. The occlusion recovery network ORnet uses a self-attention mechanism to learn the feature relationship between neighboring points and the center point, and uses the self-attention mechanism to regularize the region features.
[0020] Input all the region features after self-attention normalization and use the cross-attention mechanism to construct the symmetric relationship between regions to obtain the features of the symmetric regions of the original region features;
[0021] By stitching the original data with the features of the symmetrical region, the information of the occluded part is restored, and a point cloud with complete information is generated.
[0022] More preferably, the step of constructing symmetrical relationships between regions using a cross-attention mechanism includes calculating the attention score between the input region and the symmetrical region using the following formula:
[0023]
[0024] in, For the input area data, For the input region, and These are the linear transformation matrices used to map input features to the query and key spaces, respectively; α ij The attention score represents the correlation weights used to quantify the symmetric relationships between the five regions of a coarse point cloud.
[0025] A further preferred embodiment includes: using the obtained attention weight α ijTo calculate the weighted sum of the value vector V, thus establishing a symmetric relationship between regions:
[0026]
[0027] Y = [y1, y2, ..., y5] (6)
[0028] Where Y = [y1, y2, ..., y5] represents the result of constructing the symmetric relationship of any 5 regions; This represents the result of establishing a symmetrical relationship between the i-th region feature and the ith region feature.
[0029] More preferably, in S3, the skeleton generation network SRNet includes a skeleton generator:
[0030] The skeleton generator uses downsampling to generate a first low-resolution point cloud m0 and extracts the first feature f0 of the point cloud m0; it uses a multilayer perceptron (MLP) to generate a second low-resolution point cloud m1 and extracts the second feature f1 of the point cloud m1.
[0031] The second feature f1 is enhanced by local attention in m1. An interpolation method is used to interpolate the enhanced m1 feature into the first low-resolution point cloud m0 and the first feature f0. Then, local attention is used again to enhance the point cloud m1 corresponding to the second feature f1. A multilayer perceptron (MLP) is used to input the fused point cloud features into the skeleton, generating the skeleton point cloud M. sk
[0032] Further preferably, the skeleton generation network SRNet includes a point cloud refiner, which consists of a multilayer perceptron (MLP), skeleton features, and an upsampling layer; S4 includes:
[0033] S401. Map the skeleton point cloud generated in S3 to a high-dimensional space using a multilayer perceptron (MLP).
[0034] S402. Perform upsampling and fuse the upsampled point cloud features with the skeleton features;
[0035] S403. Repeat this process multiple times until a point cloud with a complete structure and high smoothness is obtained.
[0036] The present invention also provides a single-view point cloud reconstruction system based on occlusion recovery and skeleton generation, which is used to implement the above-mentioned single-view point cloud reconstruction method based on occlusion recovery and skeleton generation, including...
[0037] The coarse point cloud generation module generates coarse point clouds from the input single-view image through a point cloud generation network.
[0038] The occlusion recovery module uses the occlusion recovery network ORnet to perform region division and feature extraction on the coarse point cloud. It learns the symmetric relationship between regions through self-attention and cross-attention mechanisms to recover the information of the occluded parts and generate a point cloud with complete information.
[0039] The skeleton generation module uses the skeleton generation network SRNet to downsample and enhance the features of the generated complete point cloud to generate a skeleton point cloud; and uses a point cloud refiner to upsample and fuse the features of the skeleton point cloud to generate a final point cloud with complete structure and high smoothness.
[0040] The single-view point cloud reconstruction method and system based on occlusion recovery and skeleton generation disclosed in this application have at least the following advantages compared with the prior art:
[0041] This application uses a single RGB or grayscale image as input. It promotes the reconstruction of high-quality point clouds by learning the symmetry relationships of objects and skeleton point clouds. It utilizes a neighborhood point similarity feature extraction network in the point cloud encoder to obtain local consistency information of the object, and a cross-attention mechanism in the point cloud decoder to model the feature information of occluded parts. A skeleton generator obtains a minimum point cloud with complete object structural information, and a point cloud refiner upsamples the feature information of the skeleton point cloud. Furthermore, the point cloud refiner reconstructs a refined point cloud of the input object using skeleton features, generating point clouds with occlusion information and smooth surfaces, significantly improving the visual quality of the reconstructed point cloud. Qualitative and quantitative comparisons with state-of-the-art methods will be presented below. Test results on the ShapeNet and Pix3D datasets show that the proposed method outperforms existing methods in both visualization and numerical comparisons. Attached Figure Description
[0042] Figure 1 This is a flowchart illustrating the single-view point cloud reconstruction method based on occlusion recovery and skeleton generation according to the present invention.
[0043] Figure 2 This is a diagram showing the overall structure of the model in the single-view point cloud reconstruction method based on occlusion recovery and skeleton generation of the present invention.
[0044] Figure 3 This is a structural diagram illustrating the self-attention mechanism of the present invention for learning the characteristic relationship between neighboring points and the center point.
[0045] Figure 4 This is a schematic diagram illustrating the symmetrical relationship between regions constructed by the cross-attention mechanism of this invention.
[0046] Figure 5 This is a structural diagram of the skeleton generation and point cloud refinement of the present invention.
[0047] Figure 6This is a comparison chart of the results of the SPOS method proposed in this invention with existing methods such as PSG, PCDNet, and R3GF.
[0048] Figure 7 This is a comparison chart of the generalization ability of the SPOS method proposed in this invention with other methods on the Pix3D dataset. Detailed Implementation
[0049] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0050] like Figure 1-2 As shown, one embodiment of the present invention provides a single-view point cloud reconstruction method based on occlusion recovery and skeleton generation, comprising the following steps:
[0051] S1. Generate a coarse point cloud from the input single-view image using a point cloud generation network; preferably, generating a coarse point cloud from the input single-view image using a point cloud generation network includes the following steps:
[0052] The point cloud generation network uses PCDNet, which includes a convolutional neural network and a random initial point cloud. The convolutional neural network performs convolution operations on the input single image to increase the feature dimension of the image and obtain the shape features of the image. The random initial point cloud uses the shape features extracted from multiple fused images to deform the point cloud, optimize the point cloud and integrate the shape feature information to generate a coarse point cloud.
[0053] PCDNet uses a convolutional network to perform convolution operations on a single input image, increasing the image's feature dimension to obtain its shape features. Randomly initialized point clouds are then deformed by fusing the extracted shape features from the image multiple times, optimizing them in each iteration to effectively integrate shape feature information. Therefore, PCDNet is highly efficient in generating coarse point clouds from images.
[0054] S2. The occlusion recovery network ORnet is used to perform region division and feature extraction on the coarse point cloud. By learning the symmetric relationship between regions through self-attention and cross-attention mechanisms, the information of the occluded parts is recovered, and a point cloud with complete information is generated. In S2, the occlusion recovery network ORnet uses a combination of the farthest point sampling method and the K-nearest neighbor method to divide the coarse point cloud generated in S1 into regions according to a fixed number of points in each region. The fixed number of points includes 1 center point and 15 neighboring points. The features of each region point are then extracted. Figure 3 The process of extracting center point features is demonstrated.
[0055] Specifically, extracting features from each region point includes the following steps:
[0056] S201. Calculate the coordinate characteristics of the center point and neighboring points within the region; assume the coordinate characteristic of the center point is q. i The characteristics of the neighborhood points are
[0057] S202. High-dimensional features of the center point and neighboring points are extracted using a multilayer perceptron, and the spatial information of all points is described using the coordinate features of all points. The coordinate features are further improved to clearly define the coordinates of the center point of the guiding region.
[0058] High-dimensional features of the center point and its neighbors are extracted using a multilayer perceptron (MLP):
[0059] Q i =MLP(q) i (7)
[0060]
[0061] Among them, Q i K represents the high-dimensional feature of the center point of the i-th region. i This represents the high-dimensional features of all neighboring points of the center point of the i-th region. An MLP consists of 1-dimensional convolutional layers.
[0062] like Figure 4 As shown, further, by learning the symmetric relationship between regions through self-attention and cross-attention mechanisms, the information of the occluded parts is recovered, and a point cloud with complete information is generated, including:
[0063] A fixed number of points in each region are input into the occlusion recovery network ORnet. The occlusion recovery network ORnet uses a self-attention mechanism to learn the feature relationship between neighboring points and the center point, and uses the self-attention mechanism to regularize the region features.
[0064] Input all the region features after self-attention normalization and use the cross-attention mechanism to construct the symmetric relationship between regions to obtain the features of the symmetric regions of the original region features;
[0065] By stitching the original data with the features of the symmetrical region, the information of the occluded part is restored, and a point cloud with complete information is generated.
[0066] ORNet uses a self-attention mechanism to learn the feature relationships between neighboring points and the center point. Based on these relationships, the center's features are further optimized to determine its spatial location. Technically, the query Q = W... Q Q i Key K = W K K i Value v = W v K i The formula for calculating the characteristic relationship is:
[0067]
[0068] in, Indicates attention score, This represents the scaling factor.
[0069] Subsequently, ω i,j Used to calculate the weighting of neighborhood points relative to the center point:
[0070] O i =MLP(ω i，j ,V) (11)
[0071] Q in the formula i This represents the feature of the center point enhanced by neighboring points.
[0072] Based on the cross-attention mechanism, when constructing symmetrical relationships between regions (see...) Figure 4 ), Considered as query Q, Q is considered as a key K and a value V. ORNet calculates the attention score between Q and K.
[0073]
[0074] Among them, matrix and It is a linear transformation matrix used to map input features to the query and key spaces. Note the weights α. ij The correlation of the symmetry relationship among the five regions of the coarse point cloud was quantified.
[0075] Using α ij We calculate the weighted sum of the value vector V to construct symmetric relationships between regions.
[0076]
[0077] Y = [y1, y2, ..., y5] (14)
[0078] Where Y = [y1, y2, ..., y5] represents the result of constructing the symmetric relationship of the 5 regions. This represents the result of establishing a symmetrical relationship between the j-th region feature and the i-th region feature.
[0079] Compare Y = [y1, y2, ..., y5] with... By stitching together data, information missing from point cloud data can be supplemented.
[0080]
[0081] S3, such as Figure 5As shown, the skeleton point cloud is downsampled and its features enhanced using the skeleton generation network SRNet to generate a skeleton point cloud; further, in S3, the skeleton generation network SRNet includes a skeleton generator:
[0082] The skeleton generator uses downsampling to generate a first low-resolution point cloud m0 and extracts the first feature f0 of the point cloud m0; it uses a multilayer perceptron (MLP) to generate a second low-resolution point cloud m1 and extracts the second feature f1 of the point cloud m1.
[0083] The second feature f1 is enhanced by local attention, and the feature enhanced by local attention m1 is interpolated into the first low-resolution point cloud m0 and the first feature f0 using an interpolation method. Then, the point cloud m1 corresponding to the second feature f1 is enhanced again by local attention.
[0084] The fused point cloud features are input into the skeleton using a multilayer perceptron (MLP) to generate the skeleton point cloud M. sk First, a skeleton generator is constructed to improve structural integrity. Specifically, the skeleton generator uses downsampling and an MLP to generate low-resolution point clouds m0 and m1 and corresponding features f0 and f1. Local attention is applied to f1 to enhance the features of m1, and interpolation is used to fuse the enhanced point cloud to a higher resolution. This process is repeated twice. We use an MLP to input the fused point cloud features into Skeleton to generate the skeleton point cloud M. sk .
[0085] M sk =SK(MLP(m0)) (15)
[0086] S4. Use a point cloud refiner to upsample and fuse the skeleton point cloud to generate a final point cloud with complete structure and high smoothness.
[0087] Furthermore, the skeleton generation network SRNet includes a point cloud refiner, which consists of a multilayer perceptron (MLP), skeleton features, and an upsampling layer; S4 includes:
[0088] S401. Map the skeleton point cloud generated in S3 to a high-dimensional space using a multilayer perceptron (MLP).
[0089] S402. Perform upsampling and fuse the upsampled point cloud features with the skeleton features;
[0090] S403. Repeat this process multiple times until a point cloud with a complete structure and high smoothness is obtained.
[0091] To achieve a highly smooth point cloud, a point cloud refiner was designed. It consists of an MLP (Multi-Level Processing), skeleton features, and an upsampling layer. Specifically, the skeleton point cloud is upsampled to a higher dimension using the MLP, and the skeleton features are then fused and fed into the upsampling layer. This process is repeated three times to obtain the final point cloud M2, which has a complete structure and high smoothness.
[0092]
[0093] The present invention also provides a single-view point cloud reconstruction system based on occlusion recovery and skeleton generation, for implementing the above-mentioned single-view point cloud reconstruction method based on occlusion recovery and skeleton generation, including:
[0094] The coarse point cloud generation module generates coarse point clouds from the input single-view image through a point cloud generation network.
[0095] The occlusion recovery module uses the occlusion recovery network ORnet to perform region division and feature extraction on the coarse point cloud. It learns the symmetric relationship between regions through self-attention and cross-attention mechanisms to recover the information of the occluded parts and generate a point cloud with complete information.
[0096] The skeleton generation module uses the skeleton generation network SRNet to downsample and enhance the features of the generated complete point cloud to generate a skeleton point cloud; and uses a point cloud refiner to upsample and fuse the features of the skeleton point cloud to generate a final point cloud with complete structure and high smoothness.
[0097] Comparison on the ShapeNet dataset:
[0098] Quantitative comparison. As shown in Tables 1 and 2 below, for a fair comparison, a single image was input to the method, and its performance was evaluated by calculating IOU, CD, and F-Score using the pre-trained model provided in this application. Regarding the IoU metric, the average SPOS of this application was improved by 21.87% and 8.01% compared to 3D-R2N2 and GAL, respectively. As shown in Table 2 below, regarding the CD metric, compared to PSG, Pixel2Mesh, PCDNet, PCDNetv2, PAPRec, RDB, R3GF, OccNet, GAMesh, and TopoNet, the average CD of SPOS was reduced by 58.68%, 58.54%, 14.34%, 12.50%, 52.70%, 63.76%, 38.90%, 70.30%, 36.69%, and 51.00%, respectively. As shown in Table 3 below, regarding F... (τ)In terms of F-Score, compared with 3D-R2N2, PSG, NMR, Pixel2Mesh, PCDNetv2, R3GF, OccNet, GAMesh, and TopoNet, the average SPOS was improved by 88.49%, 51.36%, 117.54%, 23.10%, 3.42%, 6.09%, 16.88%, 3.83%, and 11.39%, respectively. In F(2... τ In terms of metrics, compared with 3D-R2N2, PSG, NMR, PCDNetv2, R3GF, GAMesh, and TopoNet, our method achieves average improvements of 59.15%, 24.58%, 82.17%, 1.79%, 6.38%, 5.14%, and 11.17%, respectively. While existing methods propose some advanced shape generation and thinning strategies, they do not fully utilize symmetry relationships to handle occlusion issues. Furthermore, these methods neglect the potential of skeleton generation for thinning, resulting in poor surface detail representation in occluded conditions, ultimately affecting the quality of the reconstruction results.
[0099] Table 1 shows the quantitative results comparing the IOU metric with existing methods on the ShapeNet dataset.
[0100]
[0101] Table 2 shows the quantitative results comparing the CD metric with existing methods on the ShapeNet dataset.
[0102]
[0103]
[0104] Table 3 shows the quantitative results comparing the F-score of existing methods on the ShapeNet dataset.
[0105]
[0106]
[0107] Qualitative comparison. Qualitative results are as follows: Figure 6 As shown, we first visualize and compare the results of SPOS, PSG, PCDNet, and R3GF. PSG is widely considered a classic method in the field of single-view 3D reconstruction. PCDNet can directly reconstruct 3D point clouds from a single image, with each generated point cloud containing 2000 points. On the other hand, R3GF is a multi-view 3D reconstruction method. Then, we carefully selected some images that are uncommon and difficult to reconstruct in the dataset. Figure 6As can be seen, in all cases, we visually outperform the comparison methods. PSG generates highly sparse point clouds, thus failing to reconstruct the complete structure of objects. PCDNet's reconstructions typically lack fine detail. Furthermore, the multi-view reconstruction method R3GF also suffers from poor structural integrity and insufficient detail. In contrast, SPOS generates point clouds with superior integrity and detail, highlighted in the red box area in ShapeNet. This enhanced performance is attributed to the dedicated occlusion recovery and shape refinement networks in the SPOS method.
[0108] Comparison on the Pix3D dataset:
[0109] To evaluate the generalization ability of the proposed method SPOS, we tested SPOS on the Pix3D dataset using a model pre-trained on the ShapeNet dataset. Since the image formats in Pix3D and ShapeNet are different, we followed the standard Pix3D preprocessing procedure to crop Pix3D images to maintain consistency with the image format in ShapeNet.
[0110] like Figure 7 As shown, we tested our model on the Pix3D dataset with three categories: "chair," "sofa," and "table." Although most shapes in Pix3D fall outside the distribution range of ShapeNet, our model still generates reasonable predictions in the vast majority of cases. Specifically, compared to the PCDNet model, our predicted point clouds are closer to the true values in terms of shape and detail. Furthermore, our method exhibits robustness to occlusion for reconstruction of occluded parts, such as... Figure 7 As shown in the red box, this further validates the generalization ability and application potential of the SPOS model.
[0111] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.
Claims
1. A single-view point cloud reconstruction method based on occlusion recovery and skeleton generation, characterized in that, Includes the following steps: S1. Generate a coarse point cloud from the input single-view image using a point cloud generation network; S2. The ORnet occlusion recovery network is used to divide the coarse point cloud into regions and extract features. The symmetric relationship between regions is learned through self-attention and cross-attention mechanisms to recover the information of the occluded parts and generate a point cloud with complete information. S3. The generated complete point cloud is downsampled and feature-enhanced using the skeleton generation network SRNet to generate a skeleton point cloud; S4. Use a point cloud refiner to upsample and fuse the skeleton point cloud to generate a final point cloud with complete structure and high smoothness.
2. The single-view point cloud reconstruction method based on occlusion recovery and skeleton generation according to claim 1, characterized in that, The process of generating a coarse point cloud from an input single-view image using a point cloud generation network includes the following steps: The point cloud generation network uses PCDNet, which includes a convolutional neural network and a random initial point cloud. The convolutional neural network performs convolution operations on the input single image to increase the feature dimension of the image and obtain the shape features of the image. The random initial point cloud uses the shape features extracted from multiple fused images to deform the point cloud, optimize the point cloud and integrate the shape feature information to generate a coarse point cloud.
3. The single-view point cloud reconstruction method based on occlusion recovery and skeleton generation according to claim 1, characterized in that, In S2, the occlusion recovery network ORnet uses a combination of the farthest point sampling method and the K-nearest neighbor method to divide the coarse point cloud generated in S1 into regions according to a fixed number of points in each region; the fixed number of points includes 1 center point and 15 neighboring points; and extracts the features of each region point.
4. The single-view point cloud reconstruction method based on occlusion recovery and skeleton generation according to claim 3, characterized in that, The extraction of features for each region point includes the following steps: S201. Calculate the coordinate characteristics of the center point and neighboring points within the region; S202. Use a multilayer perceptron to extract high-dimensional features of the center point and neighboring points respectively, and use the coordinate features of all points to describe the spatial information of all points.
5. The single-view point cloud reconstruction method based on occlusion recovery and skeleton generation according to claim 4, characterized in that, In S2, the symmetric relationship between regions is learned through self-attention and cross-attention mechanisms to recover information from the occluded parts and generate a point cloud with complete information, including: A fixed number of points in each region are input into the occlusion recovery network ORnet. The occlusion recovery network ORnet uses a self-attention mechanism to learn the feature relationship between neighboring points and the center point, and uses the self-attention mechanism to regularize the region features. Input all the region features after self-attention normalization and use the cross-attention mechanism to construct the symmetric relationship between regions to obtain the features of the symmetric regions of the original region features; By stitching the original data with the features of the symmetrical region, the information of the occluded part is restored, and a point cloud with complete information is generated.
6. The single-view point cloud reconstruction method based on occlusion recovery and skeleton generation according to claim 5, characterized in that, The method of constructing symmetrical relationships between regions using a cross-attention mechanism includes calculating the attention score between the input region and the symmetrical region using the following formula: in, For the input area data, For the input region, and These are the linear transformation matrices used to map input features to the query and key spaces, respectively; α ij The attention score represents the correlation weights used to quantify the symmetric relationships between the five regions of a coarse point cloud.
7. The single-view point cloud reconstruction method based on occlusion recovery and skeleton generation according to claim 6, characterized in that, Also includes: Use the obtained attention weight α ij To calculate the weighted sum of the value vector V, thus establishing a symmetric relationship between regions: Y = [y1, y2, ..., y5] (3) Where Y = [y1, y2, ..., y5] represents the result of constructing the symmetric relationship of any 5 regions; This represents the result of establishing a symmetrical relationship between the i-th region feature and the ith region feature.
8. The single-view point cloud reconstruction method based on occlusion recovery and skeleton generation according to claim 1, characterized in that, In S3, the skeleton generation network SRNet includes a skeleton generator: The skeleton generator uses downsampling to generate a first low-resolution point cloud m0 and extracts the first feature f0 of the point cloud m0; it uses a multilayer perceptron (MLP) to generate a second low-resolution point cloud m1 and extracts the second feature f1 of the point cloud m1. The second feature f1 is enhanced by local attention, and the feature enhanced by local attention m1 is interpolated into the first low-resolution point cloud m0 and the first feature f0 using an interpolation method. Then, the point cloud m1 corresponding to the second feature f1 is enhanced again by local attention. The fused point cloud features are input into the skeleton using a multilayer perceptron (MLP) to generate the skeleton point cloud M. sk .
9. The single-view point cloud reconstruction method based on occlusion recovery and skeleton generation according to claim 1, characterized in that, The skeleton generation network SRNet includes a point cloud refiner, which consists of a multilayer perceptron (MLP), skeleton features, and an upsampling layer; S4 includes: S401. Map the skeleton point cloud generated in S3 to a high-dimensional space using a multilayer perceptron (MLP). S402. Perform upsampling and fuse the upsampled point cloud features with the skeleton features; S403. Repeat this process multiple times until a point cloud with a complete structure and high smoothness is obtained.
10. A single-view point cloud reconstruction system based on occlusion recovery and skeleton generation, characterized in that, The method for implementing the above-mentioned single-view point cloud reconstruction based on occlusion recovery and skeleton generation includes: The coarse point cloud generation module generates coarse point clouds from the input single-view image through a point cloud generation network. The occlusion recovery module uses the occlusion recovery network ORnet to perform region division and feature extraction on the coarse point cloud. It learns the symmetric relationship between regions through self-attention and cross-attention mechanisms to recover the information of the occluded parts and generate a point cloud with complete information. The skeleton generation module uses the skeleton generation network SRNet to downsample and enhance the features of the generated complete point cloud to generate a skeleton point cloud; and uses a point cloud refiner to upsample and fuse the features of the skeleton point cloud to generate a final point cloud with complete structure and high smoothness.