An octree and multi-granularity grid fusion-based point cloud completion method

Abstract

The application relates to an octree and multi-granularity grid fusion-based point cloud completion method, and belongs to the technical field of point cloud reconstruction, and solves the problem that existing lost point clouds cannot restore reasonable point cloud shapes due to the loss of point cloud details. The method comprises the following steps: after normalizing the point cloud to be completed, the point cloud features are extracted through an encoder; an octree is constructed, the importance scores of the nodes in the octree are calculated, the key nodes and the point cloud features thereof are extracted; the point cloud features of the key nodes are aligned and fused with the point cloud features of the point cloud to be completed through cross attention, then multi-granularity gridding projection and fusion are carried out, and the fusion features of the key nodes are obtained; the fusion features of the key nodes are distributed to the corresponding nodes of the octree, the nodes of the octree are optimized through a self-attention mechanism, the global point cloud features after optimization are transmitted into a decoder, and the completed point cloud is obtained. The effective identification and repair of the point cloud missing area are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of point cloud reconstruction, in particular to a point cloud completion method based on octree and multi-granularity grid fusion. BACKGROUND

[0002] Point cloud, as a discrete data representation of three-dimensional object surface, is obtained by sensors such as laser radar and depth camera, and is widely used in automatic driving, industrial detection and other fields. Point cloud records the spatial geometric information of the object surface, and is the key carrier connecting the physical world and the digital model. With the rapid development of three-dimensional perception and modeling technology, the completeness and accuracy of point cloud data are increasingly required.

[0003] Point cloud completion aims to predict and repair incomplete 3D point cloud data through algorithms, and its goal is to reconstruct complete 3D shapes from partial scans, occlusions or incomplete point clouds, and to restore missing geometric structures and details. This technology is a key technology link to improve the robustness and reliability of three-dimensional systems. At present, the mainstream point cloud completion methods can be mainly divided into two categories: traditional geometry processing-based methods and deep learning-based methods.

[0004] Traditional geometry processing-based methods usually rely on local geometric properties of point cloud (such as normal vector, curvature) for surface interpolation or extrapolation. Although it can maintain certain geometric accuracy locally, when the missing area of point cloud is large, there is not enough local context information to guide the repair, resulting in the failure of the method.

[0005] Deep learning-based methods usually use an encoder-decoder architecture to map the input incomplete point cloud to a low-dimensional latent space, and then decode the complete point cloud from it. Although these methods have made significant progress in overall shape generation, information compression during encoding and the dominant role of global features can easily lead to the generated completed point cloud becoming smooth and blurred in detail areas such as edges and holes, lacking sharp geometric features. When dealing with complex missing, it is difficult to ensure the continuity of the completed part and the known part in terms of geometric structure and semantics, which may lead to the generation of unreasonable or distorted geometric structures. SUMMARY

[0006] In view of the above analysis, the embodiments of the present application aim to provide a point cloud completion method based on octree and multi-granularity grid fusion, to solve the problem that the existing point cloud geometric feature recognition is not accurate, resulting in missing point cloud details and inability to restore reasonable point cloud shapes.

[0007] The embodiments of the present application provide a point cloud completion method based on octree and multi-granularity grid fusion, comprising the following steps:

[0008] After normalizing the point cloud to be completed, the point cloud features are extracted by the encoder;

[0009] An octree is constructed based on the normalized point cloud to be completed and the point cloud features, and key nodes and their point cloud features are extracted by calculating the importance scores of the nodes in the octree;

[0010] The point cloud features of the key nodes are aligned and fused with the point cloud features of the point cloud to be completed through cross attention, to obtain the aligned point cloud features of the key nodes, and then multi-granularity rasterization projection and fusion are performed to obtain the fusion features of the key nodes;

[0011] The fusion features of the key nodes are assigned to the corresponding nodes of the octree, and each node of the octree is optimized through a self-attention mechanism to obtain the optimized global point cloud features;

[0012] The optimized global point cloud features are input into a decoder to obtain the completed point cloud.

[0013] Based on the further improvement of the above method, multi-granularity rasterization projection and fusion are performed to obtain the fusion features of the key nodes, including:

[0014] The aligned point cloud features of the key nodes are projected onto three-dimensional grids of different resolutions to form a multi-resolution feature map pyramid;

[0015] A first feature is extracted by applying a dilated convolution to the medium and high resolution features of the feature map pyramid, a second feature is extracted by applying a self-attention mechanism to the low resolution features of the feature map pyramid, and the first feature and the upsampled second feature are fused through a gating fusion mechanism to obtain the fusion features of the key nodes.

[0016] Based on the further improvement of the above method, the first feature and the upsampled second feature are fused through a gating fusion mechanism, including:

[0017] The first feature and the upsampled second feature are concatenated in the channel dimension; after convolution operation on the concatenated features, a Sigmoid function is input to generate a spatially adaptive gating weight map;

[0018] The first feature and the upsampled second feature are weighted and summed using the gating weight map.

[0019] Based on the further improvement of the above method, the decoder includes two multi-layer perceptrons connected in series, wherein the first multi-layer perceptron generates a reference point cloud set according to the optimized global point cloud features; the optimized global point cloud features are propagated to the reference point cloud set and fused with the reference point cloud set, and then input into the second multi-layer perceptron to predict the displacement field of each point cloud; the displacement field of each point cloud is added to the reference point set according to the corresponding points, and after inverse normalization processing, the completed point cloud is output.

[0020] Based on the further improvement of the above method, the importance score of each node in the octree is calculated by calculating the weighted sum of the curvature, density gradient and boundary distance of each node; the curvature is obtained by calculating the proportion of the smallest eigenvalue of the covariance matrix of the point cloud within the node; the density gradient is obtained by calculating the partial derivatives of the node density field in the three coordinate directions and summing the absolute values; the boundary distance is obtained by calculating the minimum distance from the point cloud within the node to the node boundary.

[0021] Based on the further improvement of the above method, key nodes are extracted by obtaining leaf nodes whose importance scores are greater than their corresponding final segmentation thresholds. The final segmentation threshold is the final value obtained by dynamically adjusting the segmentation threshold during the octree construction process according to the importance score of the node, the current recursion depth, the node offset, and the current node side length until the maximum recursion depth or the minimum node size is reached.

[0022] Based on the above method, a further improvement is made by aligning and fusing the point cloud features of the key nodes with the point cloud features of the point cloud to be completed through cross-attention, resulting in aligned point cloud features of the key nodes, including:

[0023] The query matrix is ​​calculated based on the point cloud features of the key nodes, the key matrix and value matrix are calculated based on the point cloud features of the point cloud to be completed, and then the enhanced features of the key nodes are obtained through cross attention.

[0024] After residual connection and layer normalization of the enhanced features of key nodes, the point cloud features of the key nodes are aligned and output by the feedforward neural network.

[0025] Based on the above method, a further improvement is made to the point cloud to be completed, which involves normalization processing, including:

[0026] Obtain the 2D image corresponding to the point cloud to be completed;

[0027] The scale factor is calculated based on the camera focal length, the distance to the target object, and the pixel height in the image.

[0028] After translating the point cloud coordinates to the center point, scale them to the unit sphere space using a scale factor.

[0029] Based on the above method, further improvements are made to the octree by optimizing each node through a self-attention mechanism to obtain optimized global point cloud features, including:

[0030] Nodes belonging to the same parent node or the same depth are divided into blocks. A self-attention mechanism is calculated within each block. The calculation results of each block are subjected to residual connection and layer normalization to obtain the optimized point cloud features of each block.

[0031] By stitching together the optimized point cloud features of all blocks and mapping them to the target dimension, the optimized global point cloud features are obtained.

[0032] Based on the above method, the network models used in the method are jointly trained in an end-to-end manner. During the training process, the joint loss function is minimized and the learnable parameters of each network model are updated through backpropagation. The joint loss function is a weighted sum of chamfer distance loss, Earth movement distance loss and displacement field regularization term loss.

[0033] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0034] 1. By introducing an octree spatial index structure and a multi-granularity raster fusion mechanism, combined with attention-driven feature alignment and optimization strategies, missing regions in point clouds are effectively identified and repaired. The generated point clouds are rich in detail and structurally complete, improving computational efficiency and significantly enhancing the accuracy, detail preservation, and semantic consistency of point cloud completion.

[0035] 2. Octree segmentation based on multi-index importance scores: By integrating curvature, density gradient and boundary distance into a multi-index scoring mechanism, combined with a dynamic threshold adjustment strategy, it achieves adaptive identification of key regions for point cloud completion, accurately locates key regions such as edges and holes, and combines multi-granularity feature dual-channel fusion to better preserve sharp features and local geometric details during the completion process, avoiding over-smoothing.

[0036] 3. By using cross-attention and block self-attention mechanisms, we ensure that the completed part is consistent with the known part in terms of geometric structure and semantics, thus generating a reasonable overall shape.

[0037] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0038] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0039] Figure 1 This is a flowchart of a point cloud completion method based on the fusion of an octree and a multi-granularity raster in an embodiment of the present invention. Detailed Implementation

[0040] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0041] A specific embodiment of the present invention discloses a point cloud completion method based on the fusion of octree and multi-granularity raster, such as... Figure 1 As shown, it includes the following steps:

[0042] S1. After normalizing the point cloud to be completed, the point cloud features are extracted by the encoder.

[0043] Point cloud data is usually collected by sensors such as 3D LiDAR and depth cameras, and sometimes some areas are missing due to occlusion, noise or limited viewpoint.

[0044] First, the point cloud data to be completed is normalized to eliminate scale differences, including:

[0045] Obtain the 2D image corresponding to the point cloud to be completed; that is, obtain the 2D RGB depth map corresponding to the 3D point cloud data, so that the data of these two modalities match in physical size.

[0046] The scale factor is calculated based on the camera focal length, the distance to the target object, and the pixel height in the image.

[0047] After translating the point cloud coordinates to the center point, scale them to the unit sphere space using a scale factor.

[0048] Specifically, the normalized point cloud coordinates are calculated using the following formula:

[0049] ,

[0050] in, and They represent the first The normalized coordinates and original coordinates of a point cloud Indicates the scale factor. Indicates the camera's focal length; This indicates the distance from the target object to the camera, in meters. This represents the actual pixel height of the target object in the image; Indicates the coordinates of the center point; This indicates the total number of point clouds to be completed; This represents the maximum distance from all point cloud coordinates to the center point, serving as the normalization reference.

[0051] Furthermore, the normalized point cloud is input into the encoder to extract its depth features. In this embodiment, the encoder uses a Dynamic Graph Convolutional Neural Network (DGCNN), which outputs multi-scale features as point cloud features through multiple edge convolution operations.

[0052] Specifically, feature extraction using dynamic graph convolutional neural networks includes:

[0053] For each point in the normalized point cloud, multiple nearest neighbors are found in the feature space defined by cosine similarity to construct a local graph.

[0054] Calculate the feature difference between each point and its nearest neighbor, extract edge features through a multilayer perceptron with shared weights, and aggregate them through max pooling; repeat the above edge convolution operation at least three times, and concatenate the features output by each edge convolution operation to obtain multi-scale point cloud features.

[0055] It should be noted that when finding the nearest neighbor, the feature space of the first EdgeConv edge convolution operation is the coordinate space, and the subsequent ones are high-level feature spaces. That is, in the first EdgeConv layer, its input features are the initial three-dimensional coordinates of each point in the normalized point cloud and low-level attributes such as color. At this time, cosine similarity is calculated in this feature space to construct the local graph structure. In the subsequent EdgeConv layers, the features used to calculate cosine similarity are the high-order abstract features output by the previous EdgeConv layer.

[0056] For example, the number of nearest neighbors is 20, and 3 edge convolution operations are performed.

[0057] The formula for calculating multi-scale point cloud features is:

[0058] ,

[0059] in, This represents the point cloud features of the point cloud to be completed. , Each point cloud Dimensional features; Point cloud after normalization The number of nearest neighbors, This represents a multilayer perceptron with shared weights. Indicates feature splicing, This indicates a max pooling operation. Indicates the number of edge convolution operations. Indicates the total number of times; This represents the normalized set of point clouds. Point cloud The The nearest neighbor.

[0060] S2. Construct an octree based on the normalized point cloud to be completed and its features. Extract key nodes and their point cloud features by calculating the importance score of each node in the octree.

[0061] It should be noted that, based on the normalized point cloud data and point cloud features, an octree structure is constructed, including:

[0062] Using the bounding box of the point cloud as the root node, set the initial segmentation threshold and the minimum node size (i.e., the minimum voxel size); where the minimum node size is twice the average point spacing of the point cloud (the average point spacing is obtained by globally averaging the distances between each point and its nearest neighbor).

[0063] Starting from the root node, the space containing the point cloud bounding box is recursively subdivided into eight child nodes (i.e., voxels) until the minimum node size is reached. At the same time, the segmentation threshold (subdivision granularity) is dynamically adjusted: high curvature regions are subdivided finely, and flat regions are subdivided coarsely.

[0064] During the recursive subdivision process, an importance score is calculated for each node to identify key regions (such as edges, holes, and sharp areas). The importance score is obtained by calculating the weighted sum of the curvature, density gradient, and boundary distance of each node; where curvature is obtained by calculating the proportion of the smallest eigenvalue of the point cloud covariance matrix within the node; density gradient is obtained by calculating the partial derivatives of the node's density field in the three coordinate directions and summing their absolute values; and boundary distance is obtained by calculating the minimum distance from the point cloud within the node to the node boundary.

[0065] It should be noted that curvature represents the degree of bending of the point cloud surface within a node; high curvature typically corresponds to edges or holes. Density gradient represents the rate of change of point density between nodes; abrupt changes usually occur at node boundaries. Boundary distance represents the distance from a node to a missing region of the point cloud; a smaller value indicates that the points within the node are closer to the boundary, which may be an edge or transition region, while a larger value may indicate a flat region. Together, these three factors constitute a complete importance assessment.

[0066] Specifically, the formula for calculating the importance score is as follows:

[0067] ,

[0068] in, Represents a node Importance score , and Representing nodes respectively The curvature, density gradient, and boundary distance, , and These represent the weights of curvature, density gradient, and boundary distance, respectively. ; Represents a node The minimum eigenvalue of the point cloud covariance matrix. Represents a node The sum of all eigenvalues ​​of the point cloud covariance matrix; , and These represent the nodal density fields at... , and Partial derivatives in the coordinate directions; Represents a node The geometric boundary; Represents a node Normalized point clouds, Represents a node The number of point clouds in the data.

[0069] It should be noted that the initial splitting threshold is dynamically adjusted during the octree construction process based on the node's importance score, the current recursion depth, the node offset, and the current node's side length, including:

[0070] During the recursive subdivision process, if the importance score of a node is greater than the current segmentation threshold and the current depth has not reached the maximum value, offset detection is performed. The average Euclidean distance between all points within the node and the node center is calculated as the node offset. If the node offset exceeds the preset offset threshold, it is marked as a high-sensitivity region, and a preset proportion of the current node's side length, such as 10%, is taken as the segmentation threshold for the next subdivision. The subdivision is then continued, and the segmentation threshold is dynamically adjusted to the minimum node size. Otherwise, the subdivision is terminated.

[0071] After the recursion is completed, for each leaf node, the actual effective segmentation threshold of its level is recorded as the final segmentation threshold for that node; all nodes with importance scores greater than their corresponding final segmentation thresholds are marked as key nodes.

[0072] Establish a mapping relationship between the nodes in the octree to record the relationship between each point cloud point and the octree node.

[0073] The point cloud and its corresponding DGCNN point cloud features within key nodes are extracted. Then, within each key node, the point cloud features of all point clouds are aggregated using average pooling to obtain the point cloud features of the key node. When there are... There are 1 key node, and the point cloud features of each key node are as follows: In dimension 1, the point cloud features of all key nodes are represented as follows: .

[0074] It should be noted that, Represents the set of real numbers. express A real vector space of dimension 1.

[0075] S3. Align and fuse the point cloud features of the key nodes with the point cloud features of the point cloud to be completed by cross attention to obtain the aligned point cloud features of the key nodes. Then perform multi-granularity rasterization projection and fusion to obtain the fused features of the key nodes.

[0076] A cross-attention mechanism is used to align and fuse the point cloud features of key nodes with those of the point cloud to be filled in, thereby enhancing the semantic consistency of the missing regions. This includes:

[0077] Based on the point cloud features of key nodes Calculate the query matrix Based on the point cloud features of the point cloud to be completed Calculate the key matrix Sum matrix Then, enhanced features of key nodes are obtained through cross-attention. The formula is shown below:

[0078] ,

[0079] in, , and These represent the query matrix weight, key matrix weight, and value matrix weight, respectively. Indicates the scaling factor. This indicates the transpose operation.

[0080] The enhanced features of key nodes are subjected to residual connections and layer normalization, and the aligned point cloud features of the key nodes are output through a feedforward neural network, as shown in the following formula:

[0081] ,

[0082] in, The feature matrix after the layer normalization is represented. Represents the projection matrix. Presentation layer normalization operation.

[0083] Finally, The input is fed into a feedforward neural network (FFN) for further feature processing and dimension adaptation to obtain the aligned point cloud features of the key nodes. The feedforward neural network consists of two linear transformation layers and a nonlinear activation function between them.

[0084] The formula is shown below:

[0085] ,

[0086] in, This represents the point cloud features of the aligned key nodes. This indicates that a feedforward neural network is being used for processing. This represents the dimension of the point cloud features for each aligned key node.

[0087] It should be noted that the point cloud features of the aligned key nodes not only incorporate global context information but also undergo nonlinear transformation, giving them stronger representation capabilities and providing high-quality input for subsequent multi-granularity raster fusion.

[0088] Furthermore, the aligned key node features are projected onto a multi-granularity raster and fused using a dual-path method to simultaneously capture details and global structure, including:

[0089] The point cloud features of the aligned key nodes are projected onto 3D grids of different resolutions to form a multi-resolution feature map pyramid.

[0090] Dilated convolution is applied to the medium and high resolution features of the feature map pyramid to extract the first feature, and self-attention mechanism is applied to the low resolution features of the feature map pyramid to extract the second feature. The first feature and the upsampled second feature are then fused through a gated fusion mechanism to obtain the fused features of the key nodes.

[0091] Specifically, rasterization projection divides 3D space into voxel grids of different resolutions (e.g., 128). 3 64 3 32 3 For each key node, its point cloud features are interpolated (e.g., trilinear interpolation) to the corresponding grid position. High resolution (128³) preserves edge details, is first voxelized, and then downsampled to medium resolution (64³) and low resolution (32³) using a 3×3×3 convolution with a stride of 2. Trilinear interpolation is used for each downsampling step, maintaining geometric continuity, thus completing multi-scale downsampling. Features are aggregated within each voxel grid using max pooling or average pooling as raster feature values. Finally, raster feature values ​​of different resolutions are integrated to form a multi-resolution feature map pyramid.

[0092] The feature map at each resolution level is represented by the following formula:

[0093] ,

[0094] in, Indicates the first Feature maps with high resolution. , This indicates the overall level of resolution, usually ; Indicates the first Level resolution, This indicates the highest resolution at level 1, where the depth dimension is the same as the height and width dimensions.

[0095] Furthermore, a dual-path fusion mechanism is adopted: 3D dilated convolution is applied to medium and high resolution features to extract the first feature. This is used to extract fine-grained features such as edges and textures. For example, the dilated convolution kernel size is set to 3, and the dilation / dilation ratio is set to 2.

[0096] A self-attention mechanism is applied to extract the second feature from the low-resolution features of the feature map pyramid. To capture long-range dependencies and global structural features. The second feature... Upsampling to At the same resolution, .

[0097] Finally, the first feature and the upsampled second feature are fused using a gating fusion mechanism to obtain the fused features of the key nodes, including:

[0098] The first feature is concatenated with the upsampled second feature along the channel dimension; the concatenated feature is then convolved and input into the Sigmoid function to generate a spatially adaptive gated weight map. The formula is shown below:

[0099] ,

[0100] in, This represents the Sigmoid function. This indicates a convolution operation, for example, using a 1×1 or 3×3 convolution kernel.

[0101] By using a gated weight map to perform a weighted summation of the first feature and the upsampled second feature, the fused features of the key nodes are obtained. The formula is shown below:

[0102] ,

[0103] in, This represents element-wise multiplication, i.e., the Hadamard product; This indicates element-wise addition, which merges the weighted mid-to-high frequency and low-frequency features.

[0104] After the above steps, the fused features of the key nodes contain complete information from local details to global context. These features are obtained by deeply optimizing the point cloud features of the original key nodes.

[0105] S4. Assign the fused features of the key nodes to the corresponding nodes of the octree, and optimize each node of the octree through the self-attention mechanism to obtain the optimized global point cloud features.

[0106] It should be noted that, based on the mapping relationship established when constructing the octree in step S2, the fusion features of each key node are assigned to the nodes of its respective octree according to the index of each key node.

[0107] Furthermore, starting from the root node of the octree, each node of the octree is optimized level by level using a self-attention mechanism. Specifically, this means:

[0108] Nodes belonging to the same parent node or the same depth are grouped into blocks. A self-attention mechanism is computed within each block. The computation results for each block are then subjected to residual connections and layer normalization to obtain the optimized point cloud features for each block. The optimized point cloud features of all blocks are then concatenated and mapped to the target dimension to obtain the optimized global point cloud features, as shown in the following formula:

[0109] ,

[0110] in, This represents the optimized global point cloud features. and They represent the first Point cloud features before and after block optimization Indicates the total number of blocks. ; This indicates that self-attention calculation is being performed. Indicates the first Number of point clouds within each block This indicates a splicing operation. This represents the point cloud features after all blocks are stitched together. This represents a learnable weight matrix used to reduce the feature dimension from... Dimensional mapping to target dimension .

[0111] It should be noted that the optimized global point cloud features retain details of key regions while maintaining global consistency. Furthermore, the self-attention mechanism is computed only within each block, thus reducing the computational complexity from O(N^2) to O(N^2). 2 The value decreases to O(Nlog N).

[0112] S5. Input the optimized global point cloud features into the decoder to obtain the completed point cloud.

[0113] It should be noted that the decoder includes two multilayer perceptrons connected in series. The first multilayer perceptron generates a reference point cloud set based on the optimized global point cloud features. The optimized global point cloud features are then propagated to the reference point cloud set and fused with it before being fed into the second multilayer perceptron to predict the displacement field of each point cloud. The displacement field of each point cloud is then added to the reference point set at corresponding points, and after inverse normalization, the completed point cloud is output.

[0114] Specifically, the first multilayer perceptron generates a reference point cloud set based on the optimized global point cloud features, including:

[0115] A 2D mesh is initialized, and the 2D coordinates of each mesh point are concatenated with the optimized global point cloud features to obtain a combined vector. This vector is then input into a shared multilayer perceptron. The last layer of the multilayer perceptron has an output dimension of 3. A nonlinear mapping function is used to output a 3D coordinate for each input combined vector, which is then combined to form a reference point cloud, providing a coarse spatial distribution framework.

[0116] It should be noted that the number of 2D mesh points (i.e. the number of points in the reference point cloud set) is the number of target point clouds, which is usually set based on prior knowledge of the point density of the complete object.

[0117] Furthermore, since the number of points corresponding to the optimized global point cloud features is the number of points to be filled in, which is less than the number of points in the reference point cloud set, the optimized global point cloud features are propagated to the reference point cloud set through nearest neighbor interpolation or trilinear interpolation. Then, the 3D coordinates of each point in the reference point cloud set are concatenated with the corresponding propagated global point cloud features and fed into the second multilayer perceptron to decode and output the 3D displacement field of each point.

[0118] Finally, by adding the 3D displacement field of each point to the reference point cloud point by point and then inversely normalizing it to the original physical scale, a refined and structurally complete point cloud coordinate can be obtained. The displacement field is responsible for local fine-tuning of the coarse reference points to accurately restore the detailed geometry, thus realizing end-to-end reconstruction from feature semantics to spatial coordinates.

[0119] It should be noted that the network models used in this embodiment are jointly trained end-to-end, including: the encoder in step S1, the cross-attention and multi-granularity grid fusion in step S3, the self-attention mechanism in step S4, and the decoder in step S5. During training, the joint loss function is minimized and the learnable parameters of each network model are updated through backpropagation.

[0120] Among them, the joint loss function It is the chamfer distance loss. Earth's distance traveled and displacement field The weighted sum of the regularization loss, the chamfer distance loss is used to measure the shape matching degree between the predicted point cloud and the real complete point cloud; the Earth movement distance loss is used to ensure the uniformity of the generated point distribution; and the L2 regularization loss of the displacement field is used to prevent overfitting.

[0121] Joint loss function The calculation formula is as follows:

[0122] ,

[0123] in, , and These represent the weights of the chamfer distance loss, the Earth movement distance loss, and the displacement field regularization term loss, respectively; for example, they are set to 1.0, 0.5, and 0.1, respectively. Describing the L2 norm, Indicates the quantity to be retrieved; To create a true and complete point cloud; To predict the generated point cloud, This represents a bijection from the predicted point cloud to the actual complete point cloud.

[0124] Compared with existing technologies, this embodiment provides a point cloud completion method based on octree and multi-granularity raster fusion. By introducing an octree spatial index structure and a multi-granularity raster fusion mechanism, combined with attention-driven feature alignment and optimization strategies, it effectively identifies and repairs missing regions in point clouds. The generated point clouds are rich in detail and structurally complete, improving computational efficiency and significantly enhancing the accuracy, detail preservation, and semantic consistency of point cloud completion. Based on octree segmentation with multi-index importance scores, and through a multi-index scoring mechanism that integrates curvature, density gradient, and boundary distance, combined with a dynamic threshold adjustment strategy, it achieves adaptive identification of key regions for point cloud completion, accurately locating key regions such as edges and holes. Combined with multi-granularity feature dual-channel fusion, it better preserves sharp features and local geometric details during the completion process, avoiding over-smoothing. Through cross-attention and block self-attention mechanisms, it ensures that the completed part is consistent with the known part in terms of geometric structure and semantics, generating a reasonable overall shape.

[0125] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0126] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A point cloud completion method based on octree and multi-granularity raster fusion, characterized in that, Includes the following steps: After normalizing the point cloud to be completed, the point cloud features are extracted by the encoder. An octree is constructed based on the normalized point cloud to be completed and its features. The key nodes and their point cloud features are extracted by calculating the importance score of each node in the octree. The point cloud features of key nodes are aligned and fused with the point cloud features of the point cloud to be completed by cross-attention, resulting in aligned point cloud features of key nodes. Multi-granularity raster projection and fusion are then performed to obtain fused features of key nodes. This process includes: projecting the aligned point cloud features of key nodes onto 3D rasteres of different resolutions to form a multi-resolution feature map pyramid; applying dilated convolution to the medium and high resolution features of the feature map pyramid to extract first features; applying a self-attention mechanism to the low resolution features of the feature map pyramid to extract second features; and fusing the first features with the upsampled second features through a gated fusion mechanism to obtain fused features of key nodes. The fused features of the key nodes are assigned to the corresponding nodes of the octree, and the nodes of the octree are optimized through a self-attention mechanism to obtain the optimized global point cloud features. The optimized global point cloud features are fed into the decoder to obtain the completed point cloud.

2. The point cloud completion method based on octree and multi-granularity raster fusion according to claim 1, characterized in that, The process of fusing the first feature with the upsampled second feature through a gating fusion mechanism includes: The first feature and the upsampled second feature are concatenated along the channel dimension; the concatenated features are then convolved and input into the Sigmoid function to generate a spatially adaptive gated weight map. The first feature and the upsampled second feature are weighted and summed using the gated weight graph.

3. The point cloud completion method based on octree and multi-granularity raster fusion according to claim 1, characterized in that, The decoder includes two multilayer perceptrons connected in series. The first multilayer perceptron generates a reference point cloud set based on the optimized global point cloud features. The optimized global point cloud features are propagated to the reference point cloud set and fused with the reference point cloud set before being fed into the second multilayer perceptron to predict the displacement field of each point cloud. The displacement field of each point cloud is added to the reference point set at corresponding points and then subjected to inverse normalization to output the completed point cloud.

4. The point cloud completion method based on octree and multi-granularity raster fusion according to claim 1, characterized in that, The importance score of each node in the octree is obtained by calculating the weighted sum of the curvature, density gradient and boundary distance of each node; the curvature is obtained by calculating the proportion of the minimum eigenvalue of the point cloud covariance matrix within the node. The density gradient is obtained by calculating the partial derivatives of the node density field in the three coordinate directions and summing their absolute values; the boundary distance is obtained by calculating the minimum distance from the point cloud inside the node to the node boundary.

5. The point cloud completion method based on octree and multi-granularity raster fusion according to claim 1 or 4, characterized in that, The key nodes are extracted by obtaining leaf nodes whose importance scores are greater than their corresponding final segmentation thresholds. The final segmentation threshold is the final value obtained by dynamically adjusting the segmentation threshold during the octree construction process based on the node's importance score, current recursion depth, node offset, and current node side length until the maximum recursion depth or minimum node size is reached.

6. The point cloud completion method based on octree and multi-granularity raster fusion according to claim 1, characterized in that, The step of aligning and fusing the point cloud features of key nodes with the point cloud features of the point cloud to be completed through cross-attention to obtain aligned point cloud features of key nodes includes: The query matrix is ​​calculated based on the point cloud features of the key nodes, the key matrix and value matrix are calculated based on the point cloud features of the point cloud to be completed, and then the enhanced features of the key nodes are obtained through cross attention. After residual connection and layer normalization of the enhanced features of key nodes, the point cloud features of the key nodes are aligned and output by the feedforward neural network.

7. The point cloud completion method based on octree and multi-granularity raster fusion according to claim 1, characterized in that, The normalization process for the point cloud to be completed includes: Obtain the 2D image corresponding to the point cloud to be completed; The scale factor is calculated based on the camera focal length, the distance to the target object, and the pixel height in the image. After translating the point cloud coordinates to the center point, the scale factor is used to scale it to the unit sphere space.

8. The point cloud completion method based on octree and multi-granularity raster fusion according to claim 1, characterized in that, The optimization of each node of the octree using a self-attention mechanism to obtain optimized global point cloud features includes: Nodes belonging to the same parent node or the same depth are divided into blocks. A self-attention mechanism is calculated within each block. The calculation results of each block are subjected to residual connection and layer normalization to obtain the optimized point cloud features of each block. By stitching together the optimized point cloud features of all blocks and mapping them to the target dimension, the optimized global point cloud features are obtained.

9. The point cloud completion method based on octree and multi-granularity raster fusion according to claim 1, characterized in that, The network models used in the method are obtained through end-to-end joint training. During the training process, the joint loss function is minimized and the learnable parameters of each network model are updated through backpropagation. The joint loss function is a weighted sum of chamfer distance loss, Earth movement distance loss and displacement field regularization term loss.

Images