Point cloud completion method based on vector quantization codebook space learning

Abstract

The invention discloses a point cloud completion method based on vector quantization codebook space learning, which comprises the following steps: S1, constructing a quantization auto-encoder architecture which comprises a codebook encoder, a double-codebook vector quantization module, a point generator and a decoder; s2, a learning stage: inputting the complete point cloud into a quantized auto-encoder architecture, learning a corresponding codebook space, and pre-training a decoder; and S3, a completion stage: inputting the incomplete point cloud into the learning-completed quantitative auto-encoder architecture. The invention provides a new point cloud completion model VQPCN, and compared with a traditional method based on infinite feature space extraction, the method quantizes an infinite feature space to a finite codebook space, introduces a vector quantization idea into a point cloud completion task, and improves the point cloud completion efficiency. The SA module for point cloud processing, the Transform structure, the pure encoder refinement network and the vector quantization technology are combined to realize point cloud completion, the method has remarkable advantages on a mainstream reference data set, and high-quality completion of missing point clouds is realized.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular relates to a point cloud completion method based on vector quantization codebook space learning. Background Art

[0002] A point cloud is a dataset of points in space that can represent three-dimensional shapes or objects. Point clouds are typically acquired using devices such as 3D scanners. Each point in the dataset represents a set of X, Y, and Z coordinates that describe its position in three-dimensional space. As a crucial form of 3D data representation in computer vision and robotics, point clouds, driven by both traditional algorithmic frameworks and deep neural networks, have been widely used in numerous scenarios, including 3D reconstruction and autonomous driving, due to their inherent information about object geometry and environmental structure.

[0003] However, due to factors such as occlusion, sensor range limitations, and data sparsity, point clouds are generally incomplete, making point cloud completion technology key to reconstructing accurate three-dimensional shapes from incomplete data. Point cloud completion is an important and fundamental task in 3D perception. It aims to predict and restore a complete 3D model based on the partial and incomplete point cloud shape obtained by scanning. However, traditional point cloud completion methods often directly map sparse inputs to dense outputs through continuous feature learning. When dealing with inherently incomplete and geometrically ambiguous inputs, such methods face fundamental limitations: the incompleteness of local point clouds constitutes an ill-posed inverse problem, and the solution space has the possibility of multiple reasonable complete shapes. This multimodal ambiguity makes it difficult for neural networks to establish robust geometric correspondences in the absence of explicit structural constraints, often resulting in topological anomalies or random noise distributions in the reconstruction results. Summary of the Invention

[0004] In response to the above-mentioned deficiencies in the prior art, the point cloud completion method based on vector quantization codebook space learning provided by the present invention addresses the inherent defects of traditional point cloud completion methods in the continuous feature learning process, and solves the problem of feature distribution mismatch between the retained area and the synthesized area caused by the asymmetry of the migrated features, as well as the geometric ambiguity problem caused by the incompleteness of local point clouds.

[0005] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is: a point cloud completion method based on vector quantization codebook space learning, comprising the following steps: S1. Build a quantized autoencoder architecture, including a codebook encoder, a dual-codebook vector quantization module, a point generator, and a decoder. S2, learning phase: input the complete point cloud into the quantized autoencoder architecture, learn the corresponding codebook space, and pre-train the decoder; S3, completion phase: Input the incomplete defect cloud into the learned quantized autoencoder architecture: The corresponding global features and local features are extracted through the codebook encoder, and the optimal codebook index sequence of global features and local features in the codebook space is predicted through the dual-codebook vector quantization module. Complementary codebook data entries are retrieved in the codebook space according to the optimal codebook index sequence. The corresponding global features and local features are replaced by the retrieved complementary codebook data entries to form complete global features and complete local features, which are spliced ​​to form fused codebook features. The fused codebook features are input into the point generator to generate coarse-grained point cloud data, and the hierarchical features are enhanced through the pre-trained decoder to reconstruct the complete point cloud and complete the point cloud completion.

[0006] Furthermore, in step S2, the method for learning the codebook space of the quantized autoencoder architecture is: S21, extract local and global features from the complete point cloud by combining the codebook encoder of the abstraction level and the Transformer module; S22. In the dual-codebook vector quantization module, the extracted local features and global features are matched with the local codebook and the global codebook in the current trainable codebook respectively. The local features and global features are replaced by the closest matching items in the trainable codebook to obtain the local codebook features and the global codebook features, and then the fused codebook features are obtained by concatenating them. S23, inputting the fused codebook features into the point generator to produce coarse-grained point cloud data; S24, performing hierarchical feature enhancement on the coarse-grained point cloud data through a decoder to reconstruct the current high-fidelity point cloud; S25. Repeat steps S21 to S24 until all complete point clouds are reconstructed, learn the codebook space corresponding to the complete point cloud, and obtain a pre-trained decoder.

[0007] Furthermore, the input of the codebook encoder also includes spatial clusters generated by iteratively sampling the farthest point of the input point cloud data; Each cluster in the spatial clustering cluster is represented as a circular area containing a local point subset from the input point cloud data, serving as reference data for the codebook encoder to extract global features and local features; The input point cloud data is a complete point cloud or a partial point cloud.

[0008] Furthermore, the dual-codebook vector quantization module, based on the vector quantization variational autoencoder, adopts a feature discretization paradigm and utilizes the consistency of surface-level features of point cloud data originating from the same 3D surface to cluster perceptually similar surface features into the same code vector. Furthermore, during the learning phase, the local codebook and global codebook of the complete point cloud are learned to obtain a codebook space. The local codebook serves as a high-resolution detail anchor to preserve fine-grained geometric fidelity, and the global codebook serves as a topology coordinator to infer overall shape semantics through incomplete local codebook observations.

[0009] Further, the decoder includes a first pure encoder refinement network and a second pure encoder refinement network connected in sequence; In the first pure encoder refinement network and the second pure encoder refinement network: A hierarchical MLP layer is used to extract the features of the input point cloud data, and the extracted features are passed through a Transformer encoder consisting of three stacked self-attention layers to perform feature enhancement and extract refined features.

[0010] Furthermore, in step S22, during the learning phase, in the dual codebook vector quantization module, the formula for replacing local features and global features with the closest matching item in the trainable codebook is: ; ; Where, represents the (i, j)th global feature or local feature extracted by the codebook encoder, represents the kth data entry in the codebook space, Represents the index corresponding to the entry data in the codebook space, represents the codebook space.

[0011] Furthermore, in step S3, in the completion stage, in the dual codebook vector quantization module, the formula for predicting the codebook index of the global feature and the local feature in the codebook space is: ; Where, represents the predicted codebook index, represents the convolution operation, represents the rectified linear unit, represents the batch normalization layer, represents the input features, represent local features and global features respectively.

[0012] The beneficial effects of the present invention are: (1) The method of the present invention proposes a new point cloud completion model (quantized autoencoder architecture VQPCN). Compared with the traditional method based on infinite feature space extraction, the present invention introduces the vector quantization (VQ) idea into the point cloud completion task for the first time by quantizing the infinite feature space to a finite codebook space. The hierarchical feature extraction process of point cloud processing, the Transformer structure, the pure encoder refinement network and the vector quantization technology are combined to realize point cloud completion. The model VQPCN of the present invention has achieved significant advantages on mainstream benchmark datasets. All indicators have greatly surpassed the existing state-of-the-art level, and achieved high-quality completion of missing point clouds.

[0013] (2) To address the problem of missing information details caused by direct application of vector quantization, the method of the present invention proposes a dual-codebook vector quantization module, which simultaneously obtains global information and local information to achieve effective codebook space learning and perform feature interactive fusion, making it more suitable for the completion of three-dimensional point clouds.

[0014] (3) In the method of the present invention, a pure encoder refinement network is constructed, and the selective enhancement of discriminative features is achieved through the Transformer self-attention mechanism while maintaining the global topological structure; the completion step from rough point cloud to complete point cloud is better realized through the high-quality refinement network architecture, thereby improving the quality of the completed point cloud. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 Flowchart of the point cloud completion method based on vector quantization codebook space learning provided by the present invention.

[0016] Figure 2 This is a structural diagram of the quantized autoencoder architecture provided by the present invention.

[0017] Figure 3 This is a diagram of the refined network structure of the first / second pure encoder provided by the present invention. DETAILED DESCRIPTION

[0018] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

[0019] The embodiment of the present invention provides a point cloud completion method based on vector quantization codebook space learning. Figure 1 As shown, the following steps are included: S1. Build a quantized autoencoder architecture, including a codebook encoder, a dual-codebook vector quantization module, a point generator, and a decoder. S2, learning phase: input the complete point cloud into the quantized autoencoder architecture, learn the corresponding codebook space, and pre-train the decoder; S3, completion phase: Input the incomplete defect cloud into the learned quantized autoencoder architecture: The corresponding global features and local features are extracted through the codebook encoder, and the optimal codebook index sequence of global features and local features in the codebook space is predicted through the dual-codebook vector quantization module. Complementary codebook data entries are retrieved in the codebook space according to the optimal codebook index sequence. The corresponding global features and local features are replaced by the retrieved complementary codebook data entries to form complete global features and complete local features, which are spliced ​​to form fused codebook features. The fused codebook features are input into the point generator to generate coarse-grained point cloud data, and the hierarchical features are enhanced through the pre-trained decoder to reconstruct the complete point cloud and complete the point cloud completion.

[0020] In step S1 of the embodiment of the present invention, the constructed quantized autoencoder architecture VQPCN is as follows Figure 2 As shown in the figure, by integrating the complete shape prior knowledge contained in the 3D point cloud data, a discrete representation learning based on Figure 2 The structured reasoning framework shown in the figure aims to use discrete representation space to resolve important ambiguities in completion tasks and enhance the fine details of incomplete point cloud data. It aims to break through the representation limitations of continuous feature space and establish a deterministic mapping relationship between local observations and global morphology, thereby achieving the coordinated optimization of geometric fidelity and topological consistency while suppressing feature distribution differences.

[0021] In step S2 of the embodiment of the present invention, based on the quantized autoencoder architecture constructed in step S1, in the learning phase of the embodiment, the method for the quantized autoencoder architecture to learn the codebook space is: S21, extract local and global features from the complete point cloud by combining the codebook encoder of the abstraction level and the Transformer module; S22. In the dual-codebook vector quantization module, the extracted local features and global features are matched with the local codebook and the global codebook in the current trainable codebook respectively. The local features and global features are replaced by the closest matching items in the trainable codebook to obtain the local codebook features and the global codebook features, and then the fused codebook features are obtained by concatenating them. S23, inputting the fused codebook features into the point generator to produce coarse-grained point cloud data; S24, performing hierarchical feature enhancement on the coarse-grained point cloud data through a decoder to reconstruct the current high-fidelity point cloud; S25. Repeat steps S21 to S24 until all complete point clouds are reconstructed, learn the codebook space corresponding to the complete point cloud, and obtain a pre-trained decoder.

[0022] In step S3 of the embodiment of the present invention, in the completion phase, the underdetermined completion problem is converted into a constrained code sequence prediction paradigm, which significantly reduces the mapping ambiguity. Through the codebook space learned in the learning phase and the pre-trained decoder, the geometric perception completion based on codebook index prediction is performed in the phase through the dual codebook quantization module. During the inference process, the codebook encoder extracts the local and global features of the input residual defect cloud, and infers the corresponding optimal codebook index sequence through the dual codebook vector quantization module, and then retrieves the complementary codeword in the learned codebook space. When the accurate codebook index is predicted, detail-preserving reconstruction is guaranteed by feature replacement because the decoder inherently maintains the learned geometric prior during the codeword assembly process.

[0023] In this embodiment, a quantized autoencoder architecture is used to correct the ambiguity of partial to complete geometric mapping and compensate for structural defects in the recovery process. In the learning phase, the complete point cloud is used as the initialization input to ensure geometrically consistent latent code learning, which effectively alleviates mapping uncertainty and provides a robust geometric prior through its learned shape semantics.

[0024] In an embodiment of the present invention, in the learning phase and the completion phase, the input of the codebook encoder further includes spatial clusters generated by iteratively sampling the farthest points of the input point cloud data; Each cluster in the spatial clustering cluster is represented as a circular area containing a local point subset from the input point cloud data, which serves as reference data for the codebook encoder to extract global features and local features; wherein the input point cloud data is a complete point cloud or a partial point cloud.

[0025] In an embodiment of the present invention, in a quantized autoencoder architecture, a dual-codebook vector quantization (DC-VQ) module, based on a vector quantization variational autoencoder, adopts a feature discretization paradigm and leverages the consistency of surface-level features of point cloud data originating from the same 3D surface to cluster perceptually similar surface features into the same code vector. Furthermore, during the learning phase, the local and global codebooks for the complete point cloud are learned to obtain a codebook space. The local codebook serves as a high-resolution detail anchor to preserve fine-grained geometric fidelity, and the global codebook serves as a topology coordinator to infer overall shape semantics through incomplete local codebook observations.

[0026] In an embodiment of the present invention, during the learning phase, in the dual codebook vector quantization module, the formula for replacing local features and global features with the closest matching item in the trainable codebook is: ; ; Where, represents the (i, j)th global feature or local feature extracted by the codebook encoder, represents the kth data entry in the codebook space, Represents the index corresponding to the entry data in the codebook space, represents the codebook space.

[0027] In an embodiment of the present invention, in the completion phase, in the dual codebook vector quantization module, the formula for predicting the codebook index of the global feature and the local feature in the codebook space is: ; Where, represents the predicted codebook index, represents the convolution operation, represents the rectified linear unit, represents the batch normalization layer, represents the input features, represent local features and global features respectively.

[0028] In this embodiment, the dual-codebook vector quantization module adopts a discretization mechanism to cluster perceptually similar surface features into a unified code vector, effectively alleviating the randomness of the latent space caused by the irregular sampling pattern while maintaining the representation consistency at the object level.

[0029] In this embodiment, the dual-codebook vector quantization module breaks through the problem of significant imbalance between geometric detail preservation and global structure reasoning in traditional methods by constructing a codebook space of collaboratively optimized local geometric codebook and global structure codebook. The present invention focuses on encoding high-resolution surface microstructures to suppress scanning artifacts through local geometric codebooks, while the global structure codebook realizes cross-regional topological reasoning by implicitly learning complete shape semantics. The two realize hierarchical fusion of fine-grained geometric features and macroscopic morphological constraints through a dynamic routing mechanism, thereby establishing a deterministic mapping relationship between local observations and global priors in the discrete latent space.

[0030] In an embodiment of the present invention, in order to enhance the geometric fidelity of the input point cloud, the decoder designed in the present invention includes a first pure encoder refinement network and a second pure encoder refinement network connected in sequence, which selectively amplify the discriminative features while ensuring the preservation of structural integrity; after obtaining some missing pairs, the pure encoder refinement network is used to drive the refinement process of the coarse-grained point cloud to obtain a high-fidelity final result.

[0031] In this embodiment, if Figure 3 As shown, in the first pure encoder refinement network and the second pure encoder refinement network: A hierarchical MLP layer is used to extract the features of the input point cloud data, and the extracted features are passed through a Transformer encoder consisting of three stacked self-attention layers to perform feature enhancement and extract refined features.

[0032] Specifically, in order to obtain a high-quality complete point cloud, the present invention only stacks the Transformer encoder to extract local features, uses three stacked self-attention layers to enhance feature representation, and then estimates the point offset by integrating the MLP point shuffling mechanism and combining it with the reshaping operation. The point shuffling process is achieved by redistributing the predicted offset, and its calculation formula is as follows: ; Where, Indicates a repeated operation. represents the point shuffling process, represents three stacked Transformer encoders, Represents the feature representation obtained from the previous input; specifically, the coarse-grained point cloud data P1 is input, the first pure encoder refines the network to extract feature F1, and then the refined point cloud P2 is obtained, and the second pure encoder refines the network to extract feature F2, and then the complete point cloud P3 is obtained.

[0033] Specific embodiments are used in the present invention to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

[0034] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and it should be understood that the scope of protection of the present invention is not limited to such specific descriptions and embodiments. Those skilled in the art can make various other specific variations and combinations based on the technical teachings disclosed in the present invention without departing from the essence of the present invention, and such variations and combinations are still within the scope of protection of the present invention.

Claims

1. A point cloud completion method based on vector quantization codebook space learning, characterized in that: The following steps are involved: S1. Build a quantized autoencoder architecture, including a codebook encoder, a dual-codebook vector quantization module, a point generator, and a decoder. S2, learning phase: input the complete point cloud into the quantized autoencoder architecture, learn the corresponding codebook space, and pre-train the decoder; S3, completion phase: Input the incomplete defect cloud into the learned quantized autoencoder architecture: The corresponding global features and local features are extracted through the codebook encoder, and the optimal codebook index sequence of global features and local features in the codebook space is predicted through the dual-codebook vector quantization module. Complementary codebook data entries are retrieved in the codebook space according to the optimal codebook index sequence. The corresponding global features and local features are replaced by the retrieved complementary codebook data entries to form complete global features and complete local features, which are spliced ​​to form fused codebook features. The fused codebook features are input into the point generator to generate coarse-grained point cloud data, and the hierarchical features are enhanced through the pre-trained decoder to reconstruct the complete point cloud and complete the point cloud completion.

2. The point cloud completion method based on vector quantization codebook learning according to claim 1, characterized in that: In step S2, the method for learning the codebook space of the quantized autoencoder architecture is: S21, extract local and global features from the complete point cloud by combining the codebook encoder of the abstraction level and the Transformer module; S22. In the dual-codebook vector quantization module, the extracted local features and global features are matched with the local codebook and the global codebook in the current trainable codebook respectively. The local features and global features are replaced by the closest matching items in the trainable codebook to obtain the local codebook features and the global codebook features, and then the fused codebook features are obtained by concatenating them. S23, inputting the fused codebook features into the point generator to produce coarse-grained point cloud data; S24, performing hierarchical feature enhancement on the coarse-grained point cloud data through a decoder to reconstruct the current high-fidelity point cloud; S25. Repeat steps S21 to S24 until all complete point clouds are reconstructed, learn the codebook space corresponding to the complete point cloud, and obtain a pre-trained decoder.

3. The point cloud completion method based on vector quantization codebook learning according to claim 1, characterized in that: The input of the codebook encoder also includes a spatial cluster generated by iteratively sampling the farthest point of the input point cloud data; Each cluster in the spatial clustering cluster is represented as a circular area containing a local point subset from the input point cloud data, serving as reference data for the codebook encoder to extract global features and local features; The input point cloud data is a complete point cloud or a partial point cloud.

4. The point cloud completion method based on vector quantization codebook learning according to claim 1, characterized in that: The dual-codebook vector quantization module, based on the self-vector quantization variational autoencoder, adopts a feature discretization paradigm and utilizes the consistency of surface-level features of point cloud data originating from the same 3D surface to cluster perceptually similar surface features into the same code vector. Furthermore, during the learning phase, the local codebook and global codebook of the complete point cloud are learned to obtain a codebook space. The local codebook serves as a high-resolution detail anchor to preserve fine-grained geometric fidelity, and the global codebook serves as a topology coordinator to infer overall shape semantics through incomplete local codebook observations.

5. The point cloud completion method based on vector quantization codebook learning according to claim 1, characterized in that: The decoder includes a first pure encoder refinement network and a second pure encoder refinement network connected in sequence; In the first pure encoder refinement network and the second pure encoder refinement network: A hierarchical MLP layer is used to extract the features of the input point cloud data, and the extracted features are passed through a Transformer encoder consisting of three stacked self-attention layers to perform feature enhancement and extract refined features.

6. The point cloud completion method based on vector quantization codebook learning according to claim 2, characterized in that: In step S22, during the learning phase, in the dual codebook vector quantization module, the formula for replacing local features and global features with the closest matching item in the trainable codebook is: ； ； Where, represents the (i, j)th global feature or local feature extracted by the codebook encoder, represents the kth data entry in the codebook space, Represents the index corresponding to the entry data in the codebook space, represents the codebook space.

7. The point cloud completion method based on vector quantization codebook learning according to claim 1, characterized in that: In step S3, in the completion phase, in the dual codebook vector quantization module, the formula for predicting the codebook index of the global feature and the local feature in the codebook space is: ； Where, represents the predicted codebook index, represents the convolution operation, represents the rectified linear unit, represents the batch normalization layer, represents the input features, represent local features and global features respectively.

Images