3D point cloud completion algorithm based on generative AI

Abstract

The invention discloses a 3D point cloud completion algorithm based on generative AI, and the algorithm comprises the following steps: training a coding module in advance, dividing all complete point clouds P into different parts pi which are expressed as pi = {p1,..., pn}, and training the coding module by using all the parts pi of the complete point clouds P; training a diffusion model, randomly discarding a% of points for each part pi to form a missing point cloud, representing the missing point cloud as potential representation of pi and calculating the potential representation of pi and the potential representation by using a trained coding module, and respectively recording the potential representation as ri and a label representing ri; and predicting and complementing the point cloud. According to the method, vivid content can be generated for the missing area, the method is suitable for various types of point clouds, the missing degree of the point clouds is not limited, and the practicability and the flexibility of an algorithm are improved to the maximum extent.

Description

Technical Field

[0001] This invention relates to the field of generative AI technology, and more specifically, to an algorithm for 3D point cloud completion based on generative AI. Background Technology

[0002] Recent advances in artificial intelligence have demonstrated the ability of deep neural networks to generate convincingly realistic data by learning complex features and patterns from historical data. This approach, often referred to as generative AI, has garnered particular attention in the fields of video and image synthesis. Within generative AI, three main types of deep neural networks have emerged: variational autoencoders (VAEs), generative adversarial networks (GANs), and diffusion models. Among these, diffusion models achieve an effective balance between stability and diversity. In the context of this research, a diffusion model-based algorithm is used to realistically complete the missing regions in point clouds generated by LiDAR.

[0003] Point clouds are a commonly used 3D data format that preserves the original geometric information of objects in three-dimensional space. They are widely used in fields such as digital storage, reverse engineering, architecture, 3D games, robotics, and virtual reality. Point cloud data is typically obtained using laser scanners, stereo cameras, or RGB-D scanners. During actual acquisition, factors such as occlusion, surface material transparency, light reflection, sensor resolution, and viewing angle limitations can cause loss of geometric and semantic information, resulting in incomplete point cloud data. Therefore, to better apply point cloud data to other fields, shape completion is necessary to obtain the most complete point cloud possible.

[0004] Currently, traditional point cloud completion algorithms typically make strict assumptions about the missing parts. Generally, these algorithms assume that the missing parts are flat and can be reconstructed using 3D planes inferred from surrounding points. Another common approach involves calculating neighboring 3D planes around the gap and using the average 3D plane to fill the missing part. These methods impose strict local constraints on the incomplete parts, often resulting in overly smooth results. Furthermore, there is no guarantee that the reconstructed missing parts will accurately reflect the reality of the actual data. Summary of the Invention

[0005] In response to the problems in related technologies, this invention proposes a 3D point cloud completion algorithm based on generative AI to overcome the aforementioned technical problems in existing related technologies.

[0006] The technical solution of this invention is implemented as follows:

[0007] An algorithm for 3D point cloud completion based on generative AI includes the following steps:

[0008] Step S1: Pre-train the encoding module to divide the entire point cloud P into different parts p i p i Represented as: p i ={p1,…,p n}, and using all parts p of the complete point cloud P. i To train the encoding module;

[0009] Step S2: Train the diffusion model for each part p. i Randomly discarding a% of the points to form a missing point cloud is represented as follows: And use the trained encoding module to compute p i and The potential representations are denoted as r. i and in, Represented as r i Tags;

[0010] Step S3: Perform prediction and completion of the point cloud. For a point cloud q to be completed, find the part q containing the missing region to be completed. i And use the encoding module to calculate q i The incomplete latent representation, denoted as r q Then, the diffusion model is used to generate a complete latent representation. Will Convert to complete point cloud

[0011] The encoding module includes a self-encoder and a decoder that are adapted to each other.

[0012] The autoencoder is a PointNet autoencoder, whereby the point cloud P is processed by the PointNet autoencoder to obtain an n-dimensional global feature vector, which is the latent representation of the point cloud P.

[0013] The decoder is an FCN decoder, which includes a three-layer fully convolutional neural network (FCN).

[0014] The diffusion model includes a diffusion part and a generation part, wherein;

[0015] The diffusion section is used to gradually transform the input features into random noise;

[0016] The generation part is used to gradually restore random noise into input features.

[0017] The beneficial effects of this invention are:

[0018] This invention obtains the latent representation of a point cloud by using an autoencoder and a decoder, performs inference completion using a diffusion model, transforms the completed latent representation of the point cloud into a complete point cloud using an autoencoder and a decoder, and then cascades the autoencoder and decoder with the diffusion model so that the diffusion model can infer the real missing point cloud in the latent space, thereby realizing the completion of 3D point clouds containing missing regions.

[0019] Furthermore, the latent representation obtained by the encoding module of this invention, as input to the diffusion model, makes the entire algorithm more stable and robust. At the same time, unlike previous methods that make strong assumptions about missing regions, this invention learns the features of complete point clouds from historical data, which can generate realistic content for missing regions. It is applicable to various types of point clouds and has no limitation on the degree of missing points, thus maximizing the practicality and flexibility of the algorithm. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a principle block diagram of a 3D point cloud completion algorithm based on generative AI according to an embodiment of the present invention;

[0022] Figure 2 This is a flowchart illustrating the principle of a generative AI-based 3D point cloud completion algorithm according to an embodiment of the present invention. Figure 1 ;

[0023] Figure 3 This is a block diagram of the FCN decoder principle of a generative AI-based 3D point cloud completion algorithm according to an embodiment of the present invention.

[0024] Figure 4 This is a schematic diagram of the Unet architecture principle of a 3D point cloud completion algorithm based on generative AI according to an embodiment of the present invention.

[0025] Figure 5 This is a flowchart illustrating the principle of a generative AI-based 3D point cloud completion algorithm according to an embodiment of the present invention. Figure 2 . Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0027] According to an embodiment of the present invention, an algorithm for 3D point cloud completion based on generative AI is provided.

[0028] like Figures 1-2 As shown, the 3D point cloud completion algorithm based on generative AI according to an embodiment of the present invention includes the following steps:

[0029] Step S1: Pre-train the encoding module to divide the entire point cloud P into different parts p i p i Represented as: p i ={p1,…,p n}, and using all parts p of the complete point cloud P. i To train the encoding module;

[0030] This technical solution includes an encoding module comprising a self-encoder and a decoder that are mutually compatible.

[0031] Step S2: Train the diffusion model for each part p. i Randomly discarding a% of the points to form a missing point cloud is represented as follows: And use the trained encoding module to compute p i and The potential representations are denoted as r. i and in, Represented as r i Tags;

[0032] Step S3: Perform prediction and completion of the point cloud. For a point cloud q to be completed, find the part q containing the missing region to be completed. i And use the encoding module to calculate q i The incomplete latent representation, denoted as r q Then, the diffusion model is used to generate a complete latent representation. Will Convert to complete point cloud

[0033] Using the above scheme, the latent representation of the point cloud is obtained by using an autoencoder and a decoder, and the diffusion model is used for inference and completion. The autoencoder and decoder are then used to transform the completed latent representation of the point cloud into a complete point cloud. Finally, the autoencoder and decoder are cascaded with the diffusion model so that the diffusion model can infer the real missing point cloud in the latent space, thus realizing the completion of 3D point clouds containing missing regions.

[0034] Furthermore, the use of the latent representation obtained by the encoding module as input to the diffusion model makes the entire algorithm more stable and robust. At the same time, unlike previous methods that make strong assumptions about missing regions, this invention learns the features of complete point clouds from historical data, which can generate realistic content for missing regions. It is applicable to various types of point clouds and has no limitation on the degree of missing points, thus maximizing the practicality and flexibility of the algorithm.

[0035] like Figure 1 and Figure 2 As shown, where, Figure 1 This is represented as a block diagram illustrating the principle of an autoencoder and decoder used to obtain the latent representation of a point cloud. Figure 2 This is a block diagram illustrating the principle of using a diffusion model to complete point clouds. An autoencoder and decoder architecture is employed to train and obtain a latent representation of the point cloud. This latent representation, represented as an n x 1 vector, acts as a proxy for the input point cloud.

[0036] Furthermore, the encoding module yields latent representations for each part, which serve as labels for each point cloud. Simultaneously, a diffusion model designed for this purpose aims to learn the latent representation of the complete point cloud from information from the incomplete point cloud. Finally, the output from the diffusion model is used to generate the complete point cloud via the encoding module.

[0037] In addition, such as Figure 3 As shown, the autoencoder is a PointNet autoencoder, where the point cloud P is processed by the PointNet autoencoder to obtain an n-dimensional global feature vector, which is the latent representation of the point cloud P. The decoder is an FCN decoder, which includes a three-layer fully convolutional neural network (FCN).

[0038] In addition, such as Figure 4 As shown, the diffusion model includes a diffusion part and a generation part, wherein the diffusion part is used to gradually transform the input features into random noise; and the generation part is used to gradually restore the random noise back into the input features.

[0039] In addition, such as Figure 5 As shown, the specific application is as follows:

[0040] Pre-trained encoding module: Divides all complete point clouds P into different parts p i p i Represented as: p i ={p1,…,p n The encoding module is trained using all parts P of the complete point cloud P. The trained PointNet autoencoder encodes a point cloud into a high-dimensional global feature, which is the hidden representation of the point cloud. Similarly, the trained FCN decoder reconstructs the point cloud from its hidden representation.

[0041] Train the diffusion model: for each part p i Randomly discard a% of the points to form a missing point cloud, denoted as . p is computed using a trained PointNet autoencoder. i and The potential representations are denoted as r. i and in Considered as r i The tag.

[0042] Use r i and The diffusion model is trained by means of a diffusion part and a generation part. The diffusion part gradually hides the features r of the sparse point cloud. i The noise is transformed into random noise; the generation part gradually restores the random noise into a complete point cloud hidden representation. Both parts of the infrastructure are based on the classic Unet.

[0043] Predict and complete point clouds: For a point cloud q to be completed, find the portion q containing the missing region to be completed. i q is computed using a trained PointNet autoencoder. i The incomplete latent representation, denoted as r q The trained diffusion part is used to generate the complete latent representation. Finally, the trained FCN decoder will... Convert to complete point cloud

[0044] In summary, by utilizing the above-mentioned technical solution of the present invention, a latent representation of a point cloud is obtained by using an autoencoder and a decoder, inference completion is performed using a diffusion model, the completed latent representation of the point cloud is transformed into a complete point cloud using an autoencoder and a decoder, and then the autoencoder and decoder are cascaded with the diffusion model so that the diffusion model can infer the real missing point cloud in the latent space. This enables the completion of 3D point clouds containing missing regions, generates realistic content for missing regions, is applicable to various types of point clouds, and has no limitation on the degree of missing points, thus maximizing the practicality and flexibility of the algorithm.

[0045] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Those skilled in the art, upon considering the disclosure in the specification and embodiments, will readily conceive of other embodiments of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0046] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An algorithm for 3D point cloud completion based on generative AI, characterized in that, Includes the following steps: A pre-trained encoding module divides the entire point cloud P into different parts p. i p i Represented as: p i ={p1,…,p n }, and using all parts p of the complete point cloud P. i To train the encoding module; Train the diffusion model, for each part p i Randomly discarding a% of the points to form a missing point cloud is represented as follows: And use the trained encoding module to compute p i and The potential representations are denoted as r. i and in, Represented as r i Tags; To predict and complete the point cloud, the missing regions q in the point cloud to be completed are determined. i And use the encoding module to calculate q i The incomplete latent representation, denoted as r q Then, the diffusion model is used to generate a complete latent representation. Will Convert to complete point cloud 2. The 3D point cloud completion algorithm based on generative AI according to claim 1, characterized in that, The encoding module includes a self-encoder and a decoder that are adapted to each other.

3. The 3D point cloud completion algorithm based on generative AI according to claim 1, characterized in that, The autoencoder is a PointNet autoencoder, wherein the point cloud P is processed by the PointNet autoencoder to obtain an n-dimensional global feature vector, which is the latent representation of the point cloud P.

4. The 3D point cloud completion algorithm based on generative AI according to claim 3, characterized in that, The decoder is an FCN decoder, which includes a three-layer fully convolutional neural network.

5. The 3D point cloud completion algorithm based on generative AI according to claim 1 or 4, characterized in that, The diffusion model includes: a diffusion part and a generation part, wherein; The diffusion section is used to gradually transform the input features into random noise; The generation part is used to gradually restore random noise into input features.

Images