Three-dimensional point cloud generation method based on PCT and diffusion model

Abstract

The invention discloses a three-dimensional point cloud generation method based on a PCT and a diffusion model, and relates to the technical field of computer vision and point cloud generation, and the method comprises the following steps: 1, preparing original point cloud data, carrying out global standardization processing, carrying out data enhancement, and finally dividing the original point cloud data into a training set and a verification set; step 2, PCT coding is carried out, and the point cloud input # imgabs0 # is coded into a feature representation # imgabs1 # of a hidden space; step 3, a diffusion model is constructed, and the diffusion model comprises a noise prediction model and a decoder model; 4, performing model training by using the training set, wherein the model training comprises the training of extracting the point cloud features through the PCT codes in the step 2 and the training of the noise prediction model and the decoder model in the step 3; and 5, generating a reconstructed three-dimensional point cloud on the new point cloud data by using the model obtained by training in the step 4.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and point cloud generation, and particularly to a three-dimensional point cloud generation method based on PCT and diffusion model. Background Art

[0002] A point cloud is composed of a series of points distributed in three-dimensional space, and each point carries information about its spatial position, and sometimes additional attributes such as color and intensity. Point clouds are directly obtained from real-world 3D scanning technologies and are very suitable for describing and processing real-world scenes and objects. Therefore, how to generate and improve three-dimensional point cloud data more efficiently has become an important and challenging direction in current research. To address these issues, many new methods have emerged to improve the accuracy and coverage of point cloud data generation and reduce the limitations of existing methods. Despite the disadvantages of point clouds, such as the lack of direct connection between adjacent points and the need for preprocessing to perform surface reconstruction or model repair, the development of point cloud technology and the progress of related algorithms have effectively overcome these challenges. By adopting advanced algorithms and technologies, such as deep learning, the spatial relationships between points in a point cloud can be inferred, thus achieving high accuracy in surface reconstruction and object recognition. Therefore, point clouds not only show unparalleled advantages in capturing and processing real-world data, but also are increasingly valued in the field of modern 3D model generation and processing because they support complex scenes and dynamic changes.

[0003] In recent years, diffusion models have achieved great success in the fields of computer vision and image generation, capable of generating high-quality images and having good flexibility. Research on applying diffusion models to 3D technologies has also made great progress and breakthroughs, such as Neural Wavelet domain Diffusion, Point Voxel Diffusion, SDFusion, etc.

[0004] In practical applications, to achieve satisfactory results, higher requirements are imposed on three-dimensional shape generation technologies. Researchers face multiple challenges. For example, the generated models not only need to be realistic enough and have no obvious defects, but also must have sufficient flexibility to generate different levels of detail versions of the same object. Solving these problems depends on the continuous progress of algorithms and more efficient generation and completion technologies.

[0005] Therefore, those skilled in the art are committed to developing a new point cloud generation method that can generate and improve three-dimensional point cloud data more efficiently, improve the accuracy and coverage of point cloud data generation, and solve the above limitations existing in the prior art. Summary of the Invention

[0006] In view of the above-mentioned defects of the prior art, the technical problem to be solved by the present invention is how to improve the quality and authenticity of the generated three-dimensional point cloud, make the generated three-dimensional shape more realistic, and meet the needs of complex application scenarios.

[0007] To achieve the above object, the present invention provides a method for generating a three-dimensional point cloud based on PCT and a diffusion model, the method comprising the following steps:

[0008] Step 1: Prepare the original point cloud data and perform global normalization on it according to the PointFlow approach. Then perform data augmentation and finally divide the processed data into training and validation sets for model training and evaluation.

[0009] Step 2: Perform PCT encoding and input the point cloud into PCT That is, the training set is encoded as the feature representation of the latent space Where B represents the batch size, N represents the number of points in each batch, and d in Represents the coordinate dimension of each point, d out Indicates the dimension of each batch output feature;

[0010] Step 3: Construct a diffusion model, which includes a noise prediction model and a decoder model;

[0011] Step 4: Perform model training using the training set, including the training of point cloud features extracted by PCT coding in step 2 and the training of the noise prediction model and the decoder model in step 3;

[0012] Step 5: Apply the model trained in step 4 to generate a reconstructed three-dimensional point cloud on the new point cloud data.

[0013] Furthermore, in step 2, the PCT encoding first embeds the point cloud input X into a high-dimensional feature space through the embedding layer E, which can be mathematically expressed as:

[0014] E(X)=XW+b

[0015] Among them, W is a weight matrix, Indicates that from d in dimensional space to d f Linear transformation of dimensional space; d f Represents the dimension of the high-dimensional feature space output by the embedding layer E, that is, the dimension of the initial feature space, which is usually greater than d in ; b is a bias vector, Used to add an offset based on XW;

[0016] After embedding, the PCT encoder uses four stacked levels of attention modules to map the high-dimensional feature space dimension to the final output feature space, that is, to obtain the feature representation Y of the latent space;

[0017] Let X′ = E(X), and the output of the i-th attention module A i is defined as:

[0018]

[0019] where, respectively represent the features of the query, key, and value in the i-th attention module, is the corresponding weight matrix, d k is the scaling factor;

[0020] The output of the last attention module is processed through a linear layer as follows:

[0021] H = A4(A3(A2(A1(E(C)))))

[0022] Y = HW′ + b′

[0023] where, W′ is a weight matrix, represents a linear transformation from the d f -dimensional space to the d out -dimensional space, which is used to map the high-dimensional features obtained after passing through the embedding layer E and four levels of attention modules to the final output feature space b′ is a bias vector, which is used to add an offset on the basis of HW′.

[0024] Furthermore, in step 2, d in = 3, indicating that the coordinate dimension of each point is 3; d out = 1024, indicating that the dimension of the output features of each batch is 1024.

[0025] Furthermore, in step 3, the noise prediction model is based on the U-Net network structure.

[0026] Furthermore, the noise prediction model includes a forward noise addition process and a reverse diffusion process;

[0027] In the forward noise addition process, noise is gradually applied to the feature representation Y of the latent space to perform forward diffusion, gradually transforming the features of the latent space into pure noise;

[0028] The formula for gradually adding noise is:

[0029]

[0030] where ∈ t is the noise sampled from the standard Gaussian distribution ; β t is the intensity that controls the noise, and α t = 1 - β t , which is the parameter renormalization of 1 - β t and essentially also represents the intensity of the noise; q(Y t |Y t-1 ) is the conditional probability calculation formula for calculating Y t-1 from Y t , and specifically:

[0031]

[0032] where Σ is the Gaussian noise,

[0033] Continuing to expand recursively until it is generalized to the initial feature representation Y, i.e., Y0, we can obtain:

[0034]

[0035] where ∈ is the comprehensive noise that combines all-step noises and still follows the standard Gaussian distribution, representing the intensity of the noise.

[0036] Furthermore, in the reverse diffusion process, the noise prediction model is trained to predict the noise at each step, including:

[0037] Inputting the noisy Y t , the noise prediction model outputs the estimated noise ∈ ∧ through downsampling, upsampling, and the attention mechanism;

[0038] Using the predicted noise, reversely generate the feature Y t-1 in the latent space, and through step-by-step reverse generation, finally recover the feature Y0' in the latent space from pure noise; in the reverse diffusion process, the regenerated latent space point cloud at the t - 1 step is represented as where Σ θ is the estimation of the noise by the noise prediction model,

[0039] Furthermore, the decoder model decodes the feature Y ′ 0 in the latent space recovered by the noise prediction model into the final three-dimensional point cloud data X re , completing the mapping from the latent space to the point cloud space;

[0040] The decoder model utilizes a series of fully connected layers to amplify features, gradually enhancing the dimension of latent features, which is mathematically described as:

[0041] x = F.relu(BN1(FC1(x)))

[0042] x = F.relu(BN2(FC2(x)))

[0043] x = F.relu(BN3(FC3(x)))

[0044] where each stage aims to strategically amplify the feature dimension, while integrating non - linearity and stabilizing the training process through normalization;

[0045] After amplifying the features, a series of de - convolution layers perform inverse convolution expansion, finely dispersing the concentrated features on a three - dimensional grid, aiming to spatially expand the data and readjust it to the original point cloud form:

[0046] x = F.relu(BN4(Deconv1(x)))

[0047] x = F.relu(BN5(Deconv2(x)))

[0048] x = Deconv3(x).

[0049] Furthermore, in the model training of step 4, multiple loss functions are preset to guide the optimization of the model, specifically including mean squared error, Chamfer distance, and EMD:

[0050]

[0051] When the loss function reaches a stable state or the performance of the model on the validation set is not lower than a preset threshold, the training is stopped, and the trained model parameters are saved.

[0052] Furthermore, in the model training of step 4, a regularization loss term, L2 regularization, is introduced to penalize high - weight parameters in the model and prevent overfitting.

[0053] Furthermore, step 5 includes the following sub - steps:

[0054] Step 5.1: Sample noise from a standard Gaussian distribution;

[0055] Step 5.2: Input the sampled noise into the denoising network, and generate denoised latent features through the denoising network and the reverse diffusion process. The reverse diffusion process is represented by the following formula:

[0056]

[0057] Among them, μ θ (y t , t), ∑ θ (x t , t) are the estimates of the noise mean and variance by the denoising network respectively, and

[0058] Step 5.3: Input the generated latent features into the decoder model. Through a series of linear layers, batch normalization layers, and transposed convolution layers, gradually expand from the low-dimensional feature space to the high-dimensional point cloud data, and finally generate the reconstructed 3D point cloud X re ;

[0059] The denoising network is represented as ∈ θ (y t , t) or ∈ θ (y t , t, c) according to whether external conditions are introduced.

[0060] A 3D point cloud generation method based on PCT and diffusion model provided by the present invention has at least the following technical effects:

[0061] 1. The technical solution provided by the present invention can more effectively capture the features of 3D point clouds and accurately encode them into the standard latent space by introducing a point cloud transformer as the encoder. This strategy not only unleashes the potential of the latent space but also provides an ideal data representation for subsequent diffusion model training. Compared with existing methods, it significantly improves the quality and efficiency of feature learning, enhances the expressiveness of the model, and simplifies the traditional complex 3D data training process.

[0062] 2. During the decoding process, the technical solution provided by the present invention implements an advanced dual training mechanism, which not only enhances the decoder's ability to restore the shape of 3D point clouds but also improves the accuracy of UNet in predicting noise features. Through the Diffusion Probability Model (DDPM) algorithm, it is possible to denoise from pure Gaussian noise to generate clear and detailed 3D point clouds, further improving the generation quality and detail performance of the model.

[0063] The following will further illustrate the concept, specific structure, and technical effects of the present invention with reference to the accompanying drawings to fully understand the purpose, features, and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 is a schematic diagram of a 3D point cloud generation method according to a preferred embodiment of the present invention;

[0065] Figure 2 is Figure 1Comparison chart of the generation effects of the three categories of aircraft, chairs, and cars in the illustrated embodiment with other models;

[0066] Figure 3 is Figure 1 Multi-category generation effect diagram of the illustrated embodiment. Detailed implementation manners

[0067] The following introduces multiple preferred embodiments of the present invention with reference to the accompanying drawings of the specification to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, and the protection scope of the present invention is not limited to the embodiments mentioned in the text.

[0068] The embodiment of the present invention aims to propose a three-dimensional point cloud generation method (PCT–Diffusion, PTD) based on Point Cloud Transformer (PCT) and diffusion model, which uses a feature-guided diffusion process to generate high-precision three-dimensional point clouds. By making a more accurate prediction of the noise at each step in the inverse diffusion process, the quality and authenticity of the generated three-dimensional point clouds are significantly improved. This method enhances the model's ability to restore details, generates more realistic three-dimensional shapes, and meets the requirements of complex application scenarios. The method provided by the embodiment of the present invention mainly includes three steps, namely feature extraction of point clouds, training of the diffusion model in the latent space, and dimensionality reduction.

[0069] In the framework proposed by the embodiment of the present invention, Point Cloud Transformer (PCT) is used as a key encoder component to capture and extract the intrinsic features of three-dimensional point clouds. The main goal of PCT is to transform the input point cloud into a new feature space. This transformation aims to reveal the semantic relationships between points and lay the foundation for subsequent point cloud processing tasks. The embodiment of the present invention innovatively applies the Point Cloud Transformer (PCT) architecture to three-dimensional generation tasks, successfully expanding its application to the specialized field of feature extraction, providing new tools and ways to deeply understand the complex three-dimensional spatial structure relationships. By modifying the module to conduct a deeper and more insightful analysis of the data space structure, the flexibility and wide application potential of the PCT architecture are also demonstrated. In the design of the output layer, the output dimension of the last linear layer is increased to 1024 to more effectively capture the spatial hierarchical structure in the data. This enhancement can provide a more robust feature representation, which is crucial for subsequent tasks such as denoising or synthesis.

[0070] Embodiments of the present invention demonstrate how the PCT architecture can be extended beyond its traditional applications, such as 3D classification and 3D segmentation, opening up new paths for the application of the model in other fields of 3D data processing. The operation process of PCT starts with initially embedding the input coordinates into a new feature domain. This embedding serves as a prelude to prepare for the deep processing of the data. Subsequently, the embedded features are propagated through a series of four stacked attention modules. Through this sequential advancement of the attention modules, the system can cultivate a semantically rich and discriminative representation for each point in the dataset. The final result of the feature refinement stage is a linear layer responsible for synthesizing the output feature vector, adopting a design concept consistent with the original Transformer architecture and avoiding the use of positional embeddings. This deviation is based on the inherent characteristics of point cloud data, as the coordinates of points inherently contain positional information.

[0071] Specifically, the three-dimensional point cloud generation model (PTD) based on PCT and the diffusion model mainly includes the following steps:

[0072] Data processing: Collect the largest ShapeNetCore dataset in the field of point cloud generation, classify the data for training, verify the effectiveness of the model proposed by the present invention, and evaluate the generation performance of the model;

[0073] Model construction: Based on the PCT module and making structural adjustments, extract the features of the point cloud, construct a noise prediction model based on the U-Net network structure, and a decoder model for recovering the three-dimensional spatial data of the point cloud from the feature space;

[0074] Model training: Include the training of the point cloud feature extraction module, the noise prediction module in the feature space, and the decoder module;

[0075] Point cloud generation: Use the trained diffusion model and decoder model to generate a three-dimensional point cloud model similar to the dataset distribution.

[0076] The term "latent space" refers to the representation space generated by encoding the input data, while the "reconstruction error" refers to the difference between the original data and the reconstructed data during the data reconstruction process, and this error is used to optimize the performance of the model.

[0077] Embodiment 1

[0078] As Figure 1 shown, an embodiment of the present invention provides a three-dimensional point cloud generation method based on PCT and the diffusion model, including the following steps:

[0079] Step 1: Prepare the original point cloud data, perform global standardization on the original point cloud data following the practice of PointFlow, then perform data augmentation, and finally divide the processed data into a training set and a validation set for model training and evaluation;

[0080] Step 2: Perform PCT encoding, input the point cloud with PCT i.e., the training set, and encode it into the feature representation of the latent space where B represents the batch size, N represents the number of points in each batch, and d in represents the coordinate dimension of each point, and d out represents the dimension of the output features of each batch;

[0081] Step 3: Construct a diffusion model, which includes a noise prediction model and a decoder model;

[0082] Step 4: Use the training set for model training, including the training of extracting point cloud features through PCT encoding in Step 2 and the training of the noise prediction model and the decoder model in Step 3;

[0083] Step 5: Apply the model trained in Step 4 to generate a reconstructed three-dimensional point cloud on new point cloud data.

[0084] Specifically, in Step 1, according to previous research, the ShapeNetCore.v2.PC15k dataset is used for 3D shape generation experiments. The main categories include chairs, airplanes, and cars. For each shape, 5000 points are sampled from the dataset for training and evaluation. The dataset division and preprocessing steps follow the practice of PointFlow to ensure that the data of the entire dataset are globally standardized. Model evaluation follows the methodology established in previous research. There are several metrics for evaluating point cloud generation models, and each metric has its advantages and disadvantages. Consistent with recent research, the embodiments of the present invention adopt the 1-NNA metric, and at the same time use the Chamfer Distance (CD) and the Earth Mover Distance (EMD) as the main metrics. The above metrics evaluate the similarity in distribution between the generated shape and the validation dataset, and can effectively measure the quality and diversity of the generated output.

[0085] Specifically, in the provided settings, N is 5000. d in = 3, indicating that the coordinate dimension of each point is 3, that is, the three-dimensional coordinate input (x, y, z) of each point; d out = 1024, indicating that the dimension of the output features of each batch is 1024. This method designably processes point cloud data by utilizing the inherent spatial information in the point coordinates.

[0086] Embodiment 2

[0087] Based on Example 1, in step 2, the encoding process in PCT starts with embedding the input coordinates into a higher-dimensional feature space. Specifically, the PCT encoding first embeds the point cloud input X into a high-dimensional feature space through the embedding layer E, which is mathematically expressed as:

[0088] E(X) = XW + b

[0089] where W is a weight matrix, representing a linear transformation from a d in -dimensional space to a d f -dimensional space; d f represents the dimension of the high-dimensional feature space output by the embedding layer E, that is, the dimension of the initial feature space, which is usually greater than d in ; b is a bias vector, used to add an offset on the basis of XW;

[0090] After embedding, the core of the PCT encoder lies in using stacked attention modules. Each attention module operates according to the principles defined by the original Transformer architecture, removing the positional encoding because the spatial relationship is essentially controlled by the coordinates of the points. The PCT encoder uses four stacked levels of attention modules to map the dimension of the high-dimensional feature space to the final output feature space, that is, to obtain the feature representation Y of the latent space;

[0091] Let X' = E(X), and the output of the i-th attention module A i is defined as:

[0092]

[0093] where, respectively represent the features of the query, key, and value in the i-th attention module, is the corresponding weight matrix, d k is the scaling factor; d k is usually the dimension of the key.

[0094] The output of the last attention module is processed through a linear layer as follows:

[0095] H = A4(A3(A2(A1(E(X)))))

[0096] Y = HW' + b'

[0097] where W' is a weight matrix, representing a linear transformation from a d f -dimensional space to a d out -dimensional space, used to transform the high-dimensional features obtained after passing through the embedding layer E and four levels of attention modules Mapped to the final output feature space b′ is a bias vector, used to add an offset based on HW′.

[0098] Therefore, the output Y, as the encoded feature space, captures the key discriminative features and semantic associations between points to support further point cloud processing tasks. By integrating concepts from the Transformer architecture and adapting the positional embeddings according to the uniqueness of the point cloud data structure, the PCT model effectively addresses the key challenges in point cloud processing. This approach not only ensures robustness but also improves the model's generalization ability in different tasks within the point cloud domain.

[0099] Embodiment 3

[0100] Based on Embodiment 2, in Step 3, the noise prediction model is based on the U-Net network structure. The noise prediction model includes a forward noise addition process and a reverse diffusion process;

[0101] In the forward noise addition process, noise is gradually applied to the feature representation Y in the latent space for forward diffusion, gradually transforming the features in the latent space into pure noise;

[0102] The formula for gradually adding noise is:

[0103]

[0104] where, ∈ t is the noise sampled from the standard Gaussian distribution ; β t is the intensity controlling the noise, α t = 1 - β t , is the parameter renormalization of 1 - β t , and essentially also represents the intensity of the noise; q(Y t |Y t-1 ) is the conditional probability calculation formula for calculating Y t-1 based on Y t , specifically:

[0105]

[0106] where, Σ is Gaussian noise,

[0107] Continuing to recursively expand until it is generalized to the initial feature representation Y, i.e., Y0, we can obtain:

[0108]

[0109] where, ∈ is the combined noise merging all-step noises and still follows the standard Gaussian distribution, represents the intensity of the noise.

[0110] During the reverse diffusion process, the noise prediction model is trained to predict the noise at each step, including:

[0111] input Y with noise t , and the noise prediction model outputs an estimate of the noise ∈ through downsampling, upsampling, and attention mechanisms ∧ ;

[0112] Using the predicted noise, reverse generate the features Y in the latent space t-1 , and through step-by-step reverse generation, finally recover the features Y0' in the latent space from pure noise; during the reverse diffusion process, the regenerated latent space point cloud at step t-1 is represented as where Σ θ is the estimate of the noise by the noise prediction model,

[0113] Embodiment 4

[0114] Based on Embodiment 3, the decoder model decodes the features Y0' in the latent space restored by the noise prediction model into the final three-dimensional point cloud data X re , completing the mapping from the latent space to the point cloud space;

[0115] The decoder model uses a series of fully connected layers to perform feature amplification to gradually enhance the dimension of the latent features, which is mathematically described as:

[0116] x = F.relu(BN1(FC1(x)))

[0117] x = F.relu(BN2(FC2(x)))

[0118] x = F.relu(BN3(FC3(x)))

[0119] where each stage aims to strategically amplify the feature dimension while integrating non-linearity and stabilizing the training process through normalization;

[0120] After amplifying the features, a series of deconvolution layers perform inverse convolution expansion to finely disperse the concentrated features on a three-dimensional grid, aiming to spatially expand the data and readjust it to the original point cloud form:

[0121] x = F.relu(BN4(Deconv1(x)))

[0122] x = F.relu(BN5(Deconv2(x)))

[0123] x = Deconv3(x)

[0124] In the process of transforming the original point cloud through multiple stages of denoising and feature encoding, the ultimate challenge lies in effectively reconstructing the original spatial geometry from the compressed feature format. The PointCloudGenerator module, acting as a decoder, performs the encoding process in reverse to restore the detailed 3D structure. This module is cleverly designed to handle the complexity of reshaping the feature space of dimension (batchsize, 1024) into detailed point cloud data (batchsize, 5000, 3). The PointCloudGenerator module combines linear layers, batch normalization layers, and transposed convolution layers to achieve the goal with high fidelity.

[0125] Example 5

[0126] In the model training stage, the goal is to optimize the parameters of PCT, Unet+Attention, and the PointCloudGenerator module so as to efficiently and accurately process point cloud data. First, following the practice of previous research, 3D shape generation experiments are conducted using the ShapeNet dataset, with the main categories including chairs, airplanes, and cars. For each shape, 5000 points are sampled from the dataset for training and evaluation. The dataset splitting and preprocessing steps follow the method of PointFlow to ensure the global normalization of the entire dataset.

[0127] Based on Example 3 or 4, during the training process, multiple loss functions are defined to guide the optimization of the model, specifically including mean squared error (MSE), Chamfer distance, and EMD (Earth Mover's Distance):

[0128]

[0129] The reconstruction loss is used to measure the difference between the point cloud X generated from the latent representation Y re and the original point cloud X. When the loss function reaches a steady state or the performance of the model on the validation set is not lower than a pre-set threshold, the training is stopped, and the trained model parameters are saved.

[0130] Specifically, to prevent overfitting, a regularization loss term, L2 regularization, is introduced to penalize high-weight parameters in the model. In some cases, a loss term for the attention mechanism is also introduced to ensure that the attention module can effectively capture important features in the point cloud. In terms of optimizer selection, the Adam optimizer is used, which is widely adopted due to its excellent performance and adaptive learning rate. During training, the learning rate is dynamically adjusted according to the model's performance to achieve the best training effect. Specifically, the input point cloud X is propagated forward through the PCT encoder to generate a compressed feature representation Y. Then, noise is gradually added to the feature representation Y to generate intermediate representations Y at multiple noise levels t . Next, the noisy representation Y t is propagated backward through the PointCloudGenerator decoder to generate the reconstructed point cloud X re . The difference between the input point cloud X and the reconstructed point cloud X re is calculated through the defined loss function, and these losses are backpropagated to update the parameters of the PCT encoder and the PointCloudGenerator decoder to minimize the loss function

[0131] During training, the performance of the model is regularly evaluated, and a validation set or a test set is used to check the reconstruction accuracy and the effectiveness of the feature representation. According to the evaluation results, the hyperparameters of the model, such as the learning rate, batch size, and the number of layers in the attention module, are adjusted. In addition, visualization tools, such as TensorBoard, are used to monitor the loss changes and model performance during training to ensure the smooth progress of the training process

[0132] When the loss function reaches a stable state or the performance of the model on the validation set no longer improves significantly, the training is stopped, and the trained model parameters are saved for subsequent testing and deployment

[0133] Example 6

[0134] Based on Example 3, 4, or 5, Step 5 includes the following sub-steps

[0135] Step 5.1: Sample noise from a standard Gaussian distribution

[0136] Step 5.2: Input the sampled noise into the denoising network, and generate denoised latent features through the denoising network and the reverse diffusion process. The reverse diffusion process is represented by the following formula

[0137]

[0138] where μ θ (y t ,t), ∑ θ (x t, t) are the estimates of the noise mean and variance by the denoising network, and The denoising network is represented as ∈ according to whether external conditions are introduced θ (y t , t) or ∈ θ (y t , t, c);

[0139] Step 5.3: Input the generated latent features into the decoder model. Through a series of linear layers, batch normalization layers, and transposed convolution layers, gradually expand from the low-dimensional feature space to the high-dimensional point cloud data, and finally generate the reconstructed three-dimensional point cloud X re .

[0140] The above process ensures that the generated point cloud is not only similar to the original point cloud in geometric structure, but also has rich feature representations semantically. Through these steps, the embodiments of the present invention can effectively recover the original point cloud data from the compressed feature representations, providing high-quality point cloud generation capabilities for various 3D data processing tasks.

[0141] To verify the superiority of the point cloud quality generated by the method provided by the embodiments of the present invention, the method provided by the embodiments of the present invention is compared with the latest research PointFlow, PVD, SetVAE, and LION in the aircraft, chair, and car categories of the ShapeNetCore dataset, as Figure 2 and Figure 3 shown. The results were evaluated using the CD-1NNA and EMD-1NNA metrics, and both metrics are widely used in the evaluation of point cloud generation. The performance on the ShapeNetCore dataset is shown in Table 1. Compared with other models, the method provided by the embodiments of the present invention shows more advantages and better generation quality in terms of the score performance of the earth mover's distance (EMD) than in the chamfer distance (CD).

[0142] Table 1 Experimental comparison between the model proposed in the present invention and current advanced models

[0143]

[0144]

[0145] In the diffusion model of the method provided by the embodiments of the present invention, the predictive noise neural network model adopts an attention-enhanced UNet structure, which enhances the denoising performance by focusing on the regions in the features that are significantly affected by noise. UNet is divided into an encoder and a decoder. The attention module is mainly integrated in the decoder to refine the attention to high-importance features, using the feedback from the encoder and the information passed from the early layers.

[0146] Use the diffusion model proposed in the embodiments of the present invention to predict the noise of the point cloud data at step t. The input feature X with t-step noise applied at step t t and step t are used as the input to the diffusion model. In this model, X t will undergo downsampling, upsampling, and attention mechanisms and finally output. In the encoder part, the input data is first processed by an initial double convolutional block, which contains two consecutive convolutional layers and ReLU activation functions for extracting primary features. As the data flows downward, more double convolutional blocks are used to gradually deepen the network while increasing the number of channels of the features, thereby capturing more abstract information. In the decoder part, the model uses transposed convolution operations for upsampling to gradually restore the resolution of the features. Importantly, the decoder also introduces an attention mechanism. Through the AttentionBlock module, the model can learn the weights at different positions during the feature fusion process, thereby enhancing the expression of key features. Finally, the output of the decoder is concatenated with the corresponding features in the encoder to further enrich the representation of information. Finally, a noise prediction value of the same dimension as the input feature point cloud data is output.

[0147] The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and variations based on the concept of the present invention without creative labor. Therefore, all technical solutions that can be obtained by those skilled in the art in the technical field based on the concept of the present invention through logical analysis, reasoning, or limited experiments on the basis of the prior art should be within the protection scope determined by the claims.

Claims

1. A three-dimensional point cloud generation method based on PCT and diffusion models, characterized in that, The method includes the following steps: Step 1: Prepare the original point cloud data, perform global normalization processing on the original point cloud data following the practice of PointFlow, then perform data augmentation processing, and finally divide the processed data into a training set and a validation set for model training and evaluation; Step 2: Perform PCT encoding and input the point cloud with PCT That is, the training set is encoded into the feature representation of the latent space where B represents the batch size, N represents the number of points in each batch, and d in represents the coordinate dimension of each point, and d out represents the dimension of the output features of each batch; Step 3: Construct a diffusion model, which includes a noise prediction model and a decoder model; Step 4: Use the training set for model training, including the training of extracting point cloud features through the PCT encoding in Step 2 and the training of the noise prediction model and the decoder model in Step 3; Step 5: Apply the model trained in Step 4 to generate a reconstructed three-dimensional point cloud on new point cloud data.

2. The three-dimensional point cloud generation method based on PCT and diffusion model according to claim 1, wherein In Step 2, the PCT encoding first embeds the point cloud input X into a high-dimensional feature space through the embedding layer E, which is mathematically expressed as: E(X) = XW + b Among them, W is a weight matrix, representing a linear transformation from a d in -dimensional space to a d f -dimensional space; d f represents the dimension of the high-dimensional feature space output by the embedding layer E, that is, the dimension of the initial feature space, which is usually greater than d in ; b is a bias vector, used to add an offset on the basis of XW; After embedding, the PCT encoder uses four stacked hierarchical attention modules to map the dimension of the high-dimensional feature space to the final output feature space, that is, to obtain the feature representation Y of the latent space; Let X′ = E(X), and the output of the i-th attention module A i is defined as: Among them, respectively represent the features of the query, key, and value in the $i$-th attention module, is the corresponding weight matrix, and $d$ k is the scaling factor; The output of the last attention module is processed through a linear layer as follows: H = A4(A3(A2(A1(E(X))))) Y = HW′ + b′ Among them, W′ is a weight matrix, representing a linear transformation from a d f -dimensional space to a d out -dimensional space, which is used to map the high-dimensional features obtained after passing through the embedding layer E and four levels of attention modules to the final output feature space b′ is a bias vector, which is used to add an offset on the basis of HW′.

3. The three-dimensional point cloud generation method based on the PCT and diffusion model according to claim 1, wherein, In step 2, d in = 3, indicating that the coordinate dimension of each point is 3; d out = 1024, indicating that the dimension of the output features of each batch is 1024.

4. The three-dimensional point cloud generation method based on PCT and diffusion model according to claim 2, wherein, In Step 3, the noise prediction model is based on the U-Net network structure.

5. The three-dimensional point cloud generation method based on the PCT and diffusion model according to claim 4, wherein The noise prediction model includes a forward noise addition process and a reverse diffusion process; In the forward noise addition process, noise is gradually applied to the feature representation Y of the latent space for forward diffusion, gradually transforming the features in the latent space into pure noise; The formula for gradually adding noise is: where, ∈ t is the noise sampled from the standard Gaussian distribution ; β t is the intensity controlling the noise, α t = 1 - β t , which is the parameter renormalization of 1 - β t and essentially also represents the intensity of the noise; q(Y t |Y t-1 ) is the conditional probability calculation formula for calculating Y t-1 based on Y t , specifically: where Σ is Gaussian noise, Continuing to expand recursively until it is generalized to the initial feature representation Y, that is, Y0, we can obtain: where ∈ is the combined noise that merges all step noises and still follows the standard Gaussian distribution, representing the intensity of the noise.

6. The three-dimensional point cloud generation method based on the PCT and diffusion model according to claim 5, wherein, In the reverse diffusion process, the noise prediction model is trained to predict the noise at each step, including: Input noisy Y t , the noise prediction model outputs an estimate of the noise ∈ ∧ ; Generate the feature Y of the latent space in reverse using the predicted noise t-1 , and through step-by-step reverse generation, finally recover the feature Y′0 of the latent space from pure noise; in the reverse diffusion process, the regenerated latent space point cloud at step t - 1 is represented as where Σ θ is the noise pre- The estimation of the measurement model for noise 7. The three-dimensional point cloud generation method based on PCT and diffusion model according to claim 6, wherein, The decoder model decodes the feature Y0' of the latent space restored by the noise prediction model into the final three-dimensional point cloud data X re , completing the mapping from the latent space to the point cloud space; The decoder model uses a series of fully connected layers to perform feature amplification to gradually enhance the dimension of the latent features, which is mathematically described as: x = F.relu(BN1(FC1(x))) x = F.relu(BN2(FC2(x))) x = F.relu(BN3(FC3(x))) where each stage aims to strategically amplify the feature dimension while integrating non-linearity and stabilizing the training process through normalization; After amplifying the features, a series of deconvolution layers perform inverse convolution expansion to finely disperse the concentrated features on a three-dimensional grid, aiming to spatially expand the data and readjust it to the form of the original point cloud: x = F.relu(BN1(Deconv1(x))) x = F.relu(BN5(Deconv2(x)0) x = Deconv3(x).

8. The three-dimensional point cloud generation method based on the PCT and diffusion model according to claim 7, wherein In the model training of Step 4, multiple loss functions are preset to guide the optimization of the model, specifically including mean squared error, Chamfer distance, and EMD: Stop training when the loss function reaches a steady state or the performance of the model on the validation set is not lower than a preset threshold, and save the trained model parameters.

9. The three-dimensional point cloud generation method based on PCT and diffusion model according to claim 8, wherein, In the model training of step 4, introduce a regularization loss term L2 regularization to penalize high-weight parameters in the model and prevent overfitting.

10. The three-dimensional point cloud generation method based on the PCT and diffusion model according to claim 8, characterized in that, Step 5 includes the following sub-steps: Step 5.1: Sample noise from a standard Gaussian distribution; Step 5.2: Input the sampled noise into the denoising network, and generate denoised latent features through the denoising network and the reverse diffusion process. The reverse diffusion process is represented by the following formula: where μ θ (y t , t), ∑ θ (x t , t) are the estimates of the noise mean and variance by the denoising network respectively, and Step 5.3: Input the generated latent features into the decoder model, and gradually expand from the low-dimensional feature space to the high-dimensional point cloud data through a series of linear layers, batch normalization layers, and transposed convolution layers, and finally generate the reconstructed three-dimensional point cloud X re ; The denoising network is expressed as ∈ according to whether external conditions are introduced θ (y t , t) or ∈ θ (y t , t, c).

Images