Flow diffusion model point cloud generation method based on neighborhood feature enhancement

Abstract

The invention discloses a flow diffusion point cloud generation method based on neighborhood feature enhancement. The method comprises the following steps: preprocessing an original point cloud; a multi-scale neighborhood feature aggregation module is constructed, the module obtains neighborhood coordinates of each point through nearest neighbor sampling of different scales, and the coordinates are aggregated into rough global features; constructing a neighborhood attention feature enhancement module, and enhancing rough features in a feature space by using geometric information in a neighborhood coordinate space; reducing redundant features in the enhanced features by using a point voxel convolutional neural network, and outputting potential vectors of the point cloud; a feature-to-stream data conversion module is designed, potential vectors are converted into prior stream features with consistent data, and the prior stream features are seamlessly integrated into a noise reduction diffusion model so as to be accurately generated from noise point clouds. According to the invention, through combination of neighborhood feature enhancement and flow diffusion, high-quality 3D point clouds are effectively generated.

Description

Technical Field

[0001] The present invention relates to the fields of three-dimensional generation and computer vision, and in particular to a flow diffusion point cloud generation method based on neighborhood feature enhancement. Background Art

[0002] Point cloud data enhances multimedia applications by providing detailed visual and spatial information, improving user immersion and interactivity. Its use in virtual reality, augmented reality, and mixed reality can create realistic 3D environments. However, acquiring point cloud data is time-consuming and sensor-dependent. Therefore, it is necessary to apply artificial intelligence methods to advance point cloud generation technology in order to provide high-fidelity data support for the multimedia field.

[0003] In recent years, deep learning-based methods have become the mainstream of 3D point cloud generation. This process usually includes several key steps. The first step is data preprocessing, including denoising, normalization, and data augmentation to ensure the consistency and diversity of model input. Next is feature extraction, where deep neural networks, such as convolutional neural networks, extract high-dimensional features from point clouds and capture the complex relationships and geometric structures between points. The third step involves the design and training of generative models, such as variational autoencoders, generative adversarial networks, normalized flow models, or autoregressive models, each of which uses different generative methods to generate new point clouds.

[0004] The closest existing technology and its review:

[0005] Among the technologies closest to the existing ones, some scholars have proposed the DualGAN algorithm (Wen C, Yu B, Tao D. Learning progressive point embeddings for 3D point cloud generation [C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2021: 10266-10275.), which uses two adversarial generative network generators to upsample the advanced point cloud and then refine the generation. The PointFlow algorithm (Yang G, Huang X, Hao Z, et al. Pointflow: 3D point cloud generation with continuous normalizing flows [C] / / Proceedings of the IEEE / CVF international conference on computer vision. 2019: 4541-4550.), uses a 3D point cloud model with continuous normalizing flows and variational reasoning, and its advantage lies in its stability compared with adversarial generative networks. SoftFlow algorithm (Kim H, Lee H, Kang WH, et al. Softflow: Probabilistic framework for normalizing flow on manifolds [J]. Advances in Neural Information Processing Systems, 2020, 33: 16388-16397), which trains the normalized flow on the manifold by conditionally distributing the perturbed input data, thereby avoiding the dimension mismatch problem. SetVae algorithm (Kim J, Yoo J, Lee J, et al. Setvae: Learning hierarchical composition for generative modeling of set-structured data [C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2021: 15059-15068.), generates diverse and complex point clouds by using probabilistic graphical models and variational reasoning.The RPG algorithm (KolW J, Chiu CY, Kuo YL, et al. RPG: Learning recursive point cloud generation [C] / / 2022 IEEE / RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2022: 544-551.) generates 3D point clouds from coarse to fine by recursively expanding points, while automatically discovering the semantic parts of the target model and realizing unsupervised part segmentation during the generation process. The PVD algorithm (Zhou L, DuY, Wu J. 3D shape generation and completion through point-voxel diffusion [C] / / Proceedings of the IEEE / CVF international conference on computer vision. 2021: 5826-5835.) generates point cloud data from Gaussian noise under a series of denoising operations by combining the denoising diffusion model with the mixed point voxel representation. The TGN algorithm (Xu R, Hui L, Han Y, et al. Transformer-based Point Cloud Generation Network [C] / / Proceedings of the 31st ACM International Conference on Multimedia. 2023: 4169-4177.) uses a Transformer-based interpolation module and a refinement module to combine local and global information, gradually upsample point cloud features, and finally generate realistic three-dimensional point clouds. Although these methods have achieved good results, they usually use hierarchical convolutional neural networks or shared full connections for feature extraction, which may ignore some neighborhood features when aggregating spatial features, and introduce uncertainty and errors in step-by-step denoising, especially for complex geometric structures, which may affect the complete recovery of details and reduce the quality of the generated point cloud. Summary of the invention

[0006] The purpose of the present invention is to provide a flow diffusion point cloud generation method based on neighborhood feature enhancement. Based on the problem that the prior art ignores some neighborhood features of point clouds, introduces uncertainty and errors in the generation process, and thus leads to the reduction of some generation quality, a multi-scale neighborhood feature aggregation module is constructed, which uses K nearest neighbors sampling at different scales to obtain the neighborhood coordinates of each point, thereby aggregating into coarse global features. Secondly, a feature enhancement module based on neighborhood attention is constructed, which uses geometric information in the coordinate space to enhance the coarse features in the feature space through the attention mechanism neighborhood. Then, a point voxel convolutional neural network is used to reduce redundant features in the enhanced features and output the latent vector of the point cloud. Finally, a feature to flow data conversion module is designed to convert the latent vector into data-consistent prior flow features, which are seamlessly integrated into the denoising diffusion model for accurate generation from noisy point clouds. This prior flow method improves the consistency and coherence of the point cloud density distribution.

[0007] The steps include:

[0008] Step 1: Preprocess the original point cloud to be encoded, including normalizing the point cloud data to [-1, 1] and dividing it into training set, test set and validation set;

[0009] Step 2: construct a multi-scale neighborhood feature aggregation module, wherein the multi-scale neighborhood feature aggregation module includes a farthest point sampling component, multiple multi-scale K nearest neighbor components and multiple coarse feature extraction components;

[0010] Step 3: construct a neighborhood attention feature enhancement module, which enhances the global coarse features obtained in step 2 through a self-attention mechanism and farthest point sampling;

[0011] Step 4: Use the point voxel convolutional neural network to reduce the redundant features in the enhanced features output in step 3 and output the latent vector of the point cloud;

[0012] Step 5: Design a feature-to-stream data conversion module, which maps the feature-to-stream data conversion module to stream data through affine transformation; the latent vector output in step 4 is used to make the sampling probability of the point cloud continuous through the feature-to-stream data conversion module;

[0013] Step 6: Input the flow data in step 5 into the diffusion model to generate point cloud;

[0014] Step 7: Compare the generated point cloud obtained in step 6 with the input point cloud and calculate the loss;

[0015] Step 8: According to the total loss obtained in step 7, use the Adam optimizer to optimize the point cloud generation method end-to-end;

[0016] Step 9: Loop through steps 2 to 8, train for a specified number of rounds, until the point cloud generation network of the flow diffusion model with enhanced neighborhood features converges, and save the optimal training model file;

[0017] Step 10: Select a single object or multiple object categories for training based on actual usage requirements and output the object type that needs to be generated.

[0018] By adopting the above technical solution, the beneficial effects of the present invention are:

[0019] 1. The present invention proposes a scale neighborhood feature aggregation module and a neighborhood attention feature enhancement module to extract and enhance point cloud features, providing reliable prior information for the diffusion model.

[0020] 2. The present invention proposes a novel feature-to-flow data conversion module, which uses standardized flow mapping to stabilize noise sampling in the diffusion model denoising process, thereby improving the quality of the generated point cloud. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a basic flow chart of the flow diffusion point cloud generation method based on neighborhood feature enhancement of the present invention.

[0022] Figure 2 It is a structural diagram of the point cloud generation network of the present invention.

[0023] Figure 3 It is a comparison chart of shape measurement indicators with the existing technology.

[0024] Figure 4 It is a process diagram of point cloud generation by the method of the present invention.

[0025] Figure 5 It is a comparison chart with the objective evaluation metrics of the existing technology.

[0026] Figure 6 It is a visualization of the feature structure of multi-item joint training.

[0027] Figure 7 This is the generated graph of the few-sample point cloud in the joint training of multiple objects of the present invention.

[0028] Figure 8 It is a completion map of the present invention on the missing point cloud data.

[0029] Fig. 9 This is a diagram showing the ablation experiment results of the three-dimensional component proposed in the present invention. DETAILED DESCRIPTION

[0030] The technical solution provided by the present application will be further described below in conjunction with specific embodiments and accompanying drawings. The advantages and features of the present application will become more apparent with the following description.

[0031] Example

[0032] Flow diffusion point cloud generation method based on neighborhood feature enhancement, Figure 1 As shown:

[0033] Step 1: Preprocess the original point cloud to be encoded, including normalizing the point cloud data to [-1, 1] and dividing it into training set, test set and validation set according to the ratio of 80%, 15% and 5%;

[0034] Step 2: Build a multi-scale neighborhood feature aggregation module ( Figure 2 ), the multi-scale neighborhood feature aggregation module includes a farthest point sampling component (FPS), multiple multi-scale K nearest neighbor components (k-NN(.)) and multiple coarse feature extraction components (ξ Ψ ); multiple multi-scale K nearest neighbor components are circular K nearest neighbors, with a certain point cloud center point as the center, and search for neighboring k points within a radius r around it. Multiple multi-scales are embodied in the present invention using two layers of circular K nearest neighbors, namely, a nearest neighbor with a radius of 0.05 and a k value of 16 and a nearest neighbor with a radius of 0.1 and a k value of 32; each coarse feature extraction component is composed of a 1×1 convolution (Conv1D), a batch normalization (BN), an activation function (ReLU) and a maximum pooling layer (Max Pooling);

[0035] Specifically, the farthest point sampling component firstly extracts the farthest point from the input point cloud set X = {x1, x2, x3, ..., x N} select m points as the initial point set S = {x1, x2, x3, ..., x m}; For each point x in set X, it calculates the minimum distance to the points in set S. Then it chooses the next point x m+1 , so that its minimum distance from the center point of the set S is maximized; repeat this step until the required number of s center points is selected to obtain a new set S = {x1, x2, x3…, x s}; The formula for the entire farthest point sampling process is as follows:

[0036]

[0037] After the multi-scale K nearest neighbor component obtains the center point set, it performs K nearest neighbor search to establish the neighborhood relationship of each point; for each point p i ∈S, calculate its distance to other points and find the k nearest points and the previous set of neighbors p ijRepresents the distance p i The jth nearest point; this step finds the k nearest points for each center point in the original data X to extract neighborhood features, which helps to construct local features when processing point cloud data; the K nearest neighbor process can be expressed as a formula:

[0038]

[0039] The coarse feature component finally summarizes the K nearest neighbors of each center point to obtain a set of new point coordinates p. Ψ (·) Aggregate the point coordinates. This component uses 1×1 convolution (Conv1D), batch normalization (BN), activation function (ReLU) and maximum pooling layer (Max Pooling) to extract the significant information of local features and concatenate the neighborhood set. Aggregate into global rough feature F, the formula is as follows:

[0040]

[0041] Step 3: Construct a neighborhood attention feature enhancement module to enhance the global coarse features obtained in step 2 through the self-attention mechanism and the farthest point sampling method.

[0042] Specifically, we first use a linear layer to map the global rough feature F to the high-dimensional initial feature map F′ of the point cloud; at the same time, for each point P in the input, we find the 16 nearest points according to its spatial position and obtain the index N of these 16 nearest points. k (p i ), the index formula is as follows:

[0043]

[0044] Then, three fully connected layers (MLP) are used on the high-dimensional initial feature map F′ to map the attention key vector q, the value vector k, and the query vector v respectively:

[0045] q=MLP Q (F)

[0046] k=MLP K (F)

[0047] v=MLP V (F)

[0048] Next, using the position encoding PE, each point p in P is grouped i The coordinates of and their K nearest neighbor index N k (p i ) to generate a new feature set and in The coordinates of P are subtracted point by point; the subtraction result is encoded by the designed position encoding fully connected layer (PEMLP), and the formula of the position encoding PE is as follows:

[0049]

[0050] By applying the same grouping operation used in the positional encoding PE above to the key vector and index N k (p i ) to reshape the query vector feature k′ and concatenate the query vector q with the key vector k′ and the position encoding PE. After being processed by the designed multi-layer attention fully connected layer (AttentionMLP), the softmax function is used to calculate the attention weight a i , perform weighted summation of the value vector v and the position encoding PE to generate the final aggregate feature representation F agg ; Aggregate feature representation F agg Reflects the differences and similarities between the features of different points and their neighboring points; attention weight a i And the aggregate feature formula F agg As shown below:

[0051]

[0052] F agg =∑a i ·(ν+PE)

[0053] Get the aggregated feature F agg Then it is mapped back to the original feature space through full connection and added to the input feature through residual connection to obtain the enhanced feature representation F out , the process is shown in the following formula:

[0054] F out =MLP(F agg )+F′

[0055] Step 4: Introduce a point voxel convolutional neural network to reduce the redundant features in the enhanced features output in step 3, thereby outputting the potential vector of the point cloud; the point voxel convolutional neural network is an existing technology, and PVCNN is used as an example to implement it ( Figure 2 shown);

[0056] Step 5: Design a feature-to-stream data conversion module, which maps the feature-to-stream data conversion module to stream data through affine transformation; the latent vector output in step 4 is used to make the sampling probability of the point cloud continuous through the feature-to-stream data conversion module;

[0057] Specifically, firstly, the flow model is used to represent the given high-dimensional feature w. The model performs the transformation from time point t0 to t1 through continuous time integration, and finally obtains the potential vector z; the formula is expressed as:

[0058]

[0059] Where w represents the prior distribution parameter; f ε (·) is a neural network, which represents the evolution function at time t that controls w over time; logP(F ε -1 (z)) is the probability density of the inverse transformation of z, that is, the density of the prior distribution after the transformation; represents the rate of change from time t0 to t1, Tr represents the trace of the matrix. The purpose is to find the value corresponding to the prior distribution at time t0;

[0060] Then the target distribution is calculated. In order to generate the corresponding latent vector z from the data, the inverse transformation w(t0) is used to find the value corresponding to the initial distribution at time t0. The formula is:

[0061]

[0062] Finally, the probability density of the target distribution is expressed. Using the inverse mapping formula, the ultimate goal is to find the target distribution p(z). The density of the target distribution can be calculated by changing the Jacobian determinant. The formula is:

[0063]

[0064] Where p(z) represents the probability density of the latent variable z; p w (w) represents the probability density of the random variable w; F ε is a transformation function that maps w to z; Yes F ε The inverse absolute value of the Jacobian determinant of w is used to adjust the density scaling factor; F ε -1 (·) is F ε The inverse function of , used to get w from z.

[0065] The core of the above formula is Figure 2 The neural network with affine coupled layers shown in Figure 2 learns a dynamic continuous transformation f θ , and derives the mapping from high-dimensional feature space to latent space through the flow model, while using the inverse transform and Jacobian determinant to calculate the final probability density; the method allows the use of a continuous transformation model to generate the target distribution.

[0066] The above process is specifically implemented through the affine coupling layer, which is composed of 12 MLP, RELU, MLP, RELU, MLP, and Sigmod layers connected in sequence.

[0067] Step 6: Input the flow data in step 5 into the diffusion model to generate point cloud; the flow data is used to replace the sampling process of the diffusion model, because the flow model has deterministic stability and learns the global point cloud features, so it can optimize the cumulative error caused by the diffusion model sampling;

[0068] Specifically, the goal of a given diffusion model is to maximize the target point cloud X( 0 ), expressed as Since it is difficult to optimize this goal directly, this paper introduces the evidence lower bound (ELBO):

[0069]

[0070] The reason for replacing the variational posterior distribution in KL divergence with the flow model is mainly to improve the flexibility and expressiveness of variational inference. The variational inference of traditional diffusion models usually uses a simple distribution (such as Gaussian distribution) to approximate the posterior distribution, but this method is limited by the distribution assumption and cannot capture complex posterior structures. The flow model provides a more flexible and scalable way to learn complex posterior distributions. Substituting the formula in step 5 into And expand it to get the final optimization target formula:

[0071]

[0072] In this way, the objective function combines the KL divergence and log-likelihood to maximize the likelihood of the generative model generating data while ensuring that the posterior distribution is close to the prior distribution; that is, the optimization goal is to train the parameters of the generative model by minimizing the KL divergence so that the model can generate samples consistent with the data distribution.

[0073] Step 7: Compare the generated point cloud obtained in step 6 with the input point cloud and calculate the total loss;

[0074]

[0075] Among them, kl_weight is a hyperparameter with a fixed value of 0.001, which is used to balance the proportion of reconstruction loss and KL divergence term; The entropy loss is used to measure the entropy of the latent variable distribution of the encoder output; The prior loss measures the logarithmic probability of the latent variable under the prior distribution and is calculated by the flow model; logP(X|z) is the reconstruction loss that measures the similarity between the sample x generated by the model and the input sample

[0076] Step 8: According to the total loss obtained in step 7, the Adam optimizer is used to optimize the point cloud generation method end-to-end; the initial learning rate of the optimizer is set to 0.002, and the weight decay is set to 0, that is, there is no regularization term; the maximum gradient norm of the gradient clipping is 10 to prevent gradient explosion; the learning rate scheduler will start to decay after 200,000 iterations, and reduce the learning rate to the termination learning rate of 0.0001 at 400,000 iterations; these settings jointly control the parameter update speed and stability during the training process;

[0077] Step 9: Loop through steps 2 to 8, train for a specified number of rounds until the point cloud generation network of the flow diffusion model with enhanced neighborhood features converges, and save the optimal training model file; the specified number of rounds is set to 1,000,000 times in the experiment, and the program can be interrupted in advance and the optimal training model file can be saved by observing the loss rate curve in real time through TensorBoard;

[0078] Step 10: Select a single object or multiple object categories for training based on actual usage requirements and output the object type that needs to be generated.

[0079] The embodiment of the present invention conducts a comparative experiment on the flow diffusion point cloud generation method based on neighborhood feature enhancement proposed in the present invention and the closest existing technology. The data set used in the experiment is ShapeNetCore v2, which is jointly proposed by researchers from Princeton University, Stanford University and other institutions; it contains more than 50,000 point cloud data models, covering 57 categories; each model has a corresponding geometry file and a unique identifier; the present invention conducts experiments on three subsets of chairs, airplanes and cars in ShapeNet and compares shape metrics with advanced algorithms; a series of state-of-the-art algorithms are selected as comparison benchmarks, which represent the diversity and latest progress in the field of 3D shape generation, including non-diffusion algorithms such as r-GAN, 1-GAN, PointFlow, SoftFlow, SetVAE and DPF-Net, and algorithms based on diffusion models such as DPM, MeshDiffusion, PVD, and specific data such as Figure 3 As shown; analysis Figure 3 The data in show that the performance of the proposed model is comparable to that of the most advanced methods in various indicators; this success is attributed to the comprehensive consideration of local and global information and the effective integration of spatial coordinates by the proposed model; in addition, the flow model stabilizes the diffusion generation process and significantly improves the quality and high-fidelity generation of 3D shapes in the generated point cloud data; to further verify the generation ability of the proposed model in this study, Figure 4The visualization results of the denoising generation process for three different object categories, namely airplanes, chairs and cars, are shown in Figure 1. These visualization results directly present the dynamic evolution of different objects from initial rough shapes to complete structures during the denoising process. In order to more comprehensively evaluate the performance of the model, it is compared with similar cutting-edge algorithms (P0intFlow and DPM) from a subjective evaluation perspective. The relevant visualization comparison results are shown in Figure 1. Figure 5 As shown in the figure; through these comparisons, we can not only observe the performance differences of the models in terms of generation accuracy and detail restoration, but also further reveal the adaptability and robustness of the models in dealing with different object categories; subjective evaluation provides an effective supplement to quantitative indicators (such as EMD, CD, etc.), comprehensively demonstrates the performance of the model in the generation of complex geometric structures, and proves its potential in practical applications; combining subjective and quantitative dual evaluations can more comprehensively demonstrate the performance of the model in the generation of complex geometric structures.

[0080] The above examples demonstrate the effectiveness of the model in generating single-category objects, but joint training on multiple categories of objects remains to be explored. Based on this problem, this study jointly trains a model on six different categories of ShapeNetCore v2 (airplane, chair, car, tower, bag, and basket) without any class conditions. In order to more intuitively demonstrate the performance of the model in the multi-category object generation task, the t-distributed stochastic neighbor embedding (t-SNE) algorithm is used to reduce the dimensionality of the latent vector generated by the model and project it onto a two-dimensional plane. The results are shown in Figure 2. Figure 6 As shown in the figure, most data points of different categories show an obvious separation trend in two-dimensional space; data points of different object categories such as airplanes, chairs, cars, towers, bags and baskets are gathered in different areas, forming relatively independent clusters; this separation trend shows that the model can effectively capture and learn the features of objects of different categories and clearly distinguish them in the latent vector space; especially in the absence of category constraints, the model still shows a strong ability to distinguish categories, which shows that it has good adaptability to multi-category features; from the point cloud generated by the joint training, some small sample data are randomly selected for display, covering three categories of objects: towers, bags and baskets; each type of point cloud contains about 100 to 200 samples, and the number of test samples is about 11 to 24; the visualization results are shown in the figure. Figure 7As shown in the figure, the generated point cloud has uniform surface coverage and no obvious gaps while maintaining novelty and high fidelity; the experimental results verify the feasibility of multi-category joint training, especially in the task of generating few-sample point cloud data, it shows strong learning ability; therefore, this research method is not only suitable for generating high-quality single-category object point clouds, but also can successfully achieve multi-category joint generation without increasing complexity, avoiding the need to train a separate model for each category; this result provides new ideas and possibilities for the practical application of multi-category object generation.

[0081] In addition to the above-mentioned embodiment analysis, the present invention also explores the upsampling capability of the proposed model under the condition of fewer input point clouds; during the reasoning process, the proposed model can generate a series of point-to-point distance samples for each input point cloud, thereby completing the point cloud; specifically, in the experiment, 150 and 200 points are used as the input of each point set of the airplane and the car, respectively, and the complete point cloud is synthesized through the above-mentioned pre-trained model, combining the global shape variables with partial point cloud information; Figure 8 The qualitative visualization results of the output point cloud are shown. The experimental results show that even when the input points are relatively sparse, the trained model can still accurately perform point cloud upsampling and generate a complete point cloud with high fidelity; this capability proves the adaptability and generalization ability of the method proposed in this patent in the case of a small amount of point cloud input, and provides a reliable solution for upsampling and completion of point cloud data.

[0082] In order to prove the effectiveness of the module design introduced in the solution of the present invention, the method of this embodiment conducted an ablation study and conducted ablation tests on the constructed multi-scale neighborhood feature aggregation module (MNFA), neighborhood attention feature enhancement module (NAFE) and point voxel convolutional neural network (PVCNN); in order to verify the effectiveness of the present invention in multi-category training, three subsets (chair, car and airplane) were selected for joint training and tested on the airplane class; each group of experiments was iterated for 200,000 steps; Fig. 9 The quantitative results of the ablation experiments are summarized, and the necessity of each module in the model is demonstrated by ablating each module. The experimental results show that the model exhibits the best performance when all modules are integrated; the performance decreases significantly after removing any module; in particular, similar numerical results are observed when PVCNN is removed and reintroduced, which further demonstrates the key role of the multi-scale neighborhood feature aggregation module (MNFA) and the neighborhood attention feature enhancement module (NAFE) in the model; these two modules work together on point cloud data to better capture the complexity of the geometric structure, thereby improving the effect of point cloud generation.

[0083] The above description is only a description of the preferred embodiments of the present application, and is not intended to limit the scope of the present application. Any changes or modifications made by any person skilled in the art based on the above disclosed technical contents shall be deemed as equivalent effective embodiments and shall fall within the scope of protection of the technical solution of the present application.

Claims

1. A flow diffusion point cloud generation method based on neighborhood feature enhancement, characterized in that: The steps include: Step 1: Preprocess the original point cloud to be encoded, including normalizing the point cloud data to [-1, 1] and dividing it into training set, test set and validation set; Step 2: construct a multi-scale neighborhood feature aggregation module, wherein the multi-scale neighborhood feature aggregation module includes a farthest point sampling component, multiple multi-scale K nearest neighbor components and multiple coarse feature extraction components; Step 3: construct a neighborhood attention feature enhancement module, which enhances the global coarse features obtained in step 2 through a self-attention mechanism and farthest point sampling; Step 4: Use the point voxel convolutional neural network to reduce the redundant features in the enhanced features output in step 3 and output the latent vector of the point cloud; Step 5: Design a feature-to-stream data conversion module, which maps the feature-to-stream data conversion module to stream data through affine transformation; the latent vector output in step 4 is used to make the sampling probability of the point cloud continuous through the feature-to-stream data conversion module; Step 6: Input the flow data in step 5 into the diffusion model to generate point cloud; Step 7: Compare the generated point cloud obtained in step 6 with the input point cloud and calculate the loss; Step 8: According to the total loss obtained in step 7, use the Adam optimizer to optimize the point cloud generation method end-to-end; Step 9: Loop through steps 2 to 8, train for a specified number of rounds, until the point cloud generation network of the flow diffusion model with enhanced neighborhood features converges, and save the optimal training model file; Step 10: Select a single object or multiple object categories for training based on actual usage requirements and output the object type that needs to be generated.

2. The method for generating a flow diffusion point cloud based on neighborhood feature enhancement according to claim 1, characterized in that: Step 2: The farthest point sampling component first samples the input point cloud set X = {x1, x2, x3, ..., x N } select m points as the initial point set S = {x1, x2, x3, ..., x m }; For each point x in set X, it calculates the minimum distance to the points in set S. Then it chooses the next point x m+1 , so that its minimum distance from the center point of the set S is maximized; repeat this step until the required number of s center points is selected to obtain a new set S = {x1, x2, x3…, x s }; The formula for the entire farthest point sampling process is as follows: Where x represents the point whose distance is to be calculated; S represents the selected point set; || xx j || represents point x and point x in the set j The Euclidean distance between them; d(x, S) means finding the minimum distance from point x to all points in set S; X\S means the points in data set X that do not belong to set S; argmax means finding the point x that maximizes d(x, S); x i+1 It means to find the point farthest from the set S from the points that have not yet been added to the set S, and use it as the next point to be selected. After the multi-scale K nearest neighbor component obtains the central point set, it performs K nearest neighbor search to establish the neighborhood relationship of each point; for each point p i ∈S, calculate its distance to other points and find the k nearest points and the previous set of neighbors p ij Represents the distance p i The jth closest point; This step finds k nearest points for each center point in the original data X to extract neighborhood features, which helps to construct local features when processing point cloud data; the K nearest neighbor process can be expressed as a formula: Where s i represents the i-th center point in the center point set; For each center point s i Find the k nearest neighbor points in the point set X; Represents the k neighbor points found for a specific center point. The coarse feature component finally summarizes the K nearest neighbors of each center point to obtain a set of new point coordinates p. Ψ (·) Aggregate point coordinates. This component uses 1×1 convolution, batch normalization, ReLU, and max pooling layers to extract significant information of local features and concatenate the neighborhood set Aggregate into rough global features, the formula is as follows: Where Contact represents the module ξ Ψ The output features are concatenated to obtain the global coarse-grained features F.

3. The method for generating a flow diffusion point cloud based on neighborhood feature enhancement according to claim 1, characterized in that: Step 3: First, a linear layer is used to map F to a high-dimensional initial feature map F′ of the point cloud; at the same time, for each point P in the input, the 16 nearest points are found according to its spatial position, and the indexes N of these 16 nearest points are obtained. k (p i ), the index formula is as follows: Where p a and p b YesP i Then, three fully connected layers are used on the high-dimensional initial feature map F′ to map the attention key vector q, the value vector k, and the query vector v respectively: q=MLP Q (F′) k=MLP K (F′) v=MLP V (F′) Next, the position encoding PE groups each point p in P i The coordinates of and their K nearest neighbor index N k (p i ) to generate a new feature set and in The coordinates of P are subtracted point by point; the subtraction result is encoded by the designed position encoding fully connected layer. The position encoding formula is as follows: By applying the same grouping operation used in the positional encoding above to the key vector and index N k (p i ) to reshape the query vector feature k′ and concatenate the query vector q with the key vector k′ and the position encoding PE. After being processed by the designed multi-layer perceptron AttnMLP, the softmax function is used to calculate the attention weight a i , perform weighted summation of the value vector ν and the position encoding PE to generate the final aggregate feature representation F agg ; F agg It reflects the differences and similarities between the features of different points and their neighboring points; the attention weight and aggregation feature formula are as follows: Get the aggregated feature F agg Then it is mapped back to the original feature space through full connection and added to the input feature through residual connection to obtain the enhanced feature representation F out , the process is shown in the following formula: F out =MLP(F agg )+F′。 4. The method for generating a flow diffusion point cloud based on neighborhood feature enhancement according to claim 1, characterized in that: Step 5: Convert discrete features into a continuous stream model and transform them from the form of data x into the form of a latent vector z: Where w represents the prior distribution parameter; f ε (·) is a neural network, which represents the evolution function at time t that controls w over time; logP(F ε -1 (z)) is the probability density of the inverse transformation of z, that is, the density of the prior distribution after the transformation; Represents the rate of change from time t0 to t1, Tr represents the trace of the matrix. The purpose is to find the value corresponding to the prior distribution at time t0, so the inverse operation of the flow is used: The inverse transformation of the input is performed using affine coupled layers; each layer transforms part of the current input while keeping the other part unchanged, ensuring a one-to-one correspondence between z and w; this method allows the probability of the target distribution to be accurately calculated by applying the change of variable formula: Where p(z) represents the probability density of the latent variable z; p w (w) represents the probability density of the random variable w; F ε is a transformation function that maps w to z; Yes F ε The inverse absolute value of the Jacobian determinant of w is used to adjust the density scaling factor; F ε -1 (·) is F ε The inverse function of , used to get w from z.

Images