A point cloud completion method and system based on a self-supervised conditional diffusion model

Abstract

The application belongs to the technical field of computer vision, and provides a point cloud completion method and system based on a self-supervised conditional diffusion model, which comprises the following steps: obtaining a target image; segmenting the obtained target image, mapping segmentation information of the target image to a point cloud space, and obtaining a preliminary incomplete point cloud; performing point cloud denoising processing on the obtained preliminary incomplete point cloud; capturing point cloud space geometric features according to the denoised preliminary incomplete point cloud and a self-supervised conditional diffusion model adopting a multi-scale feature fusion mechanism, inferring a completed point cloud, generating completed point cloud data, and completing point cloud completion based on the self-supervised conditional diffusion model.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision, and particularly relates to a point cloud completion method and system based on a self-supervised conditional diffusion model. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute prior art.

[0003] With the rapid development of 3D sensor technology, 3D point cloud, as an important data format, has been widely used in many fields due to its convenience in acquisition and high storage efficiency. However, the scanning process in reality is often limited by the viewing angle and self-occlusion phenomenon, resulting in the incompleteness of the generated point cloud data, which brings significant challenges to 3D reconstruction, augmented reality and scene understanding tasks. Therefore, effective solutions are needed to achieve high-quality point cloud completion.

[0004] To address the incompleteness of point cloud data, researchers have proposed various deep learning-based methods, usually using Chamfer distance (CD) or Earth Mover's Distance (EMD) as the loss function to reduce the difference between the generated point cloud and the real point cloud. However, CD loss, although performs well in point cloud matching, is not sensitive enough to the density distribution of the overall point cloud, which can easily lead to uneven distribution of the generated point cloud. On the other hand, EMD performs well in fine structure alignment and differentiation, but its high computational cost limits its application in complex scenarios and large-scale data processing.

[0005] Existing shape completion methods can be roughly divided into two categories: deterministic methods and probabilistic methods. Deterministic methods rely on fixed mapping functions and can achieve good performance in point cloud completion accuracy through supervised learning, but are prone to overfitting and produce artifacts or unnatural phenomena. Probabilistic methods generate more reasonable completion results through autoencoders or generative adversarial networks, but still face challenges in precision and stability when dealing with noise-polluted shapes. SUMMARY

[0006] To solve the above problems, the present application proposes a point cloud completion method and system based on a self-supervised conditional diffusion model, which generates uniform and high-quality point clouds to achieve efficient and high-quality training when using simple mean square error loss instead of traditional Chamfer distance and Earth Mover's distance loss. The ability to capture geometric details significantly improves the control ability of the point cloud completion process. Combined with the self-supervised learning mechanism, the point cloud completion effect is improved by comparing with the generated point cloud without a large amount of labeled data.

[0007] According to some embodiments, the first aspect of the present application provides a point cloud completion method based on a self-supervised conditional diffusion model, which adopts the following technical solution:

[0008] A point cloud completion method based on a self-supervised conditional diffusion model, comprising:

[0009] Obtaining a target image;

[0010] Segmenting the obtained target image, mapping the segmentation information of the target image to a point cloud space, and obtaining a preliminary incomplete point cloud;

[0011] Performing point cloud denoising processing on the obtained preliminary incomplete point cloud;

[0012] Based on the preliminary incomplete point cloud after denoising and the self-supervised conditional diffusion model adopting a multi-scale feature fusion mechanism to capture the geometric features of the point cloud space, the missing point cloud is inferred to generate completed point cloud data, and the point cloud completion based on the self-supervised conditional diffusion model is completed.

[0013] As a further technical limitation, the target object is photographed to obtain a single-view target image; after pre-processing the target image, SAM is used for image segmentation of the target image to generate segmentation information of the target image; the obtained segmentation information is mapped to a point cloud space in combination with the corresponding relationship between pixels and point clouds to obtain a preliminary incomplete point cloud.

[0014] As a further technical limitation, a denoising method based on density estimation is used to perform denoising processing on the preliminary incomplete point cloud, by removing redundant points or abnormal points, and retaining point cloud data that can reflect the geometric detail information of the target object, to obtain the preliminary incomplete point cloud after denoising.

[0015] As a further technical limitation, through a multi-scale feature fusion mechanism and a feature consistency strategy, the self-supervised conditional diffusion model captures the local and global spatial geometric features of the preliminary incomplete point cloud after denoising, integrates the incomplete shapes of different sizes in combination with the multi-scale feature fusion mechanism, the denoising branch and the conditional branch, infers the missing point cloud to generate completed point cloud data.

[0016] Further, based on the conditional branch, multiple features are extracted, the extracted features are projected back to a tensor in the same format as the original incomplete point cloud to obtain a conditional branch point cloud; based on the obtained conditional branch point cloud and the original incomplete point cloud, a point cloud loss function is obtained, the Chamfer distance between the generated point cloud and the incomplete point cloud is calculated according to the obtained point cloud loss function, the geometric information of the incomplete point cloud is captured, and the quality of the generated point cloud is improved by optimizing the obtained point cloud loss function.

[0017] As a further technical limitation, the self-supervised conditional diffusion model adopts a denoising diffusion probability network, compensates for the sparsity of the original point cloud through voxelization processing, enhances the generalization capability of multi-level feature fusion and geometric information capture, and gradually adds and removes Gaussian noise to generate high-quality point clouds from random noise.

[0018] According to some embodiments, the second aspect of the present application provides a point cloud completion system based on a self-supervised conditional diffusion model, which adopts the following technical solutions:

[0019] A point cloud completion system based on a self-supervised conditional diffusion model comprises:

[0020] An image-point cloud acquisition module configured to acquire a target image and a target point cloud;

[0021] An image-point cloud mapping module configured to segment the acquired target image, map the segmentation information of the target image to a point cloud space, and obtain a preliminary incomplete point cloud;

[0022] A point cloud denoising module configured to perform point cloud denoising processing on the obtained preliminary incomplete point cloud;

[0023] A point cloud completion module configured to capture point cloud space geometric features according to the denoised preliminary incomplete point cloud and a self-supervised conditional diffusion model adopting a multi-scale feature fusion mechanism, to infer a complete missing point cloud, to generate completed point cloud data, and to complete point cloud completion based on a self-supervised conditional diffusion model.

[0024] According to some embodiments, the third aspect of the present application provides a computer readable storage medium, which adopts the following technical solutions:

[0025] A computer readable storage medium having a program stored thereon, the program being executed by a processor to implement the steps in the point cloud completion method based on a self-supervised conditional diffusion model according to the first aspect of the present application.

[0026] According to some embodiments, the fourth aspect of the present application provides an electronic device, which adopts the following technical solutions:

[0027] An electronic device comprising a memory, a processor, and a program stored on the memory and running on the processor, the processor executing the program to implement the steps in the point cloud completion method based on a self-supervised conditional diffusion model according to the first aspect of the present application.

[0028] According to some embodiments, the fifth aspect of the present application provides a computer program product, which adopts the following technical solutions:

[0029] A computer program product comprising software code, the program in the software code performing the steps in the point cloud completion method based on a self-supervised conditional diffusion model according to the first aspect of the present application.

[0030] Compared with the prior art, the present application has the following beneficial effects:

[0031] The present application generates uniform and high-quality point clouds through a self-supervised conditional diffusion model to achieve efficient and high-quality training when using a simple mean square error loss instead of traditional Chamfer distance and Earth Mover distance losses, significantly improves the control ability of the point cloud completion process through the ability to capture geometric details, and improves the point cloud completion effect through comparison with the self-generated point cloud without a large amount of labeled data. BRIEF DESCRIPTION OF DRAWINGS

[0032] The drawings constituting a part of this embodiment are used to provide further understanding of the present embodiment, and the schematic embodiments of the present embodiment and their descriptions are used to explain the present embodiment and do not constitute an improper limitation on the present embodiment.

[0033] Figure 1 The figure is the architecture diagram of the point cloud completion method based on a self-supervised conditional diffusion model in the first embodiment of the present application.

[0034] Figure 2 The figure is the principle diagram of the point cloud completion method based on a self-supervised conditional diffusion model in the first embodiment of the present application.

[0035] Figure 3 The figure is the overall structure diagram of the self-supervised conditional diffusion model in the first embodiment of the present application.

[0036] Figure 4 The figure is the structure block diagram of the point cloud completion system based on a self-supervised conditional diffusion model in the second embodiment of the present application. DETAILED DESCRIPTION

[0037] The present application will be further described below in combination with the drawings and embodiments.

[0038] It should be pointed out that the following detailed description is exemplary and is intended to provide further description of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as generally understood by those skilled in the art to which the present application belongs.

[0039] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments in accordance with the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, steps, operations, elements, components, and / or groups thereof, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0040] In the present application, the terms such as "upper", "lower", "left", "right", "front", "back", "vertical", "horizontal", "side", "bottom" and the like indicate the orientation or positional relationship shown in the drawings, which are only the relationship words determined for the purpose of describing the structural relationship of the components or elements of the present application, and are not intended to specify any component or element in the present application, and cannot be understood as a limitation of the present application.

[0041] In the present application, the terms such as "fixedly connected", "connected", "connected" and the like should be understood broadly, which means that it can be fixedly connected, integrally connected or detachably connected; it can be directly connected or indirectly connected through an intermediate medium. For relevant scientific or technical personnel in the art, the specific meaning of the above terms in the present application can be determined according to the specific circumstances, and cannot be understood as a limitation of the present application.

[0042] In the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0043] Embodiment one

[0044] The embodiment one of the present application introduces a point cloud completion method based on a self-supervised conditional diffusion model.

[0045] The present embodiment proposes a new diffusion model, namely a self-supervised conditional diffusion model SCDNet, to generate high-fidelity and natural point cloud completion results. Denoising Diffusion Probabilistic Models (DDPM) is adopted to generate uniform and high-quality point clouds by gradually removing Gaussian noise, so as to realize efficient and high-quality training when using a simple mean square error (MSE) loss instead of traditional Chamfer distance and EMD loss. Through a multi-scale feature fusion mechanism and a feature consistency strategy, the incomplete shape input from different scales is effectively integrated to maintain spatial consistency and enhance the ability of the model to capture geometric details, thereby significantly improving the control ability of the completion process. A self-supervised learning mechanism is introduced to continuously improve the completion effect by comparing with the generated point cloud without a large amount of labeled data.

[0046] As Figure 1As shown, the embodiment uses a RealSense depth camera to capture the target object, obtaining a single-view image and its corresponding point cloud data. To extract the incomplete point cloud of the target object, the Segment Anything Model (SAM) is used to perform image segmentation on the captured image, generating an accurate segmentation result of the object; using the correspondence between pixels and point clouds, the segmentation information in the image is mapped to the point cloud space, thereby obtaining a preliminary incomplete point cloud.

[0047] In this embodiment, the obtained image can be preprocessed and data enhanced, such as rotation, scaling, translation, etc., to increase the diversity of training data and improve the adaptability of the image to different scenes.

[0048] Since the preliminary obtained point cloud may contain noise data (such as interference of other background objects or measurement error of the depth camera), the embodiment introduces a denoising algorithm to process the incomplete point cloud. The denoising algorithm removes redundant points or abnormal points to ensure that the retained point cloud data can more accurately reflect the geometric shape of the target object, thereby obtaining a cleaner incomplete point cloud; the embodiment can use a denoising method based on density estimation, which can intelligently detect and remove noise points while retaining important geometric detail information, to maximize the quality of the completed point cloud.

[0049] The embodiment inputs the denoised incomplete point cloud into a pre-set SCDNet for completion, effectively capturing local and global spatial geometric features in the point cloud. The self-supervised conditional diffusion model in the embodiment uses a multi-scale feature fusion mechanism, combining the denoising branch and the conditional branch to extract spatial information from the incomplete point cloud, predict and complete the missing parts, and generate complete point cloud data; using a self-supervised strategy, the model can better extract the features of the incomplete point cloud through the reconstruction condition, improving the completion effect and precision; the self-supervised learning mechanism ensures that the completed point cloud has overall consistency and retains rich details.

[0050] The point cloud data completed by the SCDNet in this embodiment has complete geometric morphology, which can provide basic data support for subsequent modeling, simulation, and robot operation tasks. The generated complete point cloud can not only be used for three-dimensional model reconstruction, but also be applied to fine robot operation tasks such as path planning and object grasping.

[0051] This scheme effectively improves the completeness and precision of point cloud data by combining depth cameras, image segmentation, and point cloud completion technology, and is suitable for point cloud completion tasks in complex scenes, providing a reliable data foundation for various robot applications.

[0052] The point cloud completion process adopted by this embodiment is as follows Figure 2As shown in the figure, the adopted diffusion denoising probabilistic model (DDPM) uses a Markov process to gradually add Gaussian noise to the data until the distribution of the original data is close to the prior distribution of the target; the diffusion model is applied to the point cloud completion task, the incomplete point cloud is used as conditional input, and the complete point cloud is predicted by the generative model.

[0053] Assume that the complete point cloud x in the dataset i The distribution of p data , x i The corresponding local point cloud is c i , the shape completion task is expressed as i Generate x for the condition i The generative model used for this task is the conditional diffusion model. Specifically, the conditional diffusion model consists of two Markov chains, one called the forward diffusion process and the other called the backward diffusion process. Each process has the same time step T, which is set to 1000 in this example.

[0054] The forward diffusion process is to gradually add Gaussian noise to p data Until p data Close to a standard normal distribution formal, For the sake of brevity, the number i is omitted in the following discussion of this embodiment. 0 to x T Defined as:

[0055]

[0056]

[0057] Among them, the hyperparameter β t ∈[0,1] is a small predefined constant used to control the noise level at each step.

[0058] The inverse process is a Markov process, which is used to predict and eliminate the noise introduced by the diffusion process, and to generate the inverse based on the conditional variables (i.e., part of the observed point cloud data c); T Start by gradually restoring to clean point cloud data x 0 ; The reverse process starts from x T to x 0 The conversion is defined as:

[0059]

[0060] Among them, p θ (x t-1 |x t ,c) is Gaussian distribution:

[0061]

[0062] Mean μ θ (x t ,c,t) is parameterized by neural network, while variance is a constant related to time step t. The process of generating a sample is to sample from the standard normal distribution first, and then sample from x T-1 ,x T-2 ,…,x 0 in turn according to the conditional probability of the inverse process, and finally output x 0 .

[0063] Through the forward and backward processes in this embodiment, the complex distribution characteristics of the point cloud can be effectively learned, and finally a high-quality complete point cloud is generated.

[0064] This embodiment adopts a self-supervised conditional diffusion model SCDNet (Self-supervised Conditional Diffusion Network) ∈ θ as shown in Figure 3 , the input is a noisy point cloud x t , an incomplete point cloud c, and a diffusion time step t. Intuitively, the output of ∈ θ can be understood as the offset of each point between x t and x t-1 . At the same time, the self-supervised conditional diffusion model needs to effectively fuse multi-level and multi-scale features from the conditional information c, so as to generate fine details that meet the conditional constraints while inferring the overall shape.

[0065] As one or more implementations, point cloud is different from image, because the points in three-dimensional space have no fixed order, and this disorder increases the difficulty of modeling. When training the conditional diffusion model (SCDNet), due to the sparse distribution of points in the 3D embedding space, it is difficult to capture the structural features of the data by directly optimizing the point coordinates. Therefore, this embodiment adopts a voxelization processing strategy to convert the point cloud into a dense representation, so that the diffusion model can better extract information from the voxel features;

[0066] Specifically, for each point cloud containing N points , the point coordinates of which are x, y, z, it is first voxelized into input , wherein V is the size of the voxel grid; each voxel contains a three-dimensional vector representing the average coordinates or geometric features within the voxel. In this representation, the diffusion Transformer can more effectively capture the local structure and global geometric information of the point cloud, and improve the processing capability in the sparse 3D space.

[0067] In addition, to avoid the shortcomings of traditional loss functions, the embodiment adopts a denoising diffusion probability model (DDPM). Chamfer distance (CD) and earth mover's distance (EMD) can measure the difference between point clouds, but each has its own shortcomings; that is, CD is sensitive to local matching but ignores global density, making it easy to generate point clouds with uneven distribution; while EMD is excellent in structure alignment but has high computational complexity, that is, These problems make it difficult for the model to train efficiently on large-scale data. DDPM generates high-quality point clouds from random noise by gradually adding and removing Gaussian noise, avoiding the direct use of CD and EMD. Given an initial point cloud x0 and noise ∈, the process of generating a noisy point cloud is where α t controls the noise intensity. During training, the model minimizes the following noise prediction error, that is, Here, ∈ θ is the model's prediction of the noise. Through this step-by-step denoising, the model can generate uniform and fine point clouds without relying on CD and EMD.

[0068] The combination of voxelization processing and diffusion Transformer further improves the performance of SCDNet. Voxel representation compensates for the sparsity of the original point cloud, enabling the model to have stronger generalization capabilities in multi-level feature fusion and geometric information capture. Only one voxelized point cloud and its noise need to be sampled at each time step, greatly simplifying the training process.

[0069] SCDNet uses the DDPM model to achieve efficient and high-quality point cloud generation through voxelization processing and denoising diffusion mechanism. Compared with traditional loss, DDPM improves training efficiency with a simpler mean square error (MSE) optimization method, and effectively captures the structural features in sparse 3D data, making it perform well in complex scenarios.

[0070] The embodiment adopts a double-layer feature fusion architecture to effectively integrate multi-scale features of the denoising branch and the conditional branch, aiming to improve the performance of the point cloud denoising task. In this architecture, the denoising branch is mainly responsible for feature extraction of complete point clouds, while the conditional branch extracts multi-scale conditional features from incomplete point clouds. Although the network structures of these two branches are similar, they do not share parameters to better adapt to different input characteristics. To fully utilize the context information provided by the conditional branch, the embodiment adopts a layer-by-layer fusion mechanism to integrate the features of the denoising branch and the conditional branch at each network level.

[0071] The denoising branch receives the input point cloud x t , maps it to a high-dimensional feature space through the preprocessing module ∈ x (·), that is, The multi-scale features are extracted layer by layer through N SCD-3D encoding blocks, and the feature of each layer is represented as At the same time, the conditional branch receives the incomplete point cloud c, which is also preprocessed by the preprocessing module ∈ c (·) maps it to a high-dimensional feature space The conditional branch also extracts multi-scale features through N encoding blocks

[0072] To effectively integrate the features of the denoising branch and the conditional branch in each layer, the embodiment performs layer-by-layer fusion on multiple network levels. Specifically, in the i-th layer, the feature of the denoising branch and the feature of the conditional branch are added element by element to obtain the fused feature is This fusion mechanism can effectively integrate the local and global features extracted by the denoising branch from the complete point cloud and the contextual information obtained by the conditional branch from the incomplete point cloud, thereby enhancing the modeling capability of the denoising branch at the multi-scale level.

[0073] The fused feature is taken as the input of the next layer of the denoising branch and further extracts features through the SCD-3D encoding block of the next layer, This mechanism of layer-by-layer transmission of fused features ensures that the conditional information can be effectively injected into each level of the denoising branch, improving the denoising effect of the network on point cloud data. In addition, to adapt to the time dynamic changes in the diffusion model, the embodiment introduces a time step embedding, which converts the time step t in the diffusion process into an embedding vector and processes it through a multi-layer perception (MLP). The time embedding is added to the feature of each layer to ensure that the network can dynamically adjust according to the input feature at different time steps.

[0074] The multi-scale features after fusion are reconstructed by the decoder of the denoising branch. The decoder of the denoising branch restores the high-dimensional features to the noise distribution of the original point cloud, and finally matches the output of the decoder with the real Gaussian noise through the devoxelization module. This denoising process not only utilizes the complete point cloud information extracted by the denoising branch, but also combines the contextual features provided by the conditional branch, enhancing the network's ability to recover complex point cloud data.

[0075] The double-layer feature fusion strategy in this embodiment realizes effective cooperation of conditional information and denoising information by adding the features of the denoising branch and the conditional branch layer by layer. By integrating the context information of the conditional branch in the denoising branch, this embodiment can improve the understanding ability of the denoising branch for the complete point cloud at different levels, especially when processing incomplete point clouds, the model can better restore the detailed features. In addition, the introduction of time step embedding enables the model to adapt to the dynamic changes at different time steps in the diffusion process.

[0076] To improve the completion effect of the model when processing incomplete point clouds, this embodiment introduces a self-supervised learning mechanism, which enables the model to improve the feature extraction ability of incomplete point clouds through self-supervised learning without the need for a large amount of labeled data. The core idea of this innovation is to let the conditional branch learn the features of incomplete point clouds by comparing with the point clouds generated by itself, thereby enhancing the performance of point cloud completion.

[0077] Specifically, the multi-scale features extracted by the conditional branch are After that, this embodiment designs an inverse voxelization module to map these features back to a tensor with the same format as the original incomplete point cloud c. The high-dimensional features of the conditional branch are converted into a spatial distribution consistent with the incomplete point cloud, denoted as That is, wherein, is the final output feature of the conditional branch at the Nth layer.

[0078] To enable the conditional branch to better learn and retain the features of the incomplete point cloud, the Chamfer Distance Loss (CDLoss) is introduced between the point cloud generated by the conditional branch and the real incomplete point cloud c, and its expression is

[0079] This loss function calculates the Chamfer distance between the generated point cloud and the incomplete point cloud to ensure that the features extracted by the conditional branch can effectively capture the geometric information of the incomplete point cloud. By optimizing CDLoss, the conditional branch gradually improves the quality of the generated point cloud, thereby improving the modeling ability of the incomplete point cloud.

[0080] The self-supervised mechanism in this embodiment not only continuously improves the feature extraction ability of the conditional branch by comparing the self-generated point cloud with the incomplete point cloud without the need for a large amount of labeled data, but also ensures that the information of the incomplete point cloud is fully retained in the feature fusion process after combining with the double-layer architecture. Through CDLoss optimization, the multi-scale conditional information provided by the conditional branch is more accurate, enabling the denoising branch to further improve the point cloud completion effect in the fusion process.

[0081] By introducing a self-supervised learning mechanism, the model significantly enhances the learning ability in the absence of labeled data, enabling the conditional branch to adaptively optimize the feature extraction of the incomplete point cloud. At the same time, the addition of CDLoss ensures that the geometric error between the point cloud generated by the conditional branch and the true incomplete point cloud is smaller. Combined with the double-layer feature fusion architecture, this mechanism further improves the overall effect of point cloud completion. Experimental results show that there is significant improvement in reconstructing the details of the incomplete point cloud and preserving local features.

[0082] Example analysis

[0083] This embodiment performs point cloud completion experiments on the MVP dataset; the MVP dataset contains 62400 pairs of partially complete point clouds for training and 41600 pairs of test point clouds (sampled from ShapeNet); each partial point cloud contains 2048 points. In particular, the MVP dataset provides real point clouds of different resolutions, including 2048, 4096, 8192, and 16384 points. In addition, the MVP dataset consists of partial and complete shapes of 16 shape categories (airplane, cabinet, car, chair, lamp, sofa, table, boat, bed, bench, bookshelf, bus, guitar, motorcycle, toy, and skateboard) for training and testing. By using more shape categories, training and evaluating the network on the MVP dataset becomes more challenging.

[0084] This embodiment uses Chamfer distance (CD) and F1 score to evaluate the quality of the generated point cloud, and the experimental results obtained are shown in Table 1; to compensate for the problem that CD loss is sensitive to outliers, F1 score is used to explicitly evaluate the distance between object surfaces; F1 score is defined as the harmonic mean of precision LP(ρ) and recall LR(ρ), i.e. where, And ρ is a pre-defined distance threshold. In this embodiment, MVP sets ρ = 10 -4 .

[0085] Table 1 Experimental results

[0086]

[0087] This embodiment generates uniform and high-quality point clouds through a self-supervised conditional diffusion model to achieve efficient and high-quality training when using a simple mean square error loss instead of traditional Chamfer distance and earth-moving distance loss, and significantly improves the control ability over the point cloud completion process through the ability to capture geometric details; combined with the self-supervised learning mechanism, the point cloud completion effect is improved through comparison with the point cloud generated by itself without a large amount of labeled data.

[0088] Embodiment two

[0089] Embodiment two of the present application introduces a point cloud completion system based on a self-supervised conditional diffusion model.

[0090] As shown in Figure 4 a point cloud completion system based on a self-supervised conditional diffusion model, comprising:

[0091] An image-point cloud acquisition module configured to acquire a target image and a target point cloud;

[0092] An image-point cloud mapping module configured to segment the acquired target image, map the segmentation information of the target image to a point cloud space, and obtain a preliminary incomplete point cloud;

[0093] A point cloud denoising module configured to perform point cloud denoising processing on the obtained preliminary incomplete point cloud;

[0094] A point cloud completion module configured to capture point cloud space geometric features according to the denoised preliminary incomplete point cloud and a self-supervised conditional diffusion model adopting a multi-scale feature fusion mechanism, infer a complete missing point cloud, and generate completed point cloud data, thereby completing point cloud completion based on the self-supervised conditional diffusion model.

[0095] The detailed steps are the same as those of the point cloud completion method based on the self-supervised conditional diffusion model provided in embodiment one, and will not be repeated here.

[0096] Embodiment three

[0097] Embodiment three of the present application provides a computer-readable storage medium.

[0098] A computer-readable storage medium having a program stored thereon, the program being executed by a processor to implement the steps in the point cloud completion method based on the self-supervised conditional diffusion model as described in embodiment one of the present application.

[0099] The detailed steps are the same as those of the point cloud completion method based on the self-supervised conditional diffusion model provided in embodiment one, and will not be repeated here.

[0100] Embodiment four

[0101] Embodiment four of the present application provides an electronic device.

[0102] An electronic device comprising a memory, a processor, and a program stored on the memory and running on the processor, wherein the processor executes the program to implement the steps in the point cloud completion method based on the self-supervised conditional diffusion model as described in embodiment one of the present application.

[0103] The detailed steps are the same as those of the point cloud completion method based on the self-supervised conditional diffusion model provided in embodiment one, and will not be repeated here.

[0104] Embodiment five

[0105] Embodiment five of the present application provides a computer program product.

[0106] A computer program product comprising software code, wherein the program in the software code performs the steps in the point cloud completion method based on the self-supervised conditional diffusion model as described in embodiment one of the present application.

[0107] The detailed steps are the same as the point cloud completion method based on the self-supervised conditional diffusion model provided in embodiment one, which will not be repeated here.

[0108] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Those skilled in the art can make various modifications and changes to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A point cloud completion method based on a self-supervised conditional diffusion model, characterized in that, The method comprises the following steps: acquiring a target image; segmenting the acquired target image, mapping segmentation information of the target image to a point cloud space, and obtaining a preliminary incomplete point cloud; performing point cloud denoising processing on the obtained preliminary incomplete point cloud; capturing point cloud space geometric features according to the denoised preliminary incomplete point cloud and a self-supervised conditional diffusion model adopting a multi-scale feature fusion mechanism, inferring to complete the missing point cloud, generating complete point cloud data, and completing point cloud completion based on the self-supervised conditional diffusion model; The self-supervised conditional diffusion model captures local and global spatial geometric features in the denoised preliminary incomplete point cloud through a multi-scale feature fusion mechanism and a feature consistency strategy. It integrates incomplete shapes of different sizes, infers to complete the missing point cloud, and generates complete point cloud data by combining the multi-scale feature fusion mechanism, a denoising branch, and a conditional branch. The denoising branch is used for feature extraction of complete point cloud, and the conditional branch is used for extracting multi-scale conditional features from the incomplete point cloud. To fully utilize the context information provided by the conditional branch, a layer-by-layer fusion mechanism is adopted to integrate the features of the denoising branch and the conditional branch at each network level. Based on the conditional branch extracting multi-scale features, the extracted features are mapped back to a tensor in the same format as the original incomplete point cloud to obtain a conditional branch point cloud. Based on the obtained conditional branch point cloud and the original incomplete point cloud, a point cloud loss function is obtained. The Chamfer distance between the generated point cloud and the incomplete point cloud is calculated according to the obtained point cloud loss function, the geometric information of the incomplete point cloud is captured, and the quality of the generated point cloud is improved by optimizing the obtained point cloud loss function.

2. The point cloud completion method based on a self-supervised conditional diffusion model according to claim 1, wherein, A target object is photographed to obtain a single-view target image. After pre-processing the target image, SAM is used for image segmentation of the target image to generate segmentation information of the target image. The obtained segmentation information is mapped to a point cloud space to obtain a preliminary incomplete point cloud.

3. The point cloud completion method based on a self-supervised conditional diffusion model according to claim 1, wherein, A denoising method based on density estimation is used to denoise the preliminary incomplete point cloud. By removing redundant points or abnormal points, point cloud data reflecting the geometric detail information of the target object is retained to obtain the denoised preliminary incomplete point cloud.

4. The point cloud completion method based on a self-supervised conditional diffusion model according to claim 1, wherein, The self-supervised conditional diffusion model adopts a denoising diffusion probability network. Through voxelization processing, the sparsity of the original point cloud is compensated for, the generalization ability of multi-level feature fusion and geometric information capture is enhanced, and Gaussian noise is gradually added and removed to generate high-quality point cloud from random noise.

5. A point cloud completion system based on a self-supervised conditional diffusion model, adopting the point cloud completion method based on the self-supervised conditional diffusion model in any one of claims 1-4, characterized in that, The method comprises the following steps: an image-point cloud acquisition module configured to acquire a target image and a target point cloud; an image-point cloud mapping module configured to segment the acquired target image, map segmentation information of the target image to a point cloud space, and obtain a preliminary incomplete point cloud; a point cloud denoising module configured to perform point cloud denoising processing on the obtained preliminary incomplete point cloud; a point cloud completion module configured to capture point cloud space geometric features according to the denoised preliminary incomplete point cloud and a self-supervised conditional diffusion model adopting a multi-scale feature fusion mechanism, inferring to complete the missing point cloud, generating complete point cloud data, and completing point cloud completion based on the self-supervised conditional diffusion model.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the point cloud completion method based on the self-supervised conditional diffusion model as described in any one of claims 1 to 4 are implemented.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, When the processor executes the program, the steps of the point cloud completion method based on the self-supervised conditional diffusion model are implemented as described in any one of claims 1 to 4.

8. A computer program product comprising software code, characterized in that, The program in the software code executes the steps of the point cloud completion method based on the self-supervised conditional diffusion model according to any one of claims 1 to 4.

Images