Point cloud processing method and system based on deformation context learning

Abstract

This invention discloses a point cloud processing method and system based on deformation context learning. The method includes: Step 1: Preprocessing the input point cloud data, dividing the point cloud into multiple local point cloud blocks; Step 2: Encoding the features of the local point cloud blocks to extract point cloud features; Step 3: Constructing a deformation extraction network to extract deformation information; Step 4: Constructing a deformation transfer network to transfer the deformation information to the query input point cloud, obtaining the deformed query input point cloud features; Step 5: Mapping the deformed query input point cloud features to point cloud coordinates through a prediction network and outputting the coordinates. This invention, by constructing deformation extraction and deformation transfer mechanisms, ensures that the model maintains a consistent processing objective between the training and inference phases, and explicitly utilizes point cloud geometric structure information to achieve point cloud processing, thus providing a guarantee for improving the accuracy, stability, and generalization ability of point cloud processing.

Description

Technical Field

[0001] This invention relates to the field of 3D computer vision and point cloud data processing technology, and in particular to a point cloud processing method and system based on deformation context learning. Background Technology

[0002] With the rapid development of 3D sensing devices such as LiDAR and depth cameras, 3D point cloud data has become an important data form for describing the spatial structure of real-world scenes and is widely used in fields such as autonomous driving environmental perception, robot navigation, 3D reconstruction, intelligent manufacturing, and virtual reality. In these application scenarios, point cloud data typically needs to be processed and analyzed to achieve functions such as target recognition, scene understanding, point cloud registration, point cloud denoising, and 3D structure reconstruction.

[0003] Currently, methods for processing point cloud data mainly include those based on traditional geometric features and those based on deep learning. Traditional methods typically describe point clouds using manually designed geometric features, such as analyzing local neighborhood relationships, normal vectors, or curvature to achieve point cloud matching or classification. However, these methods usually rely on manually designed features, have poor adaptability to complex scenes, and exhibit low processing accuracy and robustness in the presence of noise, occlusion, or structural changes.

[0004] With the development of deep learning technology, more and more studies are using neural networks to learn features from point cloud data. These models automatically extract features from point cloud data by constructing deep networks, achieving certain results in tasks such as point cloud classification, segmentation, and registration. In recent years, some studies have introduced context learning techniques, and researchers have begun to explore applying this technology to point cloud processing tasks. This allows models to complete different types of point cloud processing tasks with a small number of examples, thereby achieving a general-purpose point cloud processing model.

[0005] Existing point cloud context learning methods are typically based on masking modeling mechanisms. This involves randomly masking a portion of the point cloud and training the model to reconstruct the masked region using the remaining point cloud information. However, these methods still have certain shortcomings in practical applications. First, they usually predict the target point cloud directly from the masked point cloud. The model needs to rely solely on the correlation between features to infer spatial structural relationships, lacking explicit geometric prior constraints. This can easily lead to incomplete geometric structures or inaccurate shapes in the generated point cloud results. Second, existing methods suffer from inconsistencies between the training and inference phases: during training, the model can use partial target information for prediction, while during inference, it needs to directly generate the complete point cloud structure without sufficient target information. This results in decreased model generalization ability and limited performance in complex scenes.

[0006] Prior art 1: Patent CN116453108A, entitled "A Method for Understanding and Processing 3D Point Clouds Based on Asymmetric Parallel Transformer Networks," proposes a point cloud understanding and parsing method based on asymmetric parallel Transformer networks. This method explicitly captures global features through global branches and effectively combines global and local branch features using an asymmetric parallel approach, improving the model's accuracy in classification and segmentation tasks. While this invention enhances feature extraction capabilities in point cloud classification and segmentation, it still lacks the ability to explicitly model the geometric deformation relationships of point clouds and the ability to perform cross-task point cloud processing based on contextual examples.

[0007] Prior art 2: The invention patent with publication number CN121458998A and invention title "A Convolution Method for Anisotropic Point Clouds Based on Local Geometric Adaptation" overcomes to some extent the problem of limited feature extraction capability and loss of structural information caused by structural mismatch when processing highly anisotropic data such as MBES (Multibeam Echosounder) point clouds due to isotropic kernels of traditional kernel-point convolution (KPConv) and other convolution operators. Although this prior art improves the local feature extraction capability of highly anisotropic point cloud data, compared with this invention, it still lacks the ability to explicitly model the geometric deformation relationship of point clouds and the ability to implement multi-task point cloud processing based on context examples.

[0008] Prior art 3: The invention patent CN120279179A, entitled "A 3D Point Cloud Processing Method Based on Skeleton Joint Attention Convolution," proposes a skeleton joint attention convolution method that demonstrates significant advantages in 3D point cloud processing, particularly suitable for tasks such as point cloud classification, segmentation, and scene segmentation. Its applications are wide-ranging, including but not limited to pedestrian and vehicle recognition in autonomous driving, environment modeling and target grasping in robot navigation, disease diagnosis and surgical planning in medical image analysis, and digital restoration in cultural heritage preservation—critical areas sensitive to data quality fluctuations—showing excellent application prospects and cross-domain application potential. While this prior art improves feature modeling capabilities and adaptability to data quality fluctuations in point cloud classification, segmentation, and scene segmentation tasks, compared to this invention, it still lacks the ability to explicitly model point cloud geometric deformation relationships and the ability to achieve multi-task point cloud processing based on contextual examples. Summary of the Invention

[0009] The technical problem to be solved by the embodiments of the present invention is to provide a point cloud processing method and system based on deformation context learning, so as to improve the accuracy, stability and generalization ability of point cloud processing.

[0010] To address the aforementioned technical problems, this invention proposes a point cloud processing method based on deformation context learning, comprising: Step 1: Preprocess the input point cloud data by dividing the point cloud into multiple local point cloud blocks. The point cloud data includes prompt input point cloud, prompt target point cloud, and query input point cloud. Step 2: Perform feature encoding on local point cloud blocks and extract point cloud features. Point cloud features include prompt input point cloud features, prompt target point cloud features, and query input point cloud features. Step 3: Construct a deformation extraction network to extract deformation information; Step 4: Construct a deformation transfer network to transfer deformation information to the query input point cloud, thereby obtaining the deformed query input point cloud features; Step 5: Map the deformed query input point cloud features to point cloud coordinates using a prediction network and output the coordinates.

[0011] Accordingly, embodiments of the present invention also provide a point cloud processing system based on deformation context learning, comprising: Point cloud data input and preprocessing module: preprocesses the input point cloud data, dividing the point cloud into multiple local point cloud blocks. The point cloud data includes prompt input point cloud, prompt target point cloud, and query input point cloud. Point cloud feature encoding module: Encodes features of local point cloud blocks and extracts point cloud features, including prompt input point cloud features, prompt target point cloud features, and query input point cloud features; Deformation information extraction module: Constructs a deformation extraction network to extract deformation information; Deformation information transfer module: Constructs a deformation transfer network to transfer deformation information to the query input point cloud, and obtains the deformed query input point cloud features; Point cloud prediction module: The prediction network maps the deformed query input point cloud features into point cloud coordinates and outputs them.

[0012] The beneficial effects of this invention are as follows: 1. This invention can improve the model's adaptability and generalization ability to various point cloud processing tasks.

[0013] This invention extracts task deformation features from prompt sample point clouds, enabling the model to understand the input-output relationship corresponding to the current task based on contextual examples, and transfers the task deformation features to the query point cloud. This allows the same model to be applied to various point cloud processing tasks such as point cloud reconstruction, point cloud denoising, point cloud registration, and point cloud segmentation, thereby improving the model's task adaptability and cross-task generalization ability.

[0014] 2. This invention can improve the accuracy of point cloud processing in complex scenarios.

[0015] Because point cloud data often suffers from noise, occlusion, sparse sampling, local missing data, or structural changes, existing methods are prone to inaccurate target structure recovery in complex scenarios. This invention learns the geometric transformation rules between the input point cloud and the target point cloud from the prompt samples, and uses these rules to guide the processing of the query point cloud. This enables more accurate recovery or adjustment of the point cloud structure, thereby improving the accuracy of point cloud processing.

[0016] 3. This invention can enhance the collaborative expression ability between local and global features of point clouds.

[0017] In the point cloud preprocessing stage, this invention forms local point cloud blocks through point sampling and neighborhood construction, and extracts local geometric features through a point cloud coding network. At the same time, it models the overall relationship between the prompt input point cloud and the prompt target point cloud through a deformation extraction network, and extracts task-level global deformation features, thereby enhancing the collaborative expression ability between local and global features.

[0018] 4. This invention can reduce the reliance on manually designed geometric rules or specific task structures.

[0019] This invention learns task-related geometric transformation information directly from the prompt samples through deformation extraction and deformation transfer mechanisms, without the need to design independent feature rules or processing structures for each point cloud processing task, thereby improving the versatility of the technical solution. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the point cloud processing method based on deformation context learning according to an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of the model structure of an embodiment of the present invention.

[0022] Figure 3 This is a flowchart illustrating the specific algorithm of the point cloud processing method based on deformation context learning, according to an embodiment of the present invention. Detailed Implementation

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] In this embodiment of the invention, directional indicators (such as up, down, left, right, front, back, etc.) are only used to explain the relative positional relationship and movement of each component in a specific posture (as shown in the figure). If the specific posture changes, the directional indicator will also change accordingly.

[0025] Furthermore, in this invention, descriptions involving "first," "second," etc., are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features.

[0026] The point cloud processing method based on deformation context learning in this embodiment of the invention includes steps 1 to 6, please refer to... Figure 1 and Figure 2 This invention introduces a deformation modeling mechanism to perform geometric transformations on the query point cloud under the guidance of contextual examples, thereby achieving point cloud processing tasks and improving the accuracy, stability, and generalization ability of point cloud processing.

[0027] Step 1: Preprocess the input point cloud data, dividing the point cloud into multiple local point cloud blocks. First, input the point cloud data; each point cloud can be represented as a set of 3D points: ; in: This indicates the number of points in the point cloud. Indicates the first The three-dimensional coordinates of each point. In this invention, each sample contains the following data structure: prompt input point cloud. Target point cloud prompt Query input point cloud .

[0028] A fixed number of points are sampled from the point cloud using the Farthest Point Sampling (FPS) method, and the K-Nearest Neighbors (KNN) algorithm is used to construct local neighborhoods, dividing the point cloud into multiple local point cloud blocks: ; in: This represents the number of local point cloud blocks, where each local block contains... 1 neighboring point.

[0029] Step 2: Perform feature encoding on local point cloud patches using a point cloud encoding network. Extracting point cloud features: ; ; ; in: This indicates a prompt to input point cloud features. This indicates the target point cloud features. The query input point cloud features can be represented by a point cloud feature extraction structure based on a multilayer perceptron (MLP) encoding network.

[0030] Step 3: Construct a deformation extraction network to extract deformation information.

[0031] This invention constructs a Deformation Extraction Network (DEN) to extract geometric transformation information between cue samples. First, a task token is constructed. Then, the input point cloud features, the target point cloud features, and the task token are concatenated: ; in, This indicates a feature concatenation operation. This represents the sequence of tokens input to the Transformer.

[0032] Then, task features are extracted using a Transformer network: ; in, This refers to the Transformer encoding module. This represents the task feature vector. This task feature vector is used to represent the geometric deformation relationships between cue samples.

[0033] Step 4: Construct a deformation transfer network to transfer the deformation information to the query input point cloud, and obtain the deformed query input point cloud features.

[0034] This invention constructs a Deformation Transfer Network (DTN) to transfer deformation information to the query point cloud.

[0035] First, input the query point cloud features. . Task feature vector As conditional information, it is injected into the Transformer module, and deformation transfer is achieved through feature modulation: ; in, Represents a deformation-transfer network. This represents the features of the query point cloud after deformation.

[0036] Within the network, modulation parameters are obtained through linear mapping: ; ; ; And modulate the point cloud features of the query input: ; in, Representation layer normalization, This represents the attention module. Indicates the first Layer features are used to model the geometric deformation of point clouds in the manner described above. , , It is a learnable MLP, and its parameters are initialized to zero.

[0037] Step 5: Map the deformed query input point cloud features to point cloud coordinates using a prediction network and output the coordinates.

[0038]

[0039] in, Represents a prediction network. This represents the predicted point cloud.

[0040] Step 6: Calculate the distance between the predicted point cloud and the real point cloud. During model training, this invention optimizes the parameters of the point cloud encoding network, deformation extraction network, deformation transfer network, and prediction network by calculating the distance between the predicted and real point clouds. This invention uses Chamfer Distance as the loss function. ; in, To predict point clouds, For a real point cloud, p and g represent R and g respectively. Points in a point cloud.

[0041] This invention extracts task deformation features based on prompt sample point cloud pairs, and transfers the task deformation features to the query point cloud through a deformation extraction network and a deformation transfer network to achieve point cloud processing.

[0042] Example: In this example, the present invention is applied to the point cloud denoising task. The specific algorithm can be found in [reference needed]. Figure 3 First, a set of noisy point clouds is selected as the prompt input point cloud. The denoised point cloud corresponding to it is selected as the cue target point cloud. Meanwhile, another set of noisy point clouds to be processed is selected as the query input point cloud. Among them, the prompt input point cloud and the prompt target point cloud constitute a prompt sample point cloud pair, which is used to represent the processing relationship from "noisy point cloud to denoised point cloud".

[0043] Then, input the point cloud as prompted. Target point cloud prompt And query input point cloud Point sampling and neighborhood construction are performed to obtain corresponding local point cloud blocks, and point cloud features are extracted through a point cloud encoding network. Next, the prompt input point cloud features and prompt target point cloud features are input into a deformation extraction network to extract task deformation features representing the denoising task. These task deformation features are then input into a deformation transfer network to modulate and update the query input point cloud features, causing the query input point cloud to gradually deform and migrate towards the denoised target point cloud shape. Finally, the denoised predicted point cloud result is output through a prediction head.

[0044] In this embodiment, the present invention learns the geometric transformation rules between noisy and denoised point clouds by prompting sample point clouds, and transfers these rules to the point cloud to be processed, thereby achieving noise removal and geometric structure restoration in the query input point cloud. Compared with the prior art, the present invention can better preserve the overall contour and local structural information of the point cloud, improving the processing accuracy and result stability in the point cloud denoising task.

[0045] The point cloud processing system based on deformation context learning in this invention includes a point cloud data input and preprocessing module, a point cloud feature encoding module, a deformation information extraction module, a deformation information transfer module, a point cloud prediction module, and a model training module.

[0046] Point cloud data input and preprocessing module: preprocesses the input point cloud data, dividing the point cloud into multiple local point cloud blocks. The point cloud data includes prompt input point cloud, prompt target point cloud, and query input point cloud. Point cloud feature encoding module: Encodes features of local point cloud blocks and extracts point cloud features, including prompt input point cloud features, prompt target point cloud features, and query input point cloud features; Deformation information extraction module: Constructs a deformation extraction network to extract deformation information; Deformation information transfer module: Constructs a deformation transfer network to transfer deformation information to the query input point cloud, and obtains the deformed query input point cloud features; Point cloud prediction module: The prediction network maps the deformed query input point cloud features into point cloud coordinates and outputs them.

[0047] Model training module: Calculates the distance between the predicted point cloud and the real point cloud, optimizes the parameters of the point cloud encoding network, deformation extraction network, deformation transfer network, and prediction network, using Chamfer Distance as the loss function. ; in, To predict point clouds, For a real point cloud, p and g represent R and g respectively. Points in a point cloud.

[0048] As one implementation method, the point cloud data input and preprocessing module samples a fixed number of points from the point cloud using the farthest point sampling method, and constructs local neighborhoods using the K-nearest neighbor algorithm, dividing the point cloud into multiple local point cloud blocks: ; in: This represents the number of local point cloud blocks, where each local block contains... There are n neighborhood points, j∈M, R j Let j be the j-th local point cloud block.

[0049] As one implementation method, the deformation information extraction module first constructs a task token. Then, prompt for input of point cloud features. Hints at target point cloud features And concatenate the task tokens: ; in, This indicates a feature concatenation operation. This represents the sequence of tokens input to the Transformer; Then, the task feature vector is extracted using a Transformer network: ; in, This refers to the Transformer encoding module. This represents the task feature vector, which is used to represent the geometric deformation relationship between cue samples.

[0050] As one implementation method, the deformation information transfer module first inputs the query point cloud features. , to transform the task feature vector As conditional information, it is injected into the Transformer module, and deformation transfer is achieved through feature modulation: ; in, Represents a deformation-transfer network. This represents the point cloud features of the query input after deformation; Within the deformation transfer network, modulation parameters are obtained through linear mapping: ; ; ; And modulate the point cloud features of the query input: ; in, Representation layer normalization, This represents the attention module. Indicates the first Layer features, , , It is a learnable MLP, and its parameters are initialized to zero.

[0051] The deformation information transfer module achieves geometric deformation modeling of point clouds in the manner described above.

[0052] This invention constructs a deformation extraction and deformation transfer mechanism to ensure that the model maintains a consistent processing objective during the training and inference phases, and explicitly utilizes point cloud geometric structure information to achieve point cloud processing, thus providing a guarantee for improving the accuracy, stability and generalization ability of point cloud processing.

[0053] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for point cloud processing based on deformation context learning, characterized in that, include: Step 1: Preprocess the input point cloud data by dividing the point cloud into multiple local point cloud blocks. The point cloud data includes prompt input point cloud, prompt target point cloud, and query input point cloud. Step 2: Perform feature encoding on local point cloud blocks and extract point cloud features. Point cloud features include prompt input point cloud features, prompt target point cloud features, and query input point cloud features. Step 3: Construct a deformation extraction network to extract deformation information; Step 4: Construct a deformation transfer network to transfer deformation information to the query input point cloud, thereby obtaining the deformed query input point cloud features; Step 5: Map the deformed query input point cloud features to point cloud coordinates using a prediction network and output the coordinates.

2. The point cloud processing method based on deformation context learning as described in claim 1, characterized in that, Step 5 is followed by: Step 6: Calculate the distance between the predicted point cloud and the real point cloud, and optimize the parameters of the point cloud encoding network, deformation extraction network, deformation transfer network, and prediction network. Chamfer Distance is used as the loss function. ； in, To predict point clouds, For realistic point clouds.

3. The point cloud processing method based on deformation context learning as described in claim 1, characterized in that, In step 1, a fixed number of points are sampled from the point cloud using the farthest point sampling method, and the K-nearest neighbor algorithm is used to construct local neighborhoods, dividing the point cloud into multiple local point cloud blocks: ； in: This represents the number of local point cloud blocks, where each local block contains... There are n neighborhood points, j∈M, R j Let j be the j-th local point cloud block.

4. The point cloud processing method based on deformation context learning as described in claim 1, characterized in that, In step 3, the task token is first constructed. Then, prompt for input of point cloud features. Hints at target point cloud features And concatenate the task tokens: ； in, This indicates a feature concatenation operation. This represents the sequence of tokens input to the Transformer; Then, the task feature vector is extracted using a Transformer network: ； in, This refers to the Transformer encoding module. This represents the task feature vector, which is used to represent the geometric deformation relationship between cue samples.

5. The point cloud processing method based on deformation context learning as described in claim 4, characterized in that, In step 4, first input the query point cloud features. , to transform the task feature vector As conditional information, it is injected into the Transformer module, and deformation transfer is achieved through feature modulation: ； in, Represents a deformation-transfer network. This represents the point cloud features of the query input after deformation; Within the deformation transfer network, modulation parameters are obtained through linear mapping: ； ； ； And modulate the point cloud features of the query input: ； in, , , It is a learnable MLP, whose parameters are initialized to zero; Representation layer normalization, This represents the attention module. Indicates the first The layer queries input point cloud features, and the geometric deformation modeling of the point cloud is achieved through the above method.

6. A point cloud processing system based on deformation context learning, characterized in that, include: Point cloud data input and preprocessing module: preprocesses the input point cloud data, dividing the point cloud into multiple local point cloud blocks. The point cloud data includes prompt input point cloud, prompt target point cloud, and query input point cloud. Point cloud feature encoding module: Encodes features of local point cloud blocks and extracts point cloud features, including prompt input point cloud features, prompt target point cloud features, and query input point cloud features; Deformation information extraction module: Constructs a deformation extraction network to extract deformation information; Deformation information transfer module: Constructs a deformation transfer network to transfer deformation information to the query input point cloud, and obtains the deformed query input point cloud features; Point cloud prediction module: The prediction network maps the deformed query input point cloud features into point cloud coordinates and outputs them.

7. The point cloud processing system based on deformation context learning as described in claim 6, characterized in that, Also includes: Model training module: Calculates the distance between the predicted point cloud and the real point cloud, optimizes the parameters of the point cloud encoding network, deformation extraction network, deformation transfer network, and prediction network, using Chamfer Distance as the loss function. ； in, To predict point clouds, For realistic point clouds.

8. The point cloud processing system based on deformation context learning as described in claim 6, characterized in that, The point cloud data input and preprocessing module samples a fixed number of points from the point cloud using the farthest point sampling method and constructs local neighborhoods using the K-nearest neighbor algorithm, dividing the point cloud into multiple local point cloud blocks: ； in: This represents the number of local point cloud blocks, where each local block contains... There are n neighborhood points, j∈M, R j Let j be the j-th local point cloud block.

9. The point cloud processing system based on deformation context learning as described in claim 6, characterized in that, The deformation information extraction module first constructs a task token. Then, prompt for input of point cloud features. Hints at target point cloud features And concatenate the task tokens: ； in, This indicates a feature concatenation operation. This represents the sequence of tokens input to the Transformer; Then, the task feature vector is extracted using a Transformer network: ； in, This refers to the Transformer encoding module. This represents the task feature vector, which is used to represent the geometric deformation relationship between cue samples.

10. The point cloud processing system based on deformation context learning as described in claim 9, characterized in that, The deformation information transfer module first inputs the query point cloud features. , to transform the task feature vector As conditional information, it is injected into the Transformer module, and deformation transfer is achieved through feature modulation: ； in, Represents a deformation-transfer network. This represents the point cloud features of the query input after deformation; Within the deformation transfer network, modulation parameters are obtained through linear mapping: ； ； ； And modulate the point cloud features of the query input: ； in, , , It is a learnable MLP, whose parameters are initialized to zero; Representation layer normalization, This represents the attention module. Indicates the first Layer query input point cloud features, The deformation information transfer module achieves geometric deformation modeling of point clouds in the manner described above.

Images