Method for quickly and accurately marking key points in point cloud space

Abstract

The invention discloses a method for quickly and accurately labeling key points in a point cloud space, and relates to the technical field of point cloud space labeling, comprising the following steps: S1, data preprocessing; s2, candidate points are generated; s3, key point verification; s4, key point optimization; and S5, performing subsequent application and feedback. According to the method for quickly and accurately marking the key points in the point cloud space, through data preprocessing, the calculation burden of subsequent processing is effectively reduced, key features are extracted from the preprocessed point cloud, potential key point candidate points are quickly generated, key point verification and optimization are matched, the key points are accurately and accurately marked while the accuracy and effectiveness of the key points are ensured, and the key points are accurately marked. And the positions of the key points can be finely adjusted by using an optimization algorithm, so that the marking precision and robustness are further improved. According to the method, multiple advanced technologies are combined, high automation and intelligence from candidate point generation to key point verification and optimization are realized, and rapid and accurate marking of key points in a point cloud space can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of point cloud space labeling, in particular to a method for quickly and accurately labeling key points in point cloud space. BACKGROUND

[0002] Point cloud space, in short, is a data set composed of a large number of discrete points in three-dimensional space. These points exist independently and have no fixed arrangement order, collectively describing the three-dimensional form of an object or scene. Each point contains at least three coordinate information (X, Y, Z) for accurately positioning the point in three-dimensional space. Key point labeling in point cloud space is an important task involving three-dimensional point cloud data processing, which is crucial for various applications such as autonomous driving, robot navigation, three-dimensional reconstruction, object recognition, and augmented reality.

[0003] Traditional labeling methods generally involve visually inspecting point cloud data and manually selecting key points using labeling tools. Although this manual labeling method is accurate, it is inefficient and time-consuming, especially for large-scale point cloud data. Therefore, with the continuous development of labeling technology, semi-automatic / automatic labeling has gradually replaced manual labeling as the current mainstream. Semi-automatic labeling refers to using algorithms to assist labeling, for example, using feature extraction algorithms to automatically detect potential key points, and then manually verifying and adjusting. Automatic labeling, supported by deep learning or machine learning technology, realizes automatic detection and labeling of key points, especially suitable for large-scale data sets.

[0004] However, in actual operation, existing automatic labeling methods generally have low precision, making the application scenarios of automatic labeling more limited and only applicable to labeling scenarios with low precision requirements. SUMMARY

[0005] The present application aims to provide a method for quickly and accurately labeling key points in point cloud space to solve the problems raised in the background.

[0006] To achieve the above-mentioned purpose, the present application provides the following technical solution: a method for quickly and accurately labeling key points in point cloud space, comprising the following steps: S1, data preprocessing: preprocessing the original point cloud data, including denoising, downsampling, and registration, to improve the efficiency and accuracy of subsequent labeling; S2, candidate point generation: extracting key features from the preprocessed point cloud and generating a series of key point candidates based on the extracted features; S3, key point verification: verifying the generated candidate points, removing invalid candidate points, and selecting true key points; S4, Key point optimization: using optimization algorithms to adjust, correct or refine the verified key points to ensure the accuracy and representativeness of the key points; S5, Subsequent application and feedback: applying the labeled key points to subsequent tasks (such as three-dimensional reconstruction, object recognition, etc.), and feedback and optimization according to the application effect.

[0007] Further, in step S1, the denoising and downsampling operations are applied to all point cloud data, and the registration operation is executed only when the point cloud data comes from multiple perspectives or devices; Denoising: using filtering algorithms to remove noise points in the point cloud, including statistical filtering (judging whether a point is a noise point according to the neighborhood point statistical information (such as average distance, standard deviation, etc.) of each point in the point cloud, and removing these points), radius filtering (setting a search radius for each point, if the number of neighborhood points within this radius is less than a certain threshold, the point is considered a noise point and removed), conditional filtering (setting conditions according to the attributes of the points (such as coordinate range, color, intensity, etc.), removing points that do not meet the conditions); Downsampling: reducing the number of points in the point cloud by voxel grid downsampling or random downsampling, etc., to reduce the density of the point cloud, thereby reducing the computational complexity, wherein voxel grid downsampling divides the point cloud space into a series of small three-dimensional voxel grids, then selects a representative point (such as the center of the grid, the point closest to the center in the grid, etc.) from each grid to reduce the number of points in the point cloud, while random downsampling randomly selects a part of the points in the point cloud as representative points; Registration: for point cloud data from multiple perspectives or devices, using ICP algorithm, NDT algorithm or feature matching algorithm to align them to a unified coordinate system to generate a unified point cloud model.

[0008] Further, the specific operations of the ICP algorithm, NDT algorithm or feature matching algorithm are as follows: ICP algorithm: finding the nearest point pair between two point clouds by iteration, and calculating the transformation matrix to align one point cloud to another; NDT algorithm: dividing the point cloud into small cells, and representing the points in each cell with a normal distribution, then optimizing the objective function to align two point clouds; Feature matching algorithm: extracting feature points (such as corner points, edge points, etc.) from the point cloud, then matching and aligning based on these feature points.

[0009] Further, in step S2, the key features include geometric, statistical or deep learning features, and the feature extraction is as follows: Geometric features: calculating the normal vector, curvature, surface variation and other geometric properties of each point to reflect the local shape and structure of the point cloud; Statistical features: Calculate statistical information within the local neighborhood of each point, such as point density, distance distribution, direction distribution, etc., to identify abnormal points, dense areas or edges in the point cloud; Deep learning features: Use pre-trained deep learning models (such as PointNet, PointNet++, DGCNN, etc.) to extract high-level feature representations of point clouds to capture global and local information of point clouds, suitable for complex point cloud analysis tasks.

[0010] Further, the step S2, the candidate point screening method specifically includes: Threshold method: Set a threshold according to one or more attributes of the feature (such as curvature, density, etc.), and select points that meet certain conditions as candidate points, for example, points with larger curvature, higher density or significant deep learning feature values can be selected; Clustering method: Cluster analysis is performed on the extracted features, similar points are classified into a class, and the cluster center, cluster boundary or specific points within the cluster are selected as candidate points, this method is suitable for identifying structural features in point clouds, such as planes, spheres, cylinders, etc. Deep learning prediction: Use a trained deep learning model to directly predict key point positions, the model is trained based on the extracted features and outputs the probability or score of each point being a key point, and selects points with higher scores as candidate points.

[0011] Further, in the step S3, the candidate point verification adopts an automatic verification method, and the specific operation is: calculating the feature difference between the candidate point and its neighborhood points for verification, if the difference is too large, it is determined that the candidate point is not a true key point, or verifying the candidate point according to the geometric characteristics of the point cloud (such as planarity, smoothness, continuity, etc.), if the point does not meet the geometric constraints, it is determined that it is not a key point.

[0012] Further, in the step S4, the optimization algorithm includes an iterative closest point algorithm and a non-rigid registration algorithm, the iterative closest point algorithm finds the closest point pair between two point clouds (original point cloud and key point set) through iteration, and calculates a transformation matrix to make the key point set more accurately aligned to the original point cloud, the non-rigid registration algorithm is different from ICP, which allows the point cloud to deform during registration to better adapt to complex shape changes.

[0013] Further, the optimization step of the iterative closest point algorithm: 1) Initialization: Set the initial transformation matrix (usually a unit matrix) and the upper limit of the number of iterations; 2) Closest point pair matching: Find the closest point pair in the original point cloud and the key point set, specifically by calculating the nearest neighbor of each key point in the original point cloud; 3) Compute transformation matrix: based on the matched point pairs, compute the rigid body transformation (rotation and translation) required to align the keypoint set to the original point cloud; 4) Apply transformation: apply the computed transformation matrix to the keypoint set; 5) Convergence check: evaluate the degree of alignment between the transformed keypoint set and the original point cloud (e.g., by computing the mean squared error), if the predetermined convergence condition is met (e.g., the error is less than a threshold or the iteration count reaches an upper limit), stop the iteration, otherwise, return to step 2) and continue the iteration.

[0014] Further, the optimization step of the non-rigid registration algorithm: 1) Initialization: set the initial deformation field (e.g., mesh-based deformation or point-based deformation) and the iteration count upper limit; 2) Nearest point pair matching: similar to ICP, find the nearest point pairs between the original point cloud and the deformed keypoint set; 3) Compute deformation: based on the matched point pairs, compute the deformation required to better align the keypoint set to the original point cloud, specifically by solving an optimization problem to minimize the alignment error; 4) Apply deformation: apply the computed deformation to the keypoint set; 5) Convergence check: evaluate the degree of alignment between the deformed keypoint set and the original point cloud, if the convergence condition is met, stop the iteration, otherwise, return to step 2) and continue the iteration; 6) Refine deformation: during the iteration process, gradually refine the resolution of the deformation field or adjust the parameters of the optimization problem to obtain a more refined alignment result.

[0015] Further, in step S5, the keypoint application specifically includes: Three-dimensional reconstruction: use the labeled keypoint as a feature point or control point to perform three-dimensional surface reconstruction or mesh generation; Object recognition: input the labeled keypoint into the object recognition model as a feature to improve the accuracy and robustness of recognition; Positioning and navigation: use the labeled keypoint as a landmark or feature point in the SLAM (Simultaneous Localization and Mapping) system for precise positioning and navigation.

[0016] The present application provides a method for fast and accurate keypoint labeling in point cloud space, which has the following beneficial effects: This invention effectively reduces the computational burden of subsequent processing through data preprocessing, namely denoising, downsampling, and registration. This allows for the extraction of key features from the preprocessed point cloud, rapidly generating potential keypoint candidate points. Combined with keypoint verification and optimization, this method ensures the accuracy and effectiveness of keypoints while enabling fine-tuning of keypoint positions using optimization algorithms, further improving the accuracy and robustness of annotation. This method integrates multiple advanced technologies, achieving a high degree of automation and intelligence from candidate point generation to keypoint verification and optimization. It reduces reliance on manual operations, improves annotation consistency and repeatability, and enables rapid and accurate keypoint annotation in point cloud space.

[0017] In addition, this method also has an application and feedback mechanism. By applying the labeled key points to actual tasks and providing feedback and optimization based on the application results, this mechanism forms a closed-loop labeling process that can continuously iterate and improve the labeling quality, ensuring that the labeling results always meet the actual application requirements. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the steps of a method for rapid and accurate annotation of key points in point cloud space according to the present invention. Detailed Implementation

[0019] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and should not be construed as limiting the scope of the invention.

[0020] like Figure 1 As shown, a method for fast and accurate annotation of key points in point cloud space includes the following steps: S1. Data Preprocessing: Preprocessing the raw point cloud data, including denoising, downsampling, and registration, to improve the efficiency and accuracy of subsequent annotation. Denoising and downsampling operations are applied to all point cloud data, while registration is performed only if the point cloud data comes from multiple viewpoints or devices. Denoising: Noise removal is achieved by using filtering algorithms to remove noisy points from the point cloud. These algorithms include statistical filtering (which determines whether a point is a noise point based on the statistical information of its neighboring points (such as average distance, standard deviation, etc.) and removes these points), radius filtering (which sets a search radius for each point; if the number of neighboring points within this radius is less than a certain threshold, the point is considered a noise point and is removed), and conditional filtering (which sets conditions based on the point's attributes (such as coordinate range, color, intensity, etc.) and removes points that do not meet the conditions). Downsampling: Methods such as voxel grid downsampling or random downsampling are used to reduce the number of points in the point cloud and reduce the density of the point cloud, thereby reducing computational complexity. Voxel grid downsampling divides the point cloud space into a series of small three-dimensional voxel grids, and then selects a representative point from each grid (such as the grid center, the point closest to the center in the grid, etc.) to reduce the number of points in the point cloud. Random downsampling, on the other hand, randomly selects a portion of the points in the point cloud as representative points. Registration: For point cloud data from multiple viewpoints or devices, the ICP algorithm, NDT algorithm, or feature matching algorithm are used to align them into a unified coordinate system to generate a unified point cloud model. In this embodiment, the specific operations of the ICP algorithm, NDT algorithm, or feature matching algorithm are as follows: The ICP algorithm iteratively finds the nearest point pair between two point clouds and calculates the transformation matrix to align one point cloud to the other. NDT algorithm: Divide the point cloud into small cells, and represent the points in each cell using a normal distribution. Then, align the two point clouds by optimizing the objective function. Feature matching algorithm: Extract feature points (such as corner points, edge points, etc.) from the point cloud, and then perform matching and alignment based on these feature points.

[0021] S2. Candidate Point Generation: Extract key features from the preprocessed point cloud and generate a series of key point candidate points based on these features. Key features include geometric, statistical, or deep learning features, and the feature extraction process is as follows: Geometric features: Calculate the geometric properties of each point, such as normal vector, curvature, and surface changes, to reflect the local shape and structure of the point cloud; Statistical features: Calculate statistical information in the local neighborhood of each point, such as point density, distance distribution, and orientation distribution, to identify outliers, dense regions, or edges in the point cloud; Deep learning features: High-level feature representations of point clouds are extracted using pre-trained deep learning models (such as PointNet, PointNet++, DGCNN, etc.) to capture global and local information of point clouds, which is suitable for complex point cloud analysis tasks. 5. In step S2, the candidate point screening method specifically includes: Thresholding method: Set a threshold based on one or more attributes of the feature (such as curvature, density, etc.) to filter out points that meet specific conditions as candidate points. For example, you can select points with large curvature, high density or significant deep learning feature values. Clustering method: The extracted features are subjected to cluster analysis, similar points are grouped into one class, and specific points within the cluster, such as the cluster center, cluster boundary, or cluster, are selected as candidate points. This method is suitable for identifying structural features in point clouds, such as planes, spheres, cylinders, etc. Deep learning prediction: The key point location is directly predicted using a trained deep learning model. The model is trained based on the extracted features and outputs the probability or score that each point is a key point. Points with higher scores are selected as candidate points.

[0022] S3. Keypoint Verification: Verify the generated candidate points, remove invalid candidate points, and select true keypoints. Candidate point verification adopts an automatic verification method. Specifically, it calculates the feature differences between the candidate point and its neighboring points for verification. If the difference is too large, the candidate point is determined not to be a true keypoint. Alternatively, it verifies the candidate points based on the geometric characteristics of the point cloud (such as planarity, smoothness, continuity, etc.). Points that do not meet the geometric constraints are determined not to be keypoints.

[0023] S4. Keypoint Optimization: Optimization algorithms are used to adjust, correct, or refine the verified keypoints to ensure their accuracy and representativeness. In this embodiment, the optimization algorithms include the iterative nearest point algorithm and the non-rigid registration algorithm. The iterative nearest point algorithm iteratively finds the closest point pair between two point clouds (the original point cloud and the keypoint set) and calculates the transformation matrix to more accurately align the keypoint set to the original point cloud. The non-rigid registration algorithm, unlike ICP, allows the point cloud to deform during the registration process to better adapt to complex shape changes.

[0024] The optimization steps of the iterative nearest point algorithm are as follows: 1) Initialization: Set the initial transformation matrix (usually the identity matrix) and the upper limit of the number of iterations; 2) Nearest point pair matching: Find the nearest point pair in the original point cloud and the key point set, specifically by calculating the nearest neighbor of each key point in the original point cloud; 3) Calculate the transformation matrix: Based on the matched point pairs, calculate the rigid body transformations (rotation and translation) required to align the keypoint set to the original point cloud. 4) Apply the transformation: Apply the calculated transformation matrix to the keypoint set; 5) Convergence check: Evaluate the alignment between the transformed keypoint set and the original point cloud (e.g., by calculating the mean square error). If the predetermined convergence condition is met (e.g., the error is less than the threshold or the upper limit of the number of iterations is reached), stop the iteration; otherwise, return to step 2) to continue the iteration.

[0025] Optimization steps for non-rigid registration algorithms: 1) Initialization: Set the initial deformation field (such as mesh-based deformation or point-based deformation) and the upper limit of the number of iterations; 2) Nearest Point Pair Matching: Similar to ICP, find the nearest point pairs between the original point cloud and the deformed keypoint set; 3) Calculate deformation: Based on the matched point pairs, calculate the deformation required to better align the key point set to the original point cloud, specifically by solving an optimization problem to minimize the alignment error; 4) Apply deformation: Apply the calculated deformation to the key point set; 5) Convergence check: Evaluate the alignment between the deformed keypoint set and the original point cloud. If the convergence condition is met, stop the iteration; otherwise, return to step 2) to continue the iteration. 6) Refine the deformation: During the iteration process, gradually refine the resolution of the deformation field or adjust the parameters of the optimization problem to obtain more refined alignment results.

[0026] S5. Subsequent Application and Feedback: Apply the marked key points to subsequent tasks, and provide feedback and optimization based on the application results. Specific key point applications include: 3D reconstruction: using labeled key points as feature points or control points to reconstruct 3D surfaces or generate meshes; Object recognition: Annotated key points are input as features into the object recognition model to improve the accuracy and robustness of recognition; Location and navigation: Using marked key points as landmarks or feature points, precise location and navigation are performed in a SLAM (Simultaneous Localization and Mapping) system.

[0027] The embodiments of the present invention are given for illustrative and descriptive purposes only, and are not intended to be exhaustive or to limit the invention to the forms disclosed. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to better illustrate the principles and practical application of the invention, and to enable those skilled in the art to understand the invention and to design various embodiments with various modifications suitable for a particular purpose.

Claims

1. A method for rapid and accurate annotation of key points in point cloud space, characterized in that, Includes the following steps: S1. Data preprocessing: Preprocess the raw point cloud data, including denoising, downsampling, and registration; S2. Candidate point generation: Extract key features from the preprocessed point cloud and generate a series of key point candidate points based on the extracted features; S3. Key point verification: Verify the generated candidate points, remove invalid candidate points, and select the real key points; S4. Key point optimization: Use optimization algorithms to adjust, correct or refine the verified key points.

2. The method for rapid and accurate annotation of key points in point cloud space according to claim 1, characterized in that, In step S1, the denoising and downsampling operations are applied to all point cloud data, and the registration operation is performed only if the point cloud data comes from multiple views or devices. Denoising: Noise points in the point cloud are removed using filtering algorithms, including statistical filtering, radius filtering, and conditional filtering. Downsampling: Reduce the number of points in the point cloud by using voxel grid downsampling or random downsampling methods, thereby reducing the density of the point cloud; Registration: For point cloud data from multiple viewpoints or devices, the ICP algorithm, NDT algorithm, or feature matching algorithm are used to align them into a unified coordinate system to generate a unified point cloud model.

3. The method for rapid and accurate annotation of key points in point cloud space according to claim 2, characterized in that, The specific operations of the ICP algorithm, NDT algorithm, or feature matching algorithm are as follows: The ICP algorithm iteratively finds the nearest point pair between two point clouds and calculates the transformation matrix to align one point cloud to the other. NDT algorithm: Divide the point cloud into small cells, and represent the points in each cell using a normal distribution. Then, align the two point clouds by optimizing the objective function. Feature matching algorithm: Extract feature points from point cloud, and then perform matching and alignment based on these feature points.

4. The method for rapid and accurate annotation of key points in point cloud space according to claim 1, characterized in that, In step S2, the key features include geometric, statistical, or deep learning features, and the feature extraction is as follows: Geometric features: Calculate the geometric properties of each point to reflect the local shape and structure of the point cloud; Statistical features: Calculate statistical information within the local neighborhood of each point to identify outliers, dense regions, or edges in the point cloud; Deep learning features: High-level feature representations of point clouds are extracted using pre-trained deep learning models to capture global and local information of point clouds, suitable for complex point cloud analysis tasks.

5. The method for rapid and accurate annotation of key points in point cloud space according to claim 1, characterized in that, In step S2, the candidate point screening method specifically includes: Thresholding method: Set a threshold based on one or more attributes of the feature to filter out points that meet specific conditions as candidate points; Clustering method: Perform cluster analysis on the extracted features, group similar points into one class, and select cluster centers, cluster boundaries, or specific points within the cluster as candidate points; Deep learning prediction: The key point location is directly predicted using a trained deep learning model. The model is trained based on the extracted features and outputs the probability or score that each point is a key point. Points with higher scores are selected as candidate points.

6. The method for rapid and accurate annotation of key points in point cloud space according to claim 1, characterized in that, In step S3, the candidate point verification adopts an automatic verification method. Specifically, the feature differences between the candidate point and its neighboring points are calculated for verification. If the difference is too large, the candidate point is determined not to be a real key point. Alternatively, the candidate point is verified based on the geometric characteristics of the point cloud. Points that do not meet the geometric constraints are determined not to be key points.

7. The method for rapid and accurate annotation of key points in point cloud space according to claim 1, characterized in that, In step S4, the optimization algorithm includes the iterative nearest point algorithm and the non-rigid registration algorithm. The iterative nearest point algorithm finds the nearest point pair between two point clouds iteratively and calculates the transformation matrix to make the key point set more accurately aligned to the original point cloud. The non-rigid registration algorithm is different from ICP. It allows the point cloud to deform during the registration process to better adapt to complex shape changes.

8. The method for rapid and accurate annotation of key points in point cloud space according to claim 7, characterized in that, The optimization steps of the iterative nearest point algorithm are as follows: 1) Initialization: Set the initial transformation matrix and the upper limit of the number of iterations; 2) Nearest point pair matching: Find the nearest point pair in the original point cloud and the key point set, specifically by calculating the nearest neighbor of each key point in the original point cloud; 3) Calculate the transformation matrix: Based on the matched point pairs, calculate the rigid body transformation required to align the keypoint set to the original point cloud; 4) Apply the transformation: Apply the calculated transformation matrix to the keypoint set; 5) Convergence check: Evaluate the alignment between the transformed keypoint set and the original point cloud. If the predetermined convergence condition is met, stop the iteration; otherwise, return to step 2) to continue the iteration.

9. The method for rapid and accurate annotation of key points in point cloud space according to claim 7, characterized in that, The optimization steps of the non-rigid registration algorithm are as follows: 1) Initialization: Set the initial deformation field and the upper limit of the number of iterations; 2) Nearest Point Pair Matching: Similar to ICP, find the nearest point pairs between the original point cloud and the deformed keypoint set; 3) Calculate deformation: Based on the matched point pairs, calculate the deformation required to better align the key point set to the original point cloud, specifically by solving an optimization problem to minimize the alignment error; 4) Apply deformation: Apply the calculated deformation to the key point set; 5) Convergence check: Evaluate the alignment between the deformed keypoint set and the original point cloud. If the convergence condition is met, stop the iteration; otherwise, return to step 2) to continue the iteration. 6) Refine the deformation: During the iteration process, gradually refine the resolution of the deformation field or adjust the parameters of the optimization problem to obtain more refined alignment results.

10. The method for rapid and accurate annotation of key points in point cloud space according to claim 1, characterized in that, The method further includes the following steps: S5. Subsequent Application and Feedback: Apply the labeled key points to subsequent tasks, and provide feedback and optimization based on the application results; the application of key points specifically includes: 3D reconstruction: using labeled key points as feature points or control points to reconstruct 3D surfaces or generate meshes; Object recognition: Annotated key points are input as features into the object recognition model to improve the accuracy and robustness of recognition; Location and navigation: Using marked key points as landmarks or feature points, precise location and navigation are performed in the SLAM system.

Images