Three-dimensional point cloud data processing method for robot vision processing system

Abstract

The application provides a three-dimensional point cloud data processing method for a robot vision processing system, comprising the following steps: S1: using a farthest point sampling method to down-sample a point cloud to obtain a center point, and reducing the size of the point cloud in proportion; S2: taking the center point obtained by sampling as the center, and using a k-nearest neighbor method to search for neighbor points in the original point cloud; S3: splicing the three-dimensional coordinates of the neighbor points and the neighbor point features into a neighbor feature matrix M1; S4: performing grouped window normalization on the neighbor feature matrix M1 to obtain a feature matrix M2; S5: preprocessing the feature matrix M2 into a feature matrix M3; S6: splicing the feature matrix M3 and the center point features to obtain a feature matrix M4; and S7: using a multi-layer perception network and a max-pooling function to extract features from the feature matrix M4, and taking the extracted features as new center point features. The application solves the problems of low calculation efficiency and insufficient sampling accuracy of the existing feature enhancement scheme.

Description

Technical Field

[0001] This invention belongs to the general field of image data processing, specifically a method for processing three-dimensional point cloud data. Background Technology

[0002] With the development of modern artificial intelligence technology, robots are playing an increasingly important role in our lives. Security robots and robotic vacuum cleaners have already entered the public eye. Robots interact with the real world primarily through two types of data: images and 3D point cloud data. 3D point cloud data is a true record of object coordinates, and accurately processing this data is crucial for robots to accurately assess their surroundings.

[0003] In real-world scenarios, robots collect point cloud data using onboard millimeter-wave radar or lidar, and then analyze this data through a system to obtain action commands. Because the point cloud data directly obtained by radar equipment is quite large, preprocessing steps such as downsampling are necessary before system processing to improve processing speed.

[0004] Point cloud data processing inevitably involves downsampling techniques to reduce point cloud size, thereby improving model processing efficiency and reducing robot processor costs and server response speed. After downsampling, feature enhancement is needed to compensate for information loss. The feature enhancement process first requires obtaining neighboring points, typically using the k-nearest neighbor method and the spherical neighborhood lookup method. The spherical neighborhood lookup method yields local regions of fixed size but cannot address the issue of varying densities in different areas, leading to some regions having too many neighboring points and others too few, and resulting in slower model processing speed. While the k-nearest neighbor method yields a fixed number of neighboring points, it cannot guarantee a fixed size for the resulting local regions, posing challenges for model processing. Summary of the Invention

[0005] The purpose of this invention is to provide a point cloud data processing method with high computational efficiency, high information utilization, and few parameters, so as to overcome the shortcomings of the above-mentioned point cloud processing methods.

[0006] The technical solution adopted by this invention to achieve the above objectives is: a method for processing three-dimensional point cloud data for robot vision processing systems, comprising the following steps:

[0007] S1: Collect environmental point cloud data using radar equipment, and use the farthest point sampling method to downsample the acquired environmental point cloud data to obtain the center point;

[0008] S2: For each center point obtained by downsampling, search for neighboring points in the original point cloud using the k-nearest neighbor method with the center point as the center.

[0009] S3: Concatenate the 3D coordinates of neighboring points and the features of neighboring points into a neighboring feature matrix M1;

[0010] S4: Group window normalization is performed on the neighbor feature matrix M1 to obtain feature matrix M2, which is used to distinguish neighbor points to provide texture information or spatial information.

[0011] S5: Use the first multilayer perceptron network to preprocess the feature matrix M2 into the feature matrix M3, and complete the feature processing of neighboring points;

[0012] S6: For each center point, concatenate the feature matrix M3 with the center point features to obtain the feature matrix M4;

[0013] S7: Use the second multilayer perceptron network and the max pooling function to extract features from the feature matrix M4, and use the extracted features representing texture information or spatial information as new center point features.

[0014] The point cloud is represented as a multidimensional feature matrix containing multiple features, which are one of three-dimensional spatial coordinates, RGB features, and reflection intensity.

[0015] The grouped window normalization is implemented using a function:

[0016]

[0017]

[0018] in, The normalized result is represented by feature matrix M2, where P is the 3D point cloud, FPS represents the farthest point sampling method, and x represents the feature matrix M2. i These are the coordinates of the center point obtained from downsampling. KNN stands for k-nearest neighbor method, x i,j Let x represent the i-th center point. i The j-th neighbor point, i.e., the neighbor feature matrix M1, where ε is a constant, σ is an intermediate variable, m represents the m neighbor points among the K neighbor points whose distance from the center point is less than a threshold, d represents the feature dimension, and K is the center point x. i Count the number of neighbors;

[0019] The features contained in the first to m neighboring points constitute texture information, and the features contained in the (m+1) to K neighboring points constitute spatial information.

[0020] The preprocessing of feature matrix M2 into feature matrix M3 is achieved by the following formula:

[0021] M3 = LB(M2)

[0022] Here, LB is a multilayer perceptron network without an activation function, and the output feature dimension of the network is the same as the feature dimension of the center point.

[0023] The training of the first multilayer perceptron network includes the following steps:

[0024] The dataset used for training includes point cloud data collected by the robot and corresponding label data representing the target category. The point cloud data includes three-dimensional coordinates, as well as additional RGB features or reflection intensity.

[0025] The input to the first multilayer perceptron network is the normalized feature matrix M2, and the output is a point cloud containing the optimized center point features, i.e., feature matrix M3.

[0026] The input to the second multilayer perceptron network is the normalized feature matrix M4, and the output is a point cloud containing optimized center point features.

[0027] The feature extraction of feature matrix M4 using a second multilayer perceptron network and a max pooling function includes the following steps:

[0028] The feature matrix M4 is first processed through a second-layer perceptron network for feature learning, and then the feature is aggregated using the max pooling function to obtain features that represent texture information or spatial information.

[0029] When acquiring environmental point clouds in real time, real-time online data processing is performed through steps S1 to S7 to obtain new center point features, which are used to obtain the target category label of each point through the third multilayer perceptron network to represent the predicted category.

[0030] During training, the third multilayer perceptron network takes new center point features as input and outputs a target category label for each point to represent the predicted category.

[0031] A 3D point cloud data processing system for robot vision processing systems includes:

[0032] The point cloud processing module is used to downsample the environmental point cloud collected by radar equipment to obtain the center point using the farthest point sampling method.

[0033] The neighbor point search module is used to search for neighbor points in the original point cloud using the k-nearest neighbor method for each center point obtained by downsampling, with the center point as the center; and to concatenate the 3D coordinates and features of the neighbor points into a neighbor feature matrix M1.

[0034] The window normalization module is used to perform grouped window normalization on the neighbor feature matrix M1 to obtain the feature matrix M2, which is used to distinguish neighbor points to provide texture information or spatial information.

[0035] The feature preprocessing module is used to preprocess the feature matrix M2 into the feature matrix M3 using the first multilayer perceptron network, thus completing the feature processing of neighboring points.

[0036] The feature concatenation module is used to concatenate the feature matrix M3 with the features of the center point for each center point to obtain the feature matrix M4;

[0037] The feature extraction module is used to extract features from the feature matrix M4 using a second multilayer perceptron network and a max pooling function, and uses the extracted features representing texture information or spatial information as new center point features.

[0038] This invention has the following beneficial effects and advantages: Based on the k-nearest neighbor method, and considering the characteristic that the neighborhood obtained by the k-nearest neighbor method does not have a fixed volume, a suitable normalization method is proposed to process the volume, and a preprocessing module is used to process the features of neighbor points, thereby realizing data enhancement and information compensation after downsampling of point cloud data, improving the recognition accuracy of environmental point clouds, and has the following advantages:

[0039] 1. An improvement based on the k-nearest neighbor method, which has a very high processing speed;

[0040] 2. The window normalization module effectively solves the problem that the neighborhood obtained by the k-nearest neighbor method does not have a fixed volume, improves the downsampling effect, reduces the loss of sampling information, and is beneficial to subsequent information extraction;

[0041] 3. The grouping operation through grouping window normalization enriches the sampling information, providing a variety of effective information such as texture information and spatial information, and improving the information diversity of output features; it also solves the problem of different densities in different parts.

[0042] 4. The preprocessing module's LB effectively balances local and global information, further improving the sampling effect. Moreover, it is very concise and computationally efficient. Attached Figure Description

[0043] Figure 1 A flowchart of the three-dimensional point cloud data processing method provided by the present invention. Detailed Implementation

[0044] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0045] In practical work scenarios, robots collect point cloud data using onboard millimeter-wave radar or lidar, and then analyze this data to identify the specific category of the collected points, thereby obtaining action commands. Since the point cloud data directly obtained by radar equipment is large in size, preprocessing such as downsampling is necessary before system processing to improve system speed. To improve sampling accuracy, this invention provides a method for processing 3D point cloud data for robot vision processing systems, such as... Figure 1 As shown below, see the description for details:

[0046] S1: Obtain environmental point cloud data that needs to be processed from the radar equipment loaded on the robot. The raw point cloud data obtained is usually a multi-dimensional feature matrix, and the features include the three-dimensional spatial coordinates, RGB features and reflection intensity of the point cloud.

[0047] S2: Use the farthest point sampling method to downsample the point cloud to obtain the center point, and reduce the size of the point cloud proportionally;

[0048] The farthest-point sampling method refers to the method where, for an initial point, the point furthest from it has the strongest information difference from it, thus this point can compensate for the lack of information from the initial point to the greatest extent. By repeating this process, a point cloud with a specified number of points can be obtained.

[0049] S3: Using the sampled center point as the center, search for neighboring points in the original point cloud using the k-nearest neighbor method;

[0050] The k-nearest neighbor method calculates the distances from the center point to the other points, sorts them in ascending order of distance, and selects the K points closest to the center point as the center point's neighbors.

[0051] S4: Concatenate the 3D coordinates and features of neighboring points into a neighboring feature matrix M1; where the neighboring point features include 3D spatial coordinates, RGB features, and reflection intensity.

[0052] S5: Normalize the neighbor feature matrix M1 by grouping windows to obtain the feature matrix M2;

[0053] The grouped window normalization function is:

[0054]

[0055]

[0056] Where P is the 3D point cloud, FPS is the farthest point sampling method, and x i These are the coordinates of the center point obtained from sampling. KNN is the k-nearest neighbor method. i,j It is the center point x i The j-th neighbor point, ε = 1e-5 x is a very small constant used to ensure computational stability, where m represents the m nearest neighbor points to the center point. i,j Used to substitute M1, Used to substitute M2.

[0057] In this step, m neighboring points less than the threshold out of K neighboring points are used to form the texture information of the target, and the remaining Km neighboring points are used to form the spatial information of the target.

[0058] S6: Use a multilayer perceptron network to preprocess the feature matrix M2 into the feature matrix M3;

[0059] The preprocessing module is:

[0060] M3 = LB(M2)

[0061] Here, LB is a multilayer perceptron network without activation functions, and the output feature dimension of the network is the same as that of the center point feature.

[0062] S7: Concatenate the feature matrix M3 with the center point features to obtain the feature matrix M4;

[0063] S8: Use a multilayer perceptron network and max pooling function to extract features from the feature matrix M4, and use the extracted features that represent the original point cloud features as new center point features.

[0064] S9: Utilize a multilayer perceptron network to predict the category of a point cloud based on novel centroid features. This includes the following steps:

[0065] Three multilayer perceptron networks, S6, S8, and S9, are trained simultaneously. The training dataset includes point cloud data collected by the robot and corresponding label data. The point cloud data contains three-dimensional coordinates and additional features such as color or reflection intensity. The input of the multilayer perceptron network is the normalized feature matrix M2, and the output is the predicted category for each point.

[0066] The feature matrix M2 is input into the trained multilayer perceptron network to obtain a point cloud containing optimized center point features. The multilayer perceptron network and the max pooling function are sequentially executed in a chain; features are first learned through the multilayer perceptron function, and then the max pooling function is used for feature aggregation.

[0067] The training process of the three multilayer perceptron networks is completed through S1-S9; the input of the third multilayer perceptron network contains the point cloud of optimized center point features, and the output is the target class label for each point to represent the predicted class.

[0068] S10: The environmental point cloud is acquired in real time by the radar equipment mounted on the robot. S1-S8 are repeated to obtain the optimized point cloud features. The label of each point in the point cloud is obtained through step S9, thereby realizing the recognition of the environmental point cloud.

[0069] In summary, this invention is based on the k-nearest neighbor method, utilizes window normalization to address the variable volume problem inherent in the k-nearest neighbor method, employs a grouping strategy to enhance information richness, and utilizes a preprocessing module to balance local and spatial information. Therefore, this invention solves the problems of low computational efficiency and insufficient sampling accuracy in existing feature enhancement schemes.

Claims

1. A method for processing three-dimensional point cloud data for a robot vision processing system, the method comprising: The method comprises the following steps: S1: collecting an environment point cloud by a radar device, and using a farthest point sampling method to down-sample the obtained environment point cloud to obtain a center point; S2: for each center point obtained by down-sampling, taking the center point as the center, and using a k-nearest neighbor method to search neighbor points in the original point cloud; S3: concatenating the neighbor point three-dimensional coordinates and the neighbor point features into a neighbor feature matrix M1; S4: performing grouped window normalization on the neighbor feature matrix M1 to obtain a feature matrix M2, which is used to distinguish the neighbor points into those for providing texture information or those for providing spatial information; S5: using a first multi-layer perception network to pre-process the feature matrix M2 into a feature matrix M3, and completing feature processing of the neighbor points; S6: for each center point, concatenating the feature matrix M3 and the center point feature to obtain a feature matrix M4; S7: using a second multi-layer perception network and a max-pooling function to extract features from the feature matrix M4, and taking the extracted features representing the texture information or the spatial information as new center point features.

2. The method of claim 1, wherein, The point cloud representation is a multi-dimensional feature matrix containing multiple features, and the features are one of three-dimensional space coordinates, RGB features, and reflection intensity.

3. The method of claim 1, wherein, The grouped window normalization is realized by a function: wherein, represents the normalized result, i.e., the feature matrix M2, P is a three-dimensional point cloud, FPS represents the farthest point sampling method, x i is the coordinate of the center point obtained by downsampling, KNN represents the k-nearest neighbor method, x i,j represents the jth neighbor point of the ith center point x i , i.e., the neighbor feature matrix M1, ε is a constant, σ is an intermediate variable, m represents m neighbor points in K neighbor points with a distance less than a threshold from the center point, d represents the feature dimension, and K is the number of neighbor points of the center point x i . The first to m neighbor points contain texture information, and the (m+1) to K neighbor points contain spatial information.

4. The method of claim 1, wherein, The pre-processing of the feature matrix M2 into the feature matrix M3 is realized by the following formula: M3=LB(M2) Wherein, LB is a multi-layer perception network without an activation function, and the output feature dimension of the network is the same as the center point feature dimension.

5. The method of claim 1, wherein, The training of the first multi-layer perception network and the first multi-layer perception network comprises the following steps: The data set for training includes point cloud data collected by a robot and corresponding label data representing target categories, and the point cloud data contains three-dimensional coordinates and additional RGB features or reflection intensity; The input of the first multi-layer perception network is the normalized feature matrix M2, and the output is a point cloud containing optimized center point features, that is, the feature matrix M3; The input of the second multi-layer perception network is the normalized feature matrix M4, and the output is a point cloud containing optimized center point features.

6. The method of claim 1, wherein, The feature extraction of the feature matrix M4 using the second multi-layer perception network and the max-pooling function comprises the following steps: The feature matrix M4 is first subjected to feature learning by the second multi-layer perception network, and then subjected to feature aggregation by the max-pooling function to obtain features representing texture information or spatial information.

7. The method of claim 1, wherein, When the environment point cloud is obtained in real time, real-time online data processing is performed through steps S1-S7 to obtain new center point features for obtaining target category labels of each point through a third multi-layer perception network to represent predicted categories.

8. The method of claim 7, wherein, The input of the third multi-layer perception network during training is the new center point features, and the output is the target category label of each point to represent the predicted category.

9. A three-dimensional point cloud data processing system for a robot vision processing system, characterized by It comprises: A point cloud processing module, configured to use a farthest point sampling method to down-sample an environment point cloud collected by a radar device to obtain a center point; The neighbor point searching module is configured to search neighbor points in the original point cloud by using a k-nearest neighbor method with each center point obtained by downsampling as a center for each center point; The neighbor point three-dimensional coordinates and the neighbor point features are spliced into a neighbor feature matrix M1; The window normalization module is configured to perform grouped window normalization on the neighbor feature matrix M1 to obtain a feature matrix M2, and is configured to distinguish the neighbor points into ones for providing texture information or ones for providing spatial information; The feature preprocessing module is configured to pre-process the feature matrix M2 into a feature matrix M3 by using a first multi-layer perception network, and complete feature processing of the neighbor points; The feature splicing module is configured to splice the feature matrix M3 with a center point feature for each center point to obtain a feature matrix M4; The feature extraction module is configured to extract features of the feature matrix M4 by using a second multi-layer perception network and a max-pooling function, and take the extracted features representing texture information or spatial information as new center point features.

Images