A point cloud recognition method in three-dimensional scenes

Abstract

The present invention discloses a point cloud recognition method in a three-dimensional scene, which uses the original point cloud data obtained by a laser radar scanner as input, utilizes the local geometric curvature change characteristics of the point cloud, performs targeted classification sampling processing on the part of the point cloud with obvious curvature change, and then divides the point cloud into meaningful parts at the semantic level, and marks the sampled feature points as predefined object categories, so as to identify different objects in the point cloud environment. The present invention is conducive to improving the segmentation accuracy of three-dimensional point clouds, and drives mobile robots to more effectively identify different objects in three-dimensional point cloud scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional point cloud data processing, and in particular to a point cloud recognition method in a three-dimensional scene. Background Art

[0002] In the field of three-dimensional scene recognition, point clouds are often used for object visualization, and this expression has become the representative of 3D data.

[0003] In the field of mobile robotics, semantic segmentation is a common method for distinguishing point clouds of different objects in the environment.

[0004] The representative feature point sampling methods currently used in point cloud semantic analysis include random sampling, uniform sampling, k-nearest neighbor sampling, iterative farthest point sampling, etc. However, in practical applications, the above sampling methods are based on the coordinate position information of the point cloud and ignore the geometric shape feature information of the point cloud object, resulting in low accuracy. Summary of the invention

[0005] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a point cloud recognition method in a three-dimensional scene with high recognition accuracy.

[0006] To achieve the above purpose, the technical solution provided by the present invention is:

[0007] A point cloud recognition method in a three-dimensional scene takes the original point cloud data obtained by a lidar scanner as input, utilizes the local geometric curvature change characteristics of the point cloud, performs targeted classification sampling processing on the parts of the point cloud with obvious curvature changes, and then divides the point cloud into meaningful parts at the semantic level, and marks the sampled feature points as predefined object categories, thereby identifying different objects in the point cloud environment.

[0008] Furthermore, the local plane fitting method is used to obtain the local geometric curvature change characteristics of the discrete point cloud, including:

[0009] 1) By traversing all individual points in the point cloud and a given statistical radius r, the neighborhood of the point cloud is set;

[0010] 2) According to the set neighborhood range, calculate the mean and standard deviation of the point cloud coordinate positions in the neighborhood, so as to obtain the covariance matrix C of the point cloud set:

[0011]

[0012] In the above formula, is the neighborhood mean of the point cloud, p i is a point cloud individual;

[0013] 3) Calculate the ascending eigenvalues ​​of the covariance matrix (λ1 ,λ 2 ,λ 3 ) and the corresponding eigenvector (ν 1 ,ν 2 ,ν 3 ), get the extension direction of the point cloud neighborhood;

[0014] 4) Based on the eigenvalues ​​corresponding to each extension direction, extract the minimum eigenvalue λ 1 As the point cloud plane normal ν 1 The corresponding extended modulus and the ratio γ of the minimum eigenvalue to the sum of the three eigenvalues ​​are calculated:

[0015]

[0016] Therefore, the ratio γ is used to represent the degree of fit between the point cloud and the plane. The smaller the ratio is, the less obvious the normal extension of the point cloud neighborhood is, so the higher the plane fit is, and the smaller the corresponding curvature is; vice versa.

[0017] Furthermore, in view of the sudden change of curvature at the intersection of two surfaces of a three-dimensional object, the intersection line of the two surfaces in the three-dimensional object is extracted by using the local curvature change, thereby describing the shape information of the object;

[0018] For a point cloud individual p i , find p i The curvature difference of all points in a sphere with a given statistical radius r and with the sphere as the center; in order to avoid the influence of noise, the mean of the three largest curvature differences in the sphere is taken to define p i The local curvature changes; the information of each point contains the curvature change C in addition to the original input x, y, z coordinates change .

[0019] Furthermore, the specific process of step S3 is as follows:

[0020] During the curvature calculation of the point cloud, the entire point cloud is divided and predicted in blocks; the division process is uniformly divided based on the given division resolution; for each divided area, a fixed number of samples is set for the area; the samples of each sub-block are merged and sent to the neural network for semantic label prediction.

[0021] Furthermore, based on the segmentation results, the individuals in the point cloud are searched and compared with the given threshold value for the curvature change of each point. According to the comparison result of the curvature change, it is determined whether the area is the boundary of two surfaces in the object, and sampling based on the curvature change is performed.

[0022] In areas with large curvature changes, sampling is performed using the maximum curvature change sampling method. The specific process is as follows:

[0023] First, randomly sample a point G in the area where the curvature changes greatly 1 , and then the curvature change value C of all points in the area change With G 1 The curvature change C change1 Make a difference and find the largest difference, and the corresponding point is recorded as G 2 , G 1 , G 2 As set G;

[0024] Next, solve the C of the remaining point cloud individuals in the area with large curvature changes point by point change With the point cloud individual C in the set G change The maximum difference of these differences, and the point cloud individual in the area with large curvature change corresponding to the smallest one is recorded as G j , merge it into the set G; update G in this way until G contains the number of samples set by the user.

[0025] Furthermore, in areas with small curvature changes, the iterative farthest point sampling method is used for downsampling, and the points sampled in the two areas are aggregated so that the downsampled input point cloud contains the position information of the individual point clouds and the geometric shape association information between local points.

[0026] Furthermore, a deep convolutional neural network is used to predict the semantic annotation of point clouds. During the prediction process, it is combined with the curvature features of the local point clouds to enhance the local feature association of the point clouds, thereby realizing the classification of various objects in the environment and the segmentation of target objects.

[0027] Compared with the existing technology, the principle and advantages of this solution are as follows:

[0028] This scheme uses the original point cloud data obtained by the lidar scanner as input, and uses the local geometric curvature change characteristics of the point cloud to perform targeted classification sampling processing on the parts of the point cloud where the curvature changes significantly, thereby dividing the point cloud into meaningful parts at the semantic level and marking the sampled feature points as predefined object categories, thereby identifying different objects in the point cloud environment.

[0029] This solution is beneficial to improving the segmentation accuracy of three-dimensional point clouds and driving mobile robots to more effectively identify different objects in three-dimensional point cloud scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the services required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 This is a flow chart of the principle of point cloud downsampling based on curvature change in a point cloud recognition method in a three-dimensional scene of the present invention. DETAILED DESCRIPTION

[0032] The present invention will be further described below in conjunction with specific embodiments:

[0033] A point cloud recognition method in a three-dimensional scene according to an embodiment of the present invention comprises the following steps:

[0034] S1, original point cloud data acquired by LiDAR scanner;

[0035] S2, calculate the curvature change of the point cloud; (such as Figure 1 As shown, the following steps S2 and S3 are the process of downsampling the point cloud based on curvature changes)

[0036] Among them, the local plane fitting method is used to obtain the local geometric curvature change characteristics of discrete point clouds, specifically including:

[0037] 1) By traversing all individual points in the point cloud and a given statistical radius r, the neighborhood of the point cloud is set;

[0038] 2) According to the set neighborhood range, calculate the mean and standard deviation of the point cloud coordinate positions in the neighborhood, so as to obtain the covariance matrix C of the point cloud set:

[0039]

[0040] In the above formula, is the neighborhood mean of the point cloud, p i is a point cloud individual;

[0041] 3) Calculate the ascending eigenvalues ​​of the covariance matrix (λ 1 ,λ 2 ,λ 3 ) and the corresponding eigenvector (ν 1 ,ν 2 ,ν 3 ), get the extension direction of the point cloud neighborhood;

[0042] 4) Based on the eigenvalues ​​corresponding to each extension direction, extract the minimum eigenvalue λ 1 As the point cloud plane normal ν 1 The corresponding extended modulus and the ratio γ of the minimum eigenvalue to the sum of the three eigenvalues ​​are calculated:

[0043]

[0044] Therefore, the ratio γ is used to represent the degree of fit between the point cloud and the plane. The smaller the ratio is, the less obvious the normal extension of the point cloud neighborhood is, so the higher the plane fit is, and the smaller the corresponding curvature is; vice versa.

[0045] Since the curvature of the intersection of two surfaces of a three-dimensional object changes suddenly, this embodiment extracts the intersection line of the two surfaces in the three-dimensional object by using the local curvature change, thereby describing the shape information of the object. i , find p i The curvature difference of all points in a sphere with a given statistical radius r and with the sphere as the center. To avoid the influence of noise, the average of the three largest curvature differences in the sphere is taken to define p i Therefore, after this step, the information of each point contains not only the original input x, y, z coordinates, but also the curvature change C change

[0046] S3, perform targeted classification sampling processing on the parts where the curvature of the point cloud changes significantly;

[0047] In the process of identifying the point cloud, the whole point cloud is divided and predicted in blocks; the division process is uniformly divided based on a given division resolution; for each divided area, a fixed number of samples is set for the area;

[0048] Based on the division results, the individuals in the point cloud are searched and compared with the given threshold value for the curvature change of each point. According to the comparison result of the curvature change, it is determined whether the area is the boundary of two surfaces in the object, and sampling based on the curvature change is performed.

[0049] Among them, in the area with large curvature change (HCV), sampling is performed through the (LCVS) maximum curvature change sampling method. The specific process is as follows:

[0050] First, randomly sample a point G in the area where the curvature changes greatly 1 , and then the curvature change value C of all points in the area change With G 1 The curvature change C change1 Make a difference and find the largest difference, and the corresponding point is recorded as G 2 , G 1 , G 2 As set G;

[0051] Next, solve the C of the remaining point cloud individuals in the area with large curvature changes point by point change With the point cloud individual C in the set G change The maximum difference of these differences, and the point cloud individual in the area with large curvature change corresponding to the smallest one is recorded as G j, merge it into the set G; update G in this way until G contains the number of samples set by the user.

[0052] In the area with small curvature change (LCV), the iterative farthest point (FPS) sampling method is used for downsampling, and the points sampled in the two areas are aggregated so that the downsampled input point cloud contains the position information of the individual point cloud and the geometric shape association information between local points.

[0053] S4. Use neural network to express the features of point cloud training and combine it with the curvature features of local point cloud to enhance the local feature association of point cloud and realize the classification of various objects in the environment and the segmentation of target objects.

[0054] In the semantic segmentation results obtained, if the semantic information indicates that the area is a "floor", "carpet" or other mobile robot navigation reachable area, then in its autonomous navigation trajectory planning process, a larger path reachability weight is given, thereby improving the navigation success rate. On the other hand, for unreachable obstacles such as "walls" and "cabinets" that are semantically identified, a smaller path planning weight is given to achieve intelligent obstacle avoidance. At the same time, the obtained object semantic information is used to record potential operation objects, such as "cups" and "remote controls" and other mobile robot operable objects, retaining their spatial information to provide support for further service operations. In short, the semantic information of each point cloud is used to provide a priori for the navigation and operation of the mobile robot, thereby achieving high-precision point cloud semantic segmentation and robust and reliable autonomous navigation of the mobile robot.

[0055] The embodiments described above are only preferred embodiments of the present invention and are not intended to limit the scope of implementation of the present invention. Therefore, all changes made according to the shape and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A point cloud recognition method in a three-dimensional scene, characterized in that: The original point cloud data obtained by the LiDAR scanner is used as input. The local geometric curvature change characteristics of the point cloud are used to perform targeted classification sampling processing on the parts of the point cloud with obvious curvature changes, and then the point cloud is divided into semantically meaningful parts. The sampled feature points are marked as predefined object categories, thereby identifying different objects in the point cloud environment. The local plane fitting method is used to obtain the local geometric curvature change characteristics of discrete point clouds, including: 1) By traversing all individual points in the point cloud and a given statistical radius r, the neighborhood of the point cloud is set; 2) According to the set neighborhood range, calculate the mean and standard deviation of the point cloud coordinate positions in the neighborhood, so as to obtain the covariance matrix C of the point cloud set: In the above formula, is the neighborhood mean of the point cloud, p i is a point cloud individual; 3) Calculate the ascending eigenvalues ​​(λ1, λ2, λ3) and corresponding eigenvectors (ν1, ν2, ν3) of the covariance matrix to obtain the extension direction of the point cloud neighborhood; 4) Based on the eigenvalues ​​corresponding to each extension direction, extract the minimum eigenvalue λ1 as the extension modulus corresponding to the point cloud plane normal ν1, and calculate the ratio γ of the minimum eigenvalue to the sum of the three eigenvalues: Therefore, the ratio γ is used to represent the degree of fit between the point cloud and the plane. The smaller the ratio is, the less obvious the normal extension of the point cloud neighborhood is, so the higher the plane fit is, and the smaller the corresponding curvature is; vice versa; In case of sudden change in curvature at the intersection of two faces of a three-dimensional object, the intersection line of the two faces in the three-dimensional object is extracted by using the local curvature change, thereby describing the shape information of the object. For a point cloud individual p i , find p i The curvature difference of all points in a sphere with a given statistical radius r and with the sphere as the center; in order to avoid the influence of noise, the mean of the three largest curvature differences in the sphere is taken to define p i The local curvature changes; the information of each point contains the curvature change C in addition to the original input x, y, z coordinates change ; During the point cloud recognition process, the entire point cloud is divided and predicted in blocks; the division process is uniformly divided based on a given division resolution; for each divided area, a fixed number of samples is set for the area; the samples of each sub-block are merged and sent to the neural network for semantic tag prediction; Based on the division results, the individuals in the point cloud are searched and compared with the given threshold value for the curvature change of each point. According to the comparison result of the curvature change, it is determined whether the area is the boundary of two surfaces in the object, and sampling based on the curvature change is performed. Among them, in the area with large curvature change, sampling is performed through the maximum curvature change sampling method. The specific process is as follows: First, randomly sample a point G1 in the area with large curvature change, and then pass the curvature change value C of all points in the area. change Curvature change C with G1 change1 Do the difference and find the largest difference. The corresponding point is recorded as G2. G1 and G2 are regarded as set G; Next, solve the C of the remaining point cloud individuals in the area with large curvature changes point by point change With the point cloud individual C in the set G change The maximum difference of these differences, and the point cloud individual in the area with large curvature change corresponding to the smallest one is recorded as G j , merge it into the set G; update G in this way until G contains the number of samples set by the user.

2. The point cloud recognition method in a three-dimensional scene according to claim 1, characterized in that: In areas with small curvature changes, the iterative farthest point sampling method is used for downsampling, and the points sampled in the two areas are aggregated so that the downsampled input point cloud contains the position information of the individual point clouds and the geometric shape association information between local points.

3. The point cloud recognition method in a three-dimensional scene according to claim 2, characterized in that: A deep convolutional neural network is used to predict the semantic annotation of point clouds. During the prediction process, it is combined with the curvature features of the local point clouds to enhance the local feature association of the point clouds, thereby achieving the classification of various objects in the environment and the segmentation of target objects.

Images