Point cloud data processing method and device, computer device and storage medium

Abstract

This application relates to a point cloud data processing method, apparatus, computer equipment, and storage medium. The method includes: acquiring a first frame and a second frame of point cloud data collected by a lidar system; extracting feature points from the first and second frame point clouds respectively; wherein the feature points include reflection feature points and geometric feature points; associating the reflection feature points of the first frame point cloud with the reflection feature points of the second frame point cloud to obtain a first association relationship; associating the geometric feature points of the first frame point cloud with the geometric feature points of the second frame point cloud to obtain a second association relationship; and registering the first and second frame point clouds according to the first and second association relationships to obtain a pose transformation matrix for the first and second frame point clouds. This method can improve the accuracy of lidar odometry positioning.

Description

Technical Field

[0001] This application relates to the field of lidar technology, and in particular to a point cloud data processing method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the development of LiDAR technology, LiDAR odometry, a positioning technology based on LiDAR, has been widely used in the field of autonomous driving.

[0003] Traditional methods use only a single geometric feature as the information source for point cloud data processing. However, autonomous driving scenarios are complex and varied. During autonomous driving, vehicles may encounter scenarios with unclear geometric features. In such scenarios, it is difficult to accurately locate based on a single geometric feature of the collected point cloud. As a result, LiDAR odometry positioning is inaccurate and has poor robustness. Summary of the Invention

[0004] Therefore, it is necessary to provide a point cloud data processing method, apparatus, computer equipment, and storage medium that can improve the positioning accuracy of lidar odometers in response to the above-mentioned technical problems.

[0005] A point cloud data processing method, the method comprising:

[0006] Acquire the first and second frame point clouds collected by the lidar;

[0007] Feature points are extracted from the first frame point cloud and the second frame point cloud respectively; the feature points include reflection feature points and geometric feature points.

[0008] The reflection feature points of the first frame point cloud are associated with the reflection feature points of the second frame point cloud to obtain the first association relationship;

[0009] The geometric feature points of the first frame point cloud are associated with the geometric feature points of the second frame point cloud to obtain the second association relationship;

[0010] Based on the first and second association relationships, the point clouds of the first and second frames are registered to obtain the pose transformation matrices of the point clouds of the first and second frames.

[0011] In one embodiment, reflection feature points are extracted from the first frame point cloud and the second frame point cloud, respectively, including:

[0012] Obtain the reflection intensity information of each point in the first frame point cloud and the second frame point cloud;

[0013] Based on the reflection intensity information, points with a reflection intensity greater than a preset threshold are extracted as reflection feature points.

[0014] In one embodiment, associating the reflection feature points of the first frame point cloud with the reflection feature points of the second frame point cloud includes:

[0015] The reflection feature points of the first frame point cloud and the reflection feature points of the second frame point cloud are clustered to obtain multiple reflection feature cluster objects.

[0016] Extract the centroid of each clustered object based on its reflection features as the reflection center point;

[0017] Associate the reflection center point of the first frame point cloud with the reflection center point of the second frame point cloud.

[0018] In one embodiment, associating the reflection center point of the first frame point cloud with the reflection center point of the second frame point cloud includes:

[0019] Transform any reflection center point of the first frame point cloud to the coordinate system of the second frame point cloud to obtain the reflection transformation point;

[0020] Select the point closest to the reflection conversion point from multiple reflection center points in the second frame point cloud as the corresponding reflection point;

[0021] Associate any reflection center point of the first frame point cloud with its corresponding reflection point.

[0022] In one embodiment, the geometric feature points include edge feature points, and associating the geometric feature points of the first frame point cloud with the geometric feature points of the second frame point cloud includes:

[0023] Transform any edge feature point of the first frame point cloud into the coordinate system of the second frame point cloud to obtain the edge feature transformation point;

[0024] Among the multiple edge feature points in the second frame point cloud, the point closest to the edge feature transformation point is selected as the corresponding point of the first edge feature;

[0025] Among the multiple edge feature points of the second frame point cloud, the point on the scan line adjacent to the scan line where the first edge feature point is located is selected as the second edge feature point.

[0026] Associate any edge feature point of the first frame point cloud with its corresponding first edge feature point and its corresponding second edge feature point.

[0027] In one embodiment, the geometric feature points include planar feature points, and associating the geometric feature points of the first frame point cloud with the geometric feature points of the second frame point cloud includes:

[0028] Transform any planar feature point in the first frame of the point cloud into the coordinate system of the second frame of the point cloud to obtain the planar feature transformation point;

[0029] In the second frame point cloud, the point closest to the planar feature transformation point is selected as the first planar feature corresponding point.

[0030] Among the multiple planar feature points in the second frame point cloud, the point on the scan line adjacent to the scan line where the first planar feature point is located is selected as the second planar feature point.

[0031] Among the multiple planar feature points in the second frame point cloud, the point on the same scan line as the first planar feature point is selected as the third planar feature point.

[0032] Associate any planar feature point of the first frame point cloud with its corresponding first planar feature point, second planar feature point, and third planar feature point.

[0033] In one embodiment, registration of the first frame point cloud and the second frame point cloud is performed based on a first association relationship and a second association relationship, including:

[0034] The error value between the reflection feature points of the first frame point cloud and the reflection feature points of the second frame point cloud in each associated group is calculated based on the first association relationship.

[0035] The error value between the geometric feature points of the first frame point cloud and the geometric feature points of the second frame point cloud in each associated group is calculated based on the second association relationship.

[0036] An error optimization function is constructed based on the calculated error value, and point cloud registration is performed based on the error optimization function.

[0037] A point cloud data processing device, the device comprising:

[0038] The point cloud acquisition module is used to acquire the first and second frame point clouds collected by the lidar.

[0039] The feature extraction module is used to extract feature points from the first frame point cloud and the second frame point cloud, respectively; wherein the feature points include reflection feature points and geometric feature points;

[0040] The first feature association module is used to associate the reflection feature points of the first frame point cloud with the reflection feature points of the second frame point cloud to obtain the first association relationship;

[0041] The second feature association module is used to associate the geometric feature points of the first frame point cloud with the geometric feature points of the second frame point cloud to obtain the second association relationship.

[0042] The point cloud registration module is used to register the first frame point cloud and the second frame point cloud according to the first association relationship and the second association relationship, so as to obtain the pose transformation matrix of the first frame point cloud and the second frame point cloud.

[0043] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the point cloud data processing methods described above.

[0044] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the point cloud data processing methods described above.

[0045] The aforementioned point cloud data processing method, apparatus, computer equipment, and storage medium extract reflection feature points and geometric feature points from the first and second frames of point clouds respectively, and associate these feature points with each frame. Point cloud registration is then performed based on the association relationships between the reflection feature points and geometric feature points in the two frames. Using this method, the point cloud registration process relies not only on the characteristics of the associated geometric feature points but also on the characteristics of the reflection feature points. Therefore, even in scenarios where geometric features are not obvious, the accuracy of the lidar odometry can still be ensured, improving the robustness of the lidar odometry. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating a point cloud data processing method in one embodiment;

[0047] Figure 2 This is a flowchart illustrating the steps of associating the reflection feature points of the first frame point cloud with the reflection feature points of the second frame point cloud in one embodiment.

[0048] Figure 3 This is a schematic diagram illustrating the clustering process of reflection feature points in one embodiment;

[0049] Figure 4 This is a flowchart illustrating the steps of associating edge feature points of a first frame point cloud with edge feature points of a second frame point cloud in one embodiment.

[0050] Figure 5 This is a schematic diagram illustrating the association of edge feature points in one embodiment;

[0051] Figure 6 This is a flowchart illustrating the steps of associating planar feature points of a first frame point cloud with planar feature points of a second frame point cloud in one embodiment.

[0052] Figure 7 This is a schematic diagram illustrating the association of planar feature points in one embodiment;

[0053] Figure 8 This is a structural block diagram of a point cloud data processing device in one embodiment;

[0054] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0056] The point cloud data processing method provided in this application can be implemented using computer equipment, which may include terminal equipment and / or server equipment. The server can be a standalone server or a server cluster consisting of multiple servers. The terminal can be, but is not limited to, various personal computers, vehicle-mounted terminals, laptops, smartphones, tablets, and portable wearable devices.

[0057] In one embodiment, such as Figure 1 As shown, a point cloud data processing method is provided. Taking the application of this method to a computer device as an example, the method includes the following steps:

[0058] Step S102: Acquire the first and second frame point clouds collected by the lidar.

[0059] LiDAR, also known as optical radar, measures the propagation distance between a sensor transmitter and a target object, analyzing information such as the magnitude of reflected energy, amplitude, frequency, and phase of the reflected spectrum, thus revealing the three-dimensional structural information of the target object. Each scanning cycle of a LiDAR system constitutes a frame. The first frame of point cloud data refers to the point cloud data contained in a selected frame obtained by the LiDAR scanning the target object. The second frame of point cloud data refers to the point cloud data contained in a different frame obtained by the LiDAR scanning the target object. For example, in the field of autonomous driving, the target object can be an autonomous vehicle or a scene containing an autonomous vehicle.

[0060] It is worth noting here that terms such as “first” and “second” in this application are used only to distinguish one entity or operation from another, and do not necessarily require or imply any actual relationship or sequence between these entities or operations.

[0061] In this step, the computer device receives point cloud data obtained by scanning the target object with a LiDAR within a certain time period. From the multiple frames of the LiDAR scan, it selects two different frames as the first and second frames, respectively. The point cloud data of the first frame is then acquired as the first frame point cloud, and the point cloud data of the second frame is acquired as the second frame point cloud. The first and second frames can be adjacent or non-adjacent. The time interval between the first and second frames can be customized, for example, set to 0.5 seconds.

[0062] Step S104: Extract feature points from the first frame point cloud and the second frame point cloud respectively; wherein, the feature points include reflection feature points and geometric feature points.

[0063] Feature points include reflection feature points and geometric feature points. Reflection feature points are those selected based on their reflection intensity, reflecting information about the object's reflection intensity. Geometric feature points are those selected based on their geometric smoothness, reflecting information about the object's geometry; for example, geometric feature points may include edge feature points and / or planar feature points.

[0064] Specifically, the computer device can use feature extraction algorithms to extract edge feature points and / or planar feature points of the first frame point cloud based on the geometric smoothness of the first frame point cloud, and extract edge feature points and / or planar feature points of the second frame point cloud based on the geometric smoothness of the second frame point cloud. Furthermore, it can filter the reflection feature points of the first frame point cloud based on the magnitude of the reflection intensity of each point in the first frame point cloud, and filter the reflection feature points of the second frame point cloud based on the magnitude of the reflection intensity of each point in the second frame point cloud.

[0065] In one embodiment, reflection feature points are extracted from a first frame point cloud and a second frame point cloud, respectively, including: acquiring reflection intensity information of each point in the first frame point cloud and the second frame point cloud; and extracting points with reflection intensity greater than a preset threshold as reflection feature points based on the reflection intensity information. In this embodiment, by setting a threshold for reflection intensity, high-reflection points with reflection intensity higher than a certain value can be extracted as reflection feature points, thereby improving the accuracy of reflection feature point extraction.

[0066] Step S106: Associate the reflection feature points of the first frame point cloud with the reflection feature points of the second frame point cloud to obtain the first association relationship.

[0067] Specifically, for each reflection feature point P1 in the first frame point cloud, P1 is transformed into the coordinate system of the second frame point cloud. Assuming the pose of point P1 in the first frame is T1 and its pose in the second frame is T2, the coordinates are transformed as follows: Then, among all the reflection feature points in the second frame point cloud, points whose distance to P'1 is less than a certain threshold are selected. Point P2 with the smallest distance can be selected, and points P1 and P2 are associated to obtain multiple sets of association pairs of reflection feature points. The corresponding association relationship between each reflection feature point in the two frames is the first association relationship.

[0068] Step S108: Associate the geometric feature points of the first frame point cloud with the geometric feature points of the second frame point cloud to obtain the second association relationship.

[0069] In this step, the association method between the geometric feature points of the first frame point cloud and the geometric feature points of the second frame point cloud can adopt the association method of reflection feature points mentioned above, which will not be repeated here; or it can adopt the point-feature line segment (two points) and / or point-feature plane (three points) association methods proposed in other embodiments of this application below. The corresponding association relationship between each geometric feature point in the two frames is the second association relationship.

[0070] For example, any geometric feature point in the first frame point cloud can be transformed into the coordinate system of the second frame point cloud to obtain a geometric feature transformation point; among the multiple geometric feature points in the second frame point cloud, multiple points corresponding to the geometric feature transformation point are determined to describe the geometric feature (two points are determined when the geometric feature is an edge feature; three points are determined when the geometric feature is a planar feature), and the geometric feature transformation point is associated with the multiple points in the second frame point cloud used to describe the geometric feature.

[0071] Step S108: Register the first frame point cloud and the second frame point cloud according to the first association relationship and the second association relationship to obtain the pose transformation matrix of the first frame point cloud and the second frame point cloud.

[0072] In this step, the computer device can optimize the registration of the point clouds of the two frames based on the correspondence between the reflection feature points of the two frames and the correspondence between the geometric feature points of the two frames, thereby obtaining the pose transformation matrix of the point clouds of the two frames after optimization registration.

[0073] The point cloud data processing described above involves extracting reflection feature points and geometric feature points from the first and second frame point clouds respectively, and then associating these points in both frames. Point cloud registration is performed based on the association relationships between the reflection feature points and geometric feature points in the two frames. Using this method, the point cloud registration process relies not only on the characteristics of the associated geometric feature points but also on the characteristics of the reflection feature points. Therefore, even in scenarios where geometric features are not obvious, the accuracy of the lidar odometry can still be ensured, improving the robustness of the lidar odometry.

[0074] In one embodiment, such as Figure 2As shown, the reflection feature points of the first frame point cloud are associated with the reflection feature points of the second frame point cloud, including:

[0075] S202: Cluster the reflection feature points of the first frame point cloud and the reflection feature points of the second frame point cloud respectively to obtain multiple reflection feature cluster objects.

[0076] S203: Extract the centroid of each reflection feature cluster object as the reflection center point;

[0077] S204: Associate the reflection center point of the first frame point cloud with the reflection center point of the second frame point cloud.

[0078] In this embodiment, by clustering the extracted reflection feature points, reflection feature points belonging to the same object in the scene can be grouped into the same cluster object. The centroid of each reflection feature cluster object is used as the reflection center point, and the reflection center point is used as the feature point describing the reflection information to associate the reflection center points in two frames. By clustering a large number of reflection feature points and extracting the centroid (reflection center point) of the cluster objects, the amount of point association processing can be reduced, the point association efficiency can be improved, and thus the efficiency of point cloud registration can be improved.

[0079] Furthermore, after clustering the reflection feature points, points that do not belong to any cluster can be treated as noise and removed, thereby further reducing the amount of data processing and improving the processing efficiency and accuracy of point clouds.

[0080] More specifically, such as Figure 3 As shown, Figure 3 This is a schematic diagram illustrating the clustering process of reflection feature points in one embodiment. Figure 3 The system includes coordinate system 31, point A 32, and point B 33. For each reflection feature point A and B, the angle β is calculated as shown in the figure, where β is the angle between the long side AO and AB. If β is close to 90 degrees, then points A and B are considered to be in the same object and belong to the same reflection feature cluster object. Reflection feature points that are not in any cluster object can be considered as noise points, and all points in a reflection feature cluster object with fewer than a certain threshold can also be considered as noise points.

[0081] In one embodiment, associating the reflection center point of the first frame point cloud with the reflection center point of the second frame point cloud includes: transforming any one of the reflection center points of the first frame point cloud to the coordinate system of the second frame point cloud to obtain a reflection transformation point; selecting the point closest to the reflection transformation point from multiple reflection center points in the second frame point cloud as the reflection corresponding point; and associating any one of the reflection center points of the first frame point cloud with its corresponding reflection corresponding point.

[0082] In this embodiment, since the distribution of the centroids (reflection center points) of each reflection feature cluster object is relatively sparse after clustering, any reflection center point in the first frame point cloud can be transformed into the coordinate system of the second frame point cloud, and the point closest to the transformed point can be directly selected from the reflection center points of the second frame point cloud for association, thereby improving the association efficiency.

[0083] In one embodiment, after association processing, outliers can be further removed. For reflective feature points, a method for constructing a maximum connectivity graph can be used for removal. Each set of associated points is treated as a node. If the distance between two sets of associated points is less than a certain threshold, the nodes of the two sets of associated points are considered to be consistent, and an edge is added between the nodes of the two sets of associated points. This process is repeated for each set of associated points, ultimately resulting in a graph describing node consistency. The subgraph with the maximum connectivity is then found, and all nodes in the subgraph correspond to their respective sets of associated points as interior points. For geometric feature points, outliers can be removed using error distribution. Removing outliers improves point cloud registration efficiency and reduces the processing of irrelevant or erroneous sets of associated feature points.

[0084] In one embodiment, reference Figure 4 As shown, geometric feature points can include edge feature points. Associating the geometric feature points of the first frame point cloud with the geometric feature points of the second frame point cloud includes:

[0085] S402: Transform any edge feature point of the first frame point cloud into the coordinate system of the second frame point cloud to obtain the edge feature transformation point;

[0086] S404: Select the point closest to the edge feature transformation point from among the multiple edge feature points in the second frame point cloud as the first edge feature corresponding point;

[0087] S406: Select a point on the scan line adjacent to the scan line where the first edge feature point is located from multiple edge feature points in the second frame point cloud as the second edge feature point.

[0088] S408: Associate any edge feature point of the first frame point cloud with its corresponding first edge feature point and second edge feature point.

[0089] In this embodiment, feature association processing is performed on the edge feature points of the first frame point cloud and the second frame point cloud. To explain the association process more clearly, please refer to the following... Figure 5 As shown, in Figure 5In this diagram, pi′ represents the edge feature transformation point 51 obtained by transforming the edge feature point pi (not shown) in the first frame point cloud to the coordinate system of the second frame point cloud. pj represents the corresponding point 52 of the first edge feature, and pl represents the corresponding point 53 of the second edge feature. Point pj is the edge feature point in the second frame point cloud that is closest to pi′. Points pj and pl are edge feature points on two adjacent scan lines 54 in the second frame point cloud. In this embodiment, any edge feature point in the first feature set is associated with its corresponding first edge feature point 52 and second edge feature point 53, that is, point pi is associated with points pj and pl.

[0090] In one embodiment, reference Figure 6 As shown, geometric feature points may also include planar feature points. Associating the geometric feature points of the first frame point cloud with the geometric feature points of the second frame point cloud includes:

[0091] S602: Transform any planar feature point in the first frame point cloud to the coordinate system of the second frame point cloud to obtain the planar feature transformation point;

[0092] S604: Select the point closest to the planar feature transformation point from among the multiple planar feature points in the second frame point cloud as the first planar feature corresponding point;

[0093] S606: Select a point on the scan line adjacent to the scan line where the first planar feature point is located from the multiple planar feature points of the second frame point cloud as the second planar feature point.

[0094] S608: Select a point on the same scan line as the first planar feature point from multiple planar feature points in the second frame point cloud as the third planar feature point.

[0095] S610: Associate any planar feature point of the first frame point cloud with its corresponding first planar feature point, second planar feature point, and third planar feature point.

[0096] In this embodiment, feature association processing is performed on the planar feature points of the first frame point cloud and the second frame point cloud. To explain the association process more clearly, please refer to the following... Figure 7 As shown, in Figure 7In this diagram, pi′ is the planar feature transformation point 71 obtained by transforming the planar feature point pi (not shown) in the first frame point cloud to the coordinate system of the second frame point cloud; pj is the first planar feature corresponding point 72; pl is the second planar feature corresponding point 73; pm is the third planar feature point 74; point pj is the planar feature point in the second frame point cloud that is closest to pi′; points pj and pl are planar feature points on two adjacent scan lines 75 in the second frame point cloud; and points pj and pm are planar feature points on the same scan line 75 in the second frame point cloud. In this embodiment, any planar feature point pi in the first frame point cloud is associated with its corresponding first planar feature corresponding point 72, second planar feature corresponding point 73, and third planar feature corresponding point 74, that is, point pi is associated with points pj, pl, and pm.

[0097] The aforementioned feature point association method enables any geometric feature point in a point cloud of a certain frame to be associated with multiple feature points in another point cloud that describe that geometric feature. That is, it is not just a point-to-point association between two frames, but an association between a point and a geometric feature description. This can improve the stability and accuracy of geometric feature point association, and is more conducive to subsequent error optimization calculation and registration accuracy improvement.

[0098] In one embodiment, registration of the first frame point cloud and the second frame point cloud is performed based on a first association relationship and a second association relationship, including:

[0099] The error value between the reflection feature points of the first frame point cloud and the reflection feature points of the second frame point cloud in each associated group is calculated based on the first association relationship.

[0100] The error value between the geometric feature points of the first frame point cloud and the geometric feature points of the second frame point cloud in each associated group is calculated based on the second association relationship.

[0101] An error optimization function is constructed based on the calculated error value, and point cloud registration is performed based on the error optimization function.

[0102] In this embodiment, by way of example:

[0103] 1. For each group of associated reflection feature points (or each associated reflection center point), the error value can be obtained by calculating the distance between each group of associated reflection feature points. That is, the distance between a reflection feature point in the first frame point cloud and its corresponding associated reflection feature point in the second frame point cloud. The distance calculation formula can be found below:

[0104] d i =|p i -p j |

[0105] Where, pi and p j Let pi and pj represent the coordinates of points pi and pj, respectively. Points pi and pj are a set of associated reflection feature points.

[0106] 2. For each set of edge feature points after association, you can refer to... Figure 5 As shown, the distance 56 between each edge feature transformation point 51 and the line segment 55 formed by its corresponding first edge feature point 52 and second edge feature point 53 can be calculated. That is, the distance d between point pi′ and the line segment formed by point pj and point pl can be calculated. e , will d e This serves as the error value between the edge feature points of the first frame point cloud and the edge feature points of the second frame point cloud in each associated group. The specific calculation formula is as follows:

[0107]

[0108] Wherein, p in the above formula l p j Let p represent the coordinates of points pl and pj respectively. i ′ represents the coordinates of the transformed point obtained by transforming pi into the coordinate system of the second frame point cloud.

[0109] 3. For each set of planar feature points after association, such as Figure 7 As shown, the distance 77 between each planar feature transformation point 71 and the plane 76 formed by the corresponding points 72, 73, and 74 of the first, second, and third planar features can be calculated. That is, the distance d between point pi′ and the plane formed by points pj, pl, and pm can be calculated. p , will d p This serves as the error value between the planar feature points of the first frame point cloud and the planar feature points of the second frame point cloud after association. The specific calculation formula is as follows:

[0110]

[0111] Where, p i ′ and p j Let be the coordinates of the point, and e be the plane normal vector e = (p m -p j )×(p l -p j ).

[0112] An error optimization function is constructed based on the calculated error value. The least squares sum is used as the error optimization function to construct the pose transformation matrix. Specifically, the following optimization formula can be used as a reference:

[0113]

[0114] in, This is a function of ΔT, representing the error value calculated based on each group of edge feature points. The error value is calculated based on each group of planar feature points. This represents the error value calculated based on each group of reflection feature points. By adjusting ΔT, the error is made... Minimize the value of ΔT and construct the optimal pose change matrix at this point.

[0115] It should be understood that, although Figure 1 , Figure 2 , Figure 4 and Figure 6 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 , Figure 2 , Figure 4 and Figure 6 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0116] In one embodiment, such as Figure 8 As shown, a point cloud data processing device is provided, including a point cloud acquisition module 802, a feature extraction module 804, a first feature association module 806, a second feature association module 808, and a point cloud registration module 810, wherein:

[0117] The point cloud acquisition module 802 is used to acquire the first frame and the second frame of point cloud collected by the lidar.

[0118] The feature extraction module 804 is used to extract feature points from the first frame point cloud and the second frame point cloud respectively; wherein, the feature points include reflection feature points and geometric feature points;

[0119] The first feature association module 806 is used to associate the reflection feature points of the first frame point cloud with the reflection feature points of the second frame point cloud to obtain the first association relationship.

[0120] The second feature association module 808 is used to associate the geometric feature points of the first frame point cloud with the geometric feature points of the second frame point cloud to obtain a second association relationship.

[0121] The point cloud registration module 810 is used to register the first frame point cloud and the second frame point cloud according to the first association relationship and the second association relationship to obtain the pose transformation matrix of the first frame point cloud and the second frame point cloud.

[0122] In one embodiment, the feature extraction module 804 acquires the reflection intensity information of each point in the first frame point cloud and the second frame point cloud; and extracts the points with reflection intensity greater than a preset threshold as reflection feature points based on the reflection intensity information.

[0123] In one embodiment, the first feature association module 806 performs clustering processing on the reflection feature points of the first frame point cloud and the reflection feature points of the second frame point cloud to obtain multiple reflection feature cluster objects; extracts the centroid of each reflection feature cluster object as the reflection center point; and associates the reflection center point of the first frame point cloud with the reflection center point of the second frame point cloud.

[0124] In one embodiment, the first feature association module 806 transforms any one of the reflection center points of the first frame point cloud to the coordinate system of the second frame point cloud to obtain a reflection transformation point; selects the point closest to the reflection transformation point from multiple reflection center points in the second frame point cloud as the reflection corresponding point; and associates any one of the reflection center points of the first frame point cloud with its corresponding reflection corresponding point.

[0125] In one embodiment, the second feature association module 808 transforms any edge feature point of the first frame point cloud into the coordinate system of the second frame point cloud to obtain an edge feature transformation point; selects the point closest to the edge feature transformation point from among the multiple edge feature points of the second frame point cloud as the first edge feature corresponding point; selects a point on the scan line adjacent to the scan line where the first edge feature corresponding point is located from among the multiple edge feature points of the second frame point cloud as the second edge feature corresponding point; and associates any edge feature point of the first frame point cloud with its corresponding first edge feature corresponding point and second edge feature corresponding point.

[0126] In one embodiment, the second feature association module 808 transforms any planar feature point in the first frame point cloud to the coordinate system of the second frame point cloud to obtain a planar feature transformation point; selects the point closest to the planar feature transformation point from among the multiple planar feature points in the second frame point cloud as the first planar feature corresponding point; selects a point on the scan line adjacent to the scan line where the first planar feature corresponding point is located from among the multiple planar feature points in the second frame point cloud as the second planar feature corresponding point; selects a point on the scan line with the same scan line as the first planar feature corresponding point from among the multiple planar feature points in the second frame point cloud as the third planar feature corresponding point; and associates any planar feature point in the first frame point cloud with its corresponding first planar feature corresponding point, second planar feature corresponding point, and third planar feature corresponding point.

[0127] In one embodiment, the point cloud registration module 810 calculates the error value between the reflection feature points of each group of first frame point clouds and the reflection feature points of the second frame point cloud according to the first association relationship; calculates the error value between the geometric feature points of each group of first frame point clouds and the geometric feature points of the second frame point cloud according to the second association relationship; constructs an error optimization function based on the calculated error value; and performs point cloud registration based on the error optimization function.

[0128] Specific limitations regarding the point cloud data processing device can be found in the limitations of the point cloud data processing method described above, and will not be repeated here. Each module in the aforementioned point cloud data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0129] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows. Figure 9 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a point cloud data processing method.

[0130] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0131] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the following steps: acquiring a first frame point cloud and a second frame point cloud acquired by a lidar; extracting feature points from the first frame point cloud and the second frame point cloud respectively; wherein the feature points include reflection feature points and geometric feature points; associating the reflection feature points of the first frame point cloud with the reflection feature points of the second frame point cloud to obtain a first association relationship; associating the geometric feature points of the first frame point cloud with the geometric feature points of the second frame point cloud to obtain a second association relationship; and registering the first frame point cloud and the second frame point cloud according to the first association relationship and the second association relationship to obtain the pose transformation matrix of the first frame point cloud and the second frame point cloud.

[0132] In one embodiment, when the processor executes a computer program to extract reflection feature points from the first frame point cloud and the second frame point cloud respectively, it specifically performs the following steps: obtaining reflection intensity information of each point in the first frame point cloud and the second frame point cloud; and extracting points with reflection intensity greater than a preset threshold as reflection feature points based on each reflection intensity information.

[0133] In one embodiment, when the processor executes a computer program to associate the reflection feature points of the first frame point cloud with the reflection feature points of the second frame point cloud, it specifically performs the following steps: clustering the reflection feature points of the first frame point cloud and the reflection feature points of the second frame point cloud respectively to obtain multiple reflection feature cluster objects; extracting the centroid of each reflection feature cluster object as the reflection center point; and associating the reflection center point of the first frame point cloud with the reflection center point of the second frame point cloud.

[0134] In one embodiment, when the processor executes a computer program to associate the reflection center point of the first frame point cloud with the reflection center point of the second frame point cloud, it specifically performs the following steps: transforming any one of the reflection center points of the first frame point cloud to the coordinate system of the second frame point cloud to obtain a reflection transformation point; selecting the point closest to the reflection transformation point from multiple reflection center points in the second frame point cloud as the reflection corresponding point; and associating any one of the reflection center points of the first frame point cloud with its corresponding reflection corresponding point.

[0135] In one embodiment, when the processor executes a computer program to associate the geometric feature points of the first frame point cloud with the geometric feature points of the second frame point cloud, it specifically performs the following steps: transforming any edge feature point of the first frame point cloud into the coordinate system of the second frame point cloud to obtain an edge feature transformation point; selecting the point closest to the edge feature transformation point from among the multiple edge feature points of the second frame point cloud as the first edge feature corresponding point; selecting a point on the scan line adjacent to the scan line where the first edge feature corresponding point is located from among the multiple edge feature points of the second frame point cloud as the second edge feature corresponding point; associating any edge feature point of the first frame point cloud with its corresponding first edge feature corresponding point and second edge feature corresponding point.

[0136] In one embodiment, when the processor executes a computer program to associate geometric feature points of the first frame point cloud with geometric feature points of the second frame point cloud, it specifically performs the following steps: transforming any planar feature point in the first frame point cloud to the coordinate system of the second frame point cloud to obtain a planar feature transformation point; selecting the point closest to the planar feature transformation point from among the multiple planar feature points in the second frame point cloud as the first planar feature corresponding point; selecting a point on the scan line adjacent to the scan line where the first planar feature corresponding point is located from among the multiple planar feature points in the second frame point cloud as the second planar feature corresponding point; selecting a point on the scan line with the same scan line as the scan line where the first planar feature corresponding point is located from among the multiple planar feature points in the second frame point cloud as the third planar feature corresponding point; associating any planar feature point in the first frame point cloud with its corresponding first planar feature corresponding point, second planar feature corresponding point, and third planar feature corresponding point.

[0137] In one embodiment, when the processor executes a computer program to register a first frame point cloud and a second frame point cloud according to a first association relationship and a second association relationship, it specifically performs the following steps: calculating the error value between the reflection feature points of each group of first frame point clouds and the reflection feature points of the second frame point cloud according to the first association relationship; calculating the error value between the geometric feature points of each group of first frame point clouds and the geometric feature points of the second frame point cloud according to the second association relationship; constructing an error optimization function based on the calculated error value; and performing point cloud registration based on the error optimization function.

[0138] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0139] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0140] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the characters in this article generally indicate that the preceding and following related objects have an "or" relationship.

[0141] Terms such as “first” and “second” used in this application are used only to distinguish one entity or operation from another, and do not necessarily require or imply any actual sequential relationship or order between these entities or operations.

[0142] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A point cloud data processing method, the method comprising: Acquire the first and second frame point clouds collected by the lidar; Feature points are extracted from the first frame point cloud and the second frame point cloud respectively; wherein, the feature points include reflection feature points and geometric feature points; wherein, the extracted reflection feature points are points whose reflection intensity is greater than a preset threshold. The reflection feature points of the first frame point cloud are associated with the reflection feature points of the second frame point cloud to obtain the first association relationship; The geometric feature points of the first frame point cloud are associated with the geometric feature points of the second frame point cloud to obtain a second association relationship; The error values ​​between the reflection feature points of the first frame point cloud and the reflection feature points of the second frame point cloud in each associated group are calculated according to the first association relationship; the error values ​​between the geometric feature points of the first frame point cloud and the geometric feature points of the second frame point cloud in each associated group are calculated according to the second association relationship; an error optimization function is constructed based on the calculated error values; point cloud registration is performed based on the error optimization function to obtain the pose transformation matrix of the first frame point cloud and the second frame point cloud. The step of associating the reflection feature points of the first frame point cloud with the reflection feature points of the second frame point cloud includes: performing clustering processing on the reflection feature points of the first frame point cloud and the reflection feature points of the second frame point cloud respectively to obtain multiple reflection feature cluster objects; extracting the centroid of each reflection feature cluster object as the reflection center point; transforming any one of the reflection center points of the first frame point cloud to the coordinate system of the second frame point cloud to obtain a reflection transformation point; selecting the point closest to the reflection transformation point from the multiple reflection center points in the second frame point cloud as the reflection corresponding point; and associating any one of the reflection center points of the first frame point cloud with its corresponding reflection corresponding point.

2. The method according to claim 1, characterized in that, Extracting reflection feature points from the first frame point cloud and the second frame point cloud respectively, including: Obtain the reflection intensity information of each point in the first frame point cloud and the second frame point cloud; Based on the reflection intensity information, points with a reflection intensity greater than a preset threshold are extracted as reflection feature points.

3. The method according to claim 1, characterized in that, The geometric feature points include edge feature points, and associating the geometric feature points of the first frame point cloud with the geometric feature points of the second frame point cloud includes: Transform any edge feature point of the first frame point cloud into the coordinate system of the second frame point cloud to obtain the edge feature transformation point; Among the multiple edge feature points of the second frame point cloud, the point closest to the edge feature transformation point is selected as the first edge feature corresponding point; Among the multiple edge feature points of the second frame point cloud, the point on the scan line adjacent to the scan line where the first edge feature point is located is selected as the second edge feature point. Associate any edge feature point of the first frame point cloud with its corresponding first edge feature point and its corresponding second edge feature point.

4. The method according to claim 1, characterized in that, The geometric feature points include planar feature points, and associating the geometric feature points of the first frame point cloud with the geometric feature points of the second frame point cloud includes: Transform any planar feature point in the first frame point cloud to the coordinate system of the second frame point cloud to obtain the planar feature transformation point; Among the multiple planar feature points in the second frame point cloud, the point closest to the planar feature transformation point is selected as the first planar feature corresponding point; Among the multiple planar feature points of the second frame point cloud, the point on the scan line adjacent to the scan line where the first planar feature point is located is selected as the second planar feature point. Among the multiple planar feature points in the second frame point cloud, a point on the same scan line as the first planar feature point is selected as the third planar feature point. Associate any planar feature point of the first frame point cloud with its corresponding first planar feature point, second planar feature point, and third planar feature point.

5. A point cloud data processing device, characterized in that, The device includes: The point cloud acquisition module is used to acquire the first and second frame point clouds collected by the lidar. The feature extraction module is used to extract feature points from the first frame point cloud and the second frame point cloud respectively; wherein, the feature points include reflection feature points and geometric feature points; wherein, the extracted reflection feature points are points whose reflection intensity is greater than a preset threshold. The first feature association module is used to associate the reflection feature points of the first frame point cloud with the reflection feature points of the second frame point cloud to obtain a first association relationship; The second feature association module is used to associate the geometric feature points of the first frame point cloud with the geometric feature points of the second frame point cloud to obtain a second association relationship. The point cloud registration module is used to calculate the error value between the reflection feature points of each group of first frame point clouds and the reflection feature points of the second frame point cloud according to the first association relationship; calculate the error value between the geometric feature points of each group of first frame point clouds and the geometric feature points of the second frame point cloud according to the second association relationship; construct an error optimization function based on the calculated error values; and perform point cloud registration according to the error optimization function; wherein, The first feature association module is specifically used to perform clustering processing on the reflection feature points of the first frame point cloud and the reflection feature points of the second frame point cloud to obtain multiple reflection feature cluster objects; extract the centroid of each reflection feature cluster object as the reflection center point; transform any one of the reflection center points of the first frame point cloud to the coordinate system of the second frame point cloud to obtain the reflection transformation point; select the point closest to the reflection transformation point from the multiple reflection center points in the second frame point cloud as the reflection corresponding point; and associate any one of the reflection center points of the first frame point cloud with its corresponding reflection corresponding point.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images