Data processing method of laser radar and laser detection system

Abstract

The invention provides a data processing method of a laser radar and a laser detection system. The data processing method comprises the following steps: S11, acquiring a frame of point cloud data of the laser radar; s12, determining a set of ground points in the frame of point cloud data; and S13, determining calibration parameters of the laser radar point cloud according to the set of ground points. In the embodiment of the invention, after the ground point set is extracted by using the GPU, the point cloud ground level is calibrated at a high speed, so that the radar 3D point cloud ground is aligned with the ground level of the physical world. According to the embodiment of the invention, the ground point in the 3D point cloud can be quickly obtained, the ground point is drawn by using the GPU, and the laser radar point cloud calibration parameter is obtained by using the projection of the ground point in the preset plane, so that the point cloud of the laser radar can be calibrated to be close to a world coordinate system and be basically aligned with the horizontal plane under the world coordinate system.

Description

Technical Field

[0001] The present disclosure generally relates to the field of laser radar, and more particularly to a laser radar data processing method and a laser detection system. Background Art

[0002] LiDAR (LiDAR) is a radar system that uses laser beams to detect target characteristics such as position and velocity. It is an advanced detection method that combines laser technology with photoelectric detection technology. Due to its advantages such as high resolution, good concealment, strong resistance to active interference, excellent low-altitude detection performance, small size, and light weight, LiDAR is widely used in autonomous driving, transportation communications, drones, intelligent robots, resource exploration, and other fields.

[0003] When using LiDAR for detection, such as when constructing SLAM (Simultaneous Localization and Mapping) using LiDAR point clouds, accurate ground pose is often required. Since the actual installation location of a vehicle-mounted LiDAR may not be level with the physical horizontal plane (the earth), the ground in the point cloud output by the LiDAR is often not aligned with the physical horizontal plane (the earth). Therefore, when using the point cloud, it is necessary to extract and adjust the horizontal pose of the 3D point cloud so that the point cloud ground and the earth are level before constructing the overlay frame of the point cloud. Figure 1 The original LiDAR 3D point cloud is shown. As can be seen from the figure, the horizontal plane of the LiDAR point cloud is obviously tilted and has a significant angle with the horizontal plane of the world coordinate system, requiring horizontal posture adjustment.

[0004] When using existing technologies to adjust the horizontal pose of 3D point clouds, it takes a long time and the hardware cost increases as the number of point clouds increases.

[0005] The contents of the background technology section are merely the technologies known to the inventors and do not necessarily represent the existing technologies in this field.

[0006] Public content

[0007] In view of at least one drawback of the prior art, the present disclosure provides a laser radar data processing method, comprising:

[0008] S11: Get one frame of point cloud data from the laser radar;

[0009] S12: Determine a set of ground points in the frame of point cloud data; and

[0010] S13: Determine calibration parameters of the lidar point cloud based on the set of ground points.

[0011] According to one aspect of the present disclosure, step S13 includes:

[0012] Projecting the ground point onto a preset plane;

[0013] The calibration parameters of the lidar point cloud are determined according to the projection points of the ground points on the preset plane.

[0014] According to one aspect of the present disclosure, the preset plane includes a longitudinal plane and a transverse plane, and the calibration parameters of the lidar point cloud include an angle relative to the longitudinal plane and an angle relative to the transverse plane.

[0015] According to one aspect of the present disclosure, step S13 further includes:

[0016] The projection points of the ground points on the longitudinal plane and the transverse plane are fitted respectively to determine the calibration parameters of the laser radar point cloud.

[0017] According to one aspect of the present disclosure, step S13 further includes: repeating the following process to determine calibration parameters of the lidar point cloud:

[0018] Adjusting the spatial orientation of the set of ground points at a preset angular step size;

[0019] Projecting the adjusted set of ground points onto the longitudinal plane and the transverse plane;

[0020] determining the number of pixels in the projected image;

[0021] When the number of pixels is the smallest, the angle adjustment amount of the set of ground points is used as the calibration parameter of the lidar point cloud.

[0022] According to one aspect of the present disclosure, the step of projecting the ground point onto a preset plane is achieved by orthogonally projecting the ground point onto an orthogonal view on the preset plane.

[0023] According to one aspect of the present disclosure, the preset plane includes a first plane and a second plane that are orthogonal to each other, wherein the step of determining the calibration parameters of the lidar point cloud includes: determining the calibration parameters of the point cloud in the first plane and the calibration parameters of the point cloud in the second plane based on the projection points of the ground points on the first plane and the second plane, respectively.

[0024] According to one aspect of the present disclosure, the data processing method further includes step S14: adjusting the lidar point cloud according to the calibration parameters.

[0025] According to one aspect of the present disclosure, the data processing method further includes determining the ground point by:

[0026] S121: For a data point in the frame of point cloud data, obtaining at least two first adjacent data points; and

[0027] S122: Determine whether the data point is a ground point based on the one data point and the at least two first adjacent data points.

[0028] According to one aspect of the present disclosure, the laser radar includes multiple detection channels, and step S121 includes:

[0029] determining a channel corresponding to the data point;

[0030] selecting a channel adjacent to the channel corresponding to the data point; and

[0031] The at least two first adjacent data points are selected from the points generated by the adjacent channels.

[0032] According to one aspect of the present disclosure, step S122 includes:

[0033] Determine whether the orientation of a polygon formed by the data point and the at least two first adjacent data points is horizontal.

[0034] According to one aspect of the present disclosure, when the angle between the orientation of the polygon and the horizontal plane is smaller than a threshold, the orientation of the polygon is determined to be horizontal.

[0035] According to one aspect of the present disclosure, step S122 includes:

[0036] Determine whether the normal orientation of the polygon formed by the data point and the at least two first adjacent data points is vertical.

[0037] According to one aspect of the present disclosure, when the angle between the normal orientation and the vertical direction is smaller than a threshold, the orientation of the polygon is determined to be horizontal.

[0038] According to one aspect of the present disclosure, the data processing method further includes: when determining that the data point is a ground point,

[0039] selecting a second adjacent data point from the points generated by the channel corresponding to the data point;

[0040] For the second adjacent data point, steps S121 and S122 are executed to determine whether the second adjacent data point is a ground point.

[0041] According to one aspect of the present disclosure, the data processing method further includes: when the second adjacent data point is a ground point, verifying that the data point is a ground point.

[0042] According to one aspect of the present disclosure, the data processing method further includes: annotating points verified as ground points in the point cloud data to form a set of ground points.

[0043] According to one aspect of the present disclosure, the data processing method is executed by a graphics processing unit.

[0044] The present disclosure also provides a laser detection system, comprising:

[0045] a lidar configured to detect surroundings and generate point cloud data;

[0046] A graphics processing unit communicates with the laser radar to obtain the point cloud data and is configured to execute the data processing method as described above.

[0047] According to one aspect of the present disclosure, the graphics processing unit is integrated into the lidar.

[0048] According to one aspect of the present disclosure, the laser radar is configured to represent each point with four floating-point numbers, wherein three floating-point numbers correspond to spatial position data respectively.

[0049] In the embodiments of the present disclosure, a GPU is used to extract a set of ground points and then quickly calibrate the point cloud ground level, aligning the radar 3D point cloud ground level with the ground level of the physical world. According to the embodiments of the present disclosure, ground points in the 3D point cloud can be quickly acquired, and then the GPU is used to draw the ground points. The projection of the ground points on a preset plane is used to obtain the calibration parameters of the lidar point cloud, thereby calibrating the lidar point cloud to a level close to the world coordinate system and substantially aligning it with the horizontal plane in the world coordinate system. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0051] Figure 1 The original LiDAR 3D point cloud is shown;

[0052] Figure 2 A laser radar data processing method according to the present disclosure is shown;

[0053] Figure 3 An exemplary frame of point cloud data is shown in FIG;

[0054] Figure 4A A frame of point cloud data is exemplarily shown in FIG; Figure 4B The set of ground points is shown separately in ;

[0055] Figure 4C and Figure 4D The sets of ground points observed from the side and front are shown respectively;

[0056] Figure 5 The embodiment of the present disclosure shows Figure 1 The point cloud image obtained after processing the point cloud shown;

[0057] Figure 6 It shows the positioning of the LiDAR in the world coordinate system XYZ;

[0058] Figure 7 Schematic diagram of the projection of the ground point set in the YZ plane is shown;

[0059] Figure 8 A schematic diagram showing perspective projection and orthographic projection;

[0060] Figure 9 Shows the projection diagram of the ground point before and after adjustment;

[0061] Figure 10 The following illustrates an implementation method for determining a ground point according to an embodiment of the present disclosure;

[0062] Figure 11 shows a spatial triangle formed by connecting a data point with two first adjacent data points;

[0063] Figure 12 A schematic diagram showing the logical structure of a graphics processing unit (GPU); and

[0064] Figure 13 A laser detection system according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0065] Hereinafter, only certain exemplary embodiments are briefly described. As will be appreciated by those skilled in the art, the described embodiments may be modified in various ways without departing from the spirit or scope of the present disclosure. Therefore, the drawings and description are to be considered as illustrative in nature and not restrictive.

[0066] In the description of the present disclosure, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the accompanying drawings and are only for the convenience of describing the present disclosure and simplifying the description. They do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore should not be understood as limiting the present disclosure. In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, features defined as "first" and "second" may explicitly or implicitly include one or more of the said features. In the description of the present disclosure, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0067] In the description of this disclosure, it should be noted that, unless otherwise expressly specified or limited, the terms "installed," "connected," and "connected" should be understood broadly. For example, they can refer to fixed, removable, or integral connections; mechanical, electrical, or intercommunication connections; direct or indirect connections through an intermediary; and internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of these terms in this disclosure based on specific circumstances.

[0068] In this disclosure, unless otherwise expressly specified or limited, a first feature being "above" or "below" a second feature may include the first and second features being in direct contact, or may include the first and second features not being in direct contact but being in contact via another feature between them. Furthermore, a first feature being "above," "above," and "above" a second feature may include the first feature being directly above or diagonally above the second feature, or may simply mean that the first feature is at a higher level than the second feature. A first feature being "below," "below," and "below" a second feature may include the first feature being directly above or diagonally above the second feature, or may simply mean that the first feature is at a lower level than the second feature.

[0069] The disclosure below provides many different embodiments or examples for realizing different structures of the present disclosure. In order to simplify the disclosure of the present disclosure, the components and settings of specific examples are described below. Of course, they are merely examples and are not intended to limit the present disclosure. In addition, the present disclosure may repeat reference numbers and / or reference letters in different examples, and such repetition is for the purpose of simplicity and clarity and does not in itself indicate the relationship between the various embodiments and / or settings discussed. In addition, the present disclosure provides examples of various specific processes and materials, but those of ordinary skill in the art will appreciate the application of other processes and / or the use of other materials.

[0070] The following describes embodiments of the present disclosure in conjunction with the accompanying drawings. It should be understood that the embodiments described herein are only used to illustrate and explain the present disclosure, and are not used to limit the present disclosure.

[0071] Figure 2 A laser radar data processing method 10 according to the present disclosure is shown and described in detail below with reference to the accompanying drawings. Figure 2 As shown, the data processing method 10 includes the following steps.

[0072] In step S11: obtain a frame of point cloud data from the laser radar.

[0073] LiDAR typically includes multiple detection channels, each of which includes, for example, one or more lasers and one or more corresponding detectors. The lasers emit a detection laser beam at a preset time or frequency. The laser beam undergoes optical modulation (e.g., collimation) before exiting the LiDAR. When it strikes an obstacle, it undergoes diffuse reflection, with part of the laser beam returning to the LiDAR as an echo. The echo is received by the detector and converted into an electrical signal. A signal processing circuit or processor, based on this electrical signal, can calculate obstacle information, such as its distance and / or reflectivity.

[0074] Taking a laser radar with 64 detection channels as an example, each detection channel corresponds to a specific (vertical) detection angle of the laser radar, and the angle range covered by all detection channels together constitutes the vertical field of view of the laser radar. Therefore, in one detection round, a total of 64 points are formed in a specific (vertical) direction (if the detection does not actually detect an obstacle, the distance information obtained by the detection can be considered to be infinite). Taking a rotating mechanical radar or a galvanometer or swing mirror laser radar as an example, the 64 detection channels perform a round of detection at a certain angle (angular resolution, such as 0.05 degrees or 0.1 degrees), thereby covering the (horizontal) field of view of the laser radar. For a rotating mechanical radar, its horizontal field of view is 360°; for a galvanometer or swing mirror laser radar, its horizontal field of view is, for example, 120°, 150° or other angles. It is similar for FMCW (Frequency Modulated Continuous Wave) radar and will not be described here.

[0075] Figure 3 An exemplary frame of point cloud data is shown in FIG. For example, a laser radar includes 64 detection channels and performs 500 rounds of detection in one detection cycle. One frame of point cloud data can be represented in a matrix-like form, where the 500 data in each row represent the points obtained by the same detection channel during the 500 rounds of detection, and the 64 data in the same column represent the points obtained by the 64 detection channels in the same detection round. In addition, Figure 3 Each point shown in may contain corresponding detection data, such as the distance and / or reflectivity corresponding to the point. Based on the distance of the data point and the angle corresponding to the point, the spatial coordinates of the point may be restored.

[0076] It is easy for those skilled in the art to understand that during the 64*500 detection processes, some detection processes will not generate echoes, that is, the detection beam emitted by the laser does not encounter an obstacle, or the reflected echo is not received by the detector. At this time, the corresponding point can be left blank, or its distance can be marked as infinite, or other markings can be made.

[0077] In step S12: determining a set of ground points in the frame of point cloud data.

[0078] Figure 4A A frame of point cloud data is shown as an example, where the red points represent ground points; Figure 4B The set of ground points is shown separately in .

[0079] Different methods can be used to filter out a set of ground points from a point cloud. According to one embodiment of the present disclosure, the set of ground points can be filtered out using height. The data of each point includes the height data of the point (or the height data is calculated based on the distance data and angle data of the point). By setting a height threshold (for example, relative to the installation position of the laser radar, all points that are more than 1.4m downward are ground points), the set of ground points can be filtered out. The following will describe a method for filtering a set of ground points according to another embodiment of the present disclosure.

[0080] Figure 4C and Figure 4D The following diagram shows the original ground point set, viewed from the side and front, without calibration. It can be seen that the ground point set has a certain width whether viewed from the side or from the front, and therefore the ground point set is not on the same horizontal plane (the green line in the figure is the coordinate axis of the world coordinate system). Therefore, it is necessary to determine the calibration parameters of the point cloud and adjust the point cloud to align as closely as possible with the horizontal plane of the world coordinate system.

[0081] In step S13: according to the set of ground points, calibration parameters of the lidar point cloud are determined.

[0082] exist Figure 4B Based on the ground points, the calibration parameters of the lidar point cloud can be determined according to the ground point set, so as to restore the lidar point cloud to the horizontal posture in the world coordinate system as much as possible. Figure 5 The embodiment of the present disclosure shows Figure 1 After processing the point cloud shown, it is determined Figure 1 The calibration parameters of the point cloud are restored to the horizontal pose in the world coordinate system. Figure 1 and Figure 5 visible, Figure 5 In the , the horizontal plane of the adjusted point cloud is basically aligned with the horizontal plane in the world coordinate system.

[0083] According to an embodiment of the present disclosure, the data processing method 10 further includes step S14: adjusting the laser radar point cloud according to the calibration parameters, so that the horizontal plane of the adjusted point cloud is substantially aligned with the horizontal plane in the world coordinate system.

[0084] The following describes methods for determining calibration parameters of the lidar point cloud according to different embodiments of the present disclosure.

[0085] Figure 6 The figure shows the positioning of the LiDAR in the world coordinate system XYZ, where the position of the LiDAR is the origin O (regardless of the installation height of the radar), the front of its detection direction is Y (i.e. the direction of travel of the vehicle), the lateral direction is X, and the vertical direction is Z.

[0086] The point cloud generated by the LiDAR during the detection process is referenced to its own radar coordinate system, not the world coordinate system. This is because during the detection process, the position of the LiDAR may change, for example, its pitch angle may change (rotate around the X-axis), causing the radar coordinate system to offset from the world coordinate system. For example, its detection direction is not facing straight ahead, but diagonally upward or downward; its roll angle may change (rotate around the Y-axis), causing its detection direction to deviate to the left or right, rather than facing straight ahead. The world coordinate system XYZ can be considered to be completely coincident with the initial coordinate system of the LiDAR. As the LiDAR moves with the carrier on which it is installed, the world coordinate system and the radar coordinate system deviate. The point cloud of the LiDAR is a 3D point cloud distributed in space. Therefore, according to one embodiment of the present disclosure, the ground points can be first projected onto a preset plane, and then the calibration parameters of the LiDAR point cloud can be determined based on the projection points of the ground points on the preset plane.

[0087] Since the ground point set was obtained in step S12, ideally (i.e., the radar coordinate system and the world coordinate system coincide), if the current ground is horizontal, then the ground point set should appear as a nearly horizontal straight line, whether viewed from the side (along the X-axis) or from the front (along the Y-axis). In reality, due to uneven ground, only a nearly straight line can be observed. If the observed image is not nearly straight, it indicates that the current ground point set is not on a plane when viewed from the side or front. A correction angle must be entered to make the ground point set appear as a straight line, indicating that the angle correction of the ground point set has been completed. To adjust the ground point set in the side view to a plane, a correction angle around the Y-axis must be entered. By adjusting the correction angle, the pose of the ground point cloud is changed, optimizing it to a state closest to a plane. When the ground point set appears thinnest in the image, this represents the pose that requires adjustment. The correction amount around the Y-axis is recorded. Similarly, the correction amount around the X-axis is used to correct the point cloud in the front view. The ground point set can be corrected to be horizontal with the ground. The correction amount around the X-axis is recorded.

[0088] The preset plane is, for example, Figure 6 In an ideal situation (i.e. the LiDAR coordinate system completely coincides with the world coordinate system), when a ground point is projected vertically onto the YZ plane, the projected point should be completely on the Y axis (or parallel to the Y axis). However, in practice, due to the change in the LiDAR posture, some ground points may not be on the Y axis, but may be above or below the Y axis, with a diffuse distribution, such as Figure 7At this point, the calibration parameters of the LiDAR point cloud can be determined based on the projection point of the ground point on the YZ plane. Figure 7 The projected points in the image are fitted into a straight line (e.g., by rotating them around the Y-axis). The angle between this straight line and the Y-axis can be used as the calibration parameter for the LiDAR point cloud. Rotating the LiDAR point cloud around the X-axis by this angle can at least partially horizontally calibrate the LiDAR 3D point cloud.

[0089] Alternatively, the preset plane is, for example, Figure 6 The XZ plane shown. In an ideal situation (i.e. the LiDAR coordinate system completely coincides with the world coordinate system), the ground points are projected vertically onto the XZ plane, and the projection points should be completely on the X-axis (or parallel to the X-axis). However, in reality, due to changes in the LiDAR posture, some ground points may not be located on the X-axis, but above or below the X-axis, and the distribution is diffuse. At this time, the calibration parameters of the LiDAR point cloud can be determined based on the projection points of the ground points on the XZ plane. Specifically, for example, multiple projection points are fitted into a straight line (for example, by rotating and adjusting around the X-axis), and the angle between the straight line and the X-axis can be used as the calibration parameter of the LiDAR point cloud. By rotating the LiDAR point cloud around the Y-axis by this angle, the LiDAR 3D point cloud can be at least partially horizontally calibrated.

[0090] In addition, you can also first project on the YZ plane to obtain the (first) calibration parameters of the lidar point cloud, rotate the lidar point cloud around the X-axis by the angle, and then project the adjusted point cloud on the XZ plane to obtain the (second) calibration parameters of the lidar point again, and rotate the lidar point cloud around the Y-axis by the angle, so as to calibrate the lidar point cloud more accurately.

[0091] According to one embodiment of the present disclosure, a first plane and a second plane orthogonal to each other are used as the preset planes, and ground points are vertically projected onto the first and second orthogonal planes. Then, calibration parameters of the point cloud in the first plane and calibration parameters of the point cloud in the second plane are determined based on the projection points of the ground points on the first and second planes, respectively. For example, Figure 6 The YZ and XZ planes shown are used as preset planes, and the calibration parameters of the point cloud are determined based on the projections of ground points onto the YZ and XZ planes, respectively. The YZ plane can also be called the longitudinal plane, and the XY plane can also be called the transverse plane. The calibration parameters obtained for the LiDAR point cloud include angles relative to the longitudinal plane and angles relative to the transverse plane.

[0092] According to one embodiment of the present disclosure, when projecting the ground point on a preset plane, it is preferred to adopt an orthogonal projection method, that is, to achieve it by orthogonally projecting the ground point onto an orthogonal view on the preset plane, rather than using a perspective projection method.

[0093] Figure 8 A comparison diagram of orthographic projection and perspective projection is shown. Figure 8 As shown in the figure, in the field of view FOV observed by the human eye, there is a fox near the near plane and a puppy near the far plane, and the sizes of the two are similar. When the perspective projection on the left is used, due to the perspective relationship that objects are larger when they are near and smaller when they are far, the size of the fox in the projected image will be significantly larger than the puppy, as shown in the figure below. Figure 8 As shown in the figure on the left, this is not a true reflection of the size of the two. When orthogonal projection is used, the size of the fox and the puppy in the projected image is basically the same, as shown in the figure on the left. Figure 8 As shown in the figure on the right, this is a true reflection of the size of the two. Therefore, in the embodiment of the present disclosure, an orthogonal projection (or vertical projection) can be used to avoid distortion of point clouds at different locations on the ground due to projection.

[0094] As mentioned above, the calibration parameters of the point cloud can be determined by fitting. Specifically, after projecting the ground point onto the longitudinal plane and the transverse plane, the projection points of the ground point on the longitudinal plane and the transverse plane can be fitted respectively to determine the calibration parameters of the lidar point cloud. According to another embodiment of the present disclosure, the calibration parameters of the point cloud can also be obtained by image processing.

[0095] Specifically, step S13 further includes: repeating the following process to determine the calibration parameters of the lidar point cloud:

[0096] S131: Adjusting the spatial orientation of the set of ground points with a preset angle step size;

[0097] S132: Projecting the adjusted ground point set onto the longitudinal plane and the transverse plane;

[0098] S133: Determine the number of pixels in the projected image;

[0099] S134: When the number of pixels is the least, the angle adjustment amount of the set of ground points is used as a calibration parameter of the lidar point cloud.

[0100] Specifically, such as Figure 9 As shown in , first project the ground points onto the longitudinal plane (YZ plane) and the transverse plane (XZ), and the projection points are as follows Figure 9As shown in the two figures on the left, it can be seen that the number of projection points of the ground points on the longitudinal plane and the transverse plane is large and relatively messy. Then, the spatial orientation of the set of ground points is adjusted with a preset angle step (for example, the rotation angle around the Y axis and the rotation angle around the X axis) (step S131), and then the adjusted set of ground points is re-projected onto the longitudinal plane (YZ plane) and the transverse plane (XZ) (step S132). The projection points are as follows Figure 9 The two figures on the right of the figure are shown in FIG. At this point, the number of pixels in the new projected image is determined (step S133). Steps S131, S132, and S133 are repeated until the number of pixels is minimized. The angle adjustment amount of the ground point set is used as a component of the calibration parameter of the lidar point cloud (step S134).

[0101] Those skilled in the art will readily appreciate that in the above angle adjustment process, there are two adjustment variables: the rotation angle around the X-axis and the rotation angle around the Y-axis. The adjustment target is the number of pixels in the two projected images. Therefore, adjustment can be performed using gradient descent until the minimum number of pixels is achieved. Furthermore, the projected images can be generated on two canvases or on a single canvas using different color markers. Alternatively, the sum of the number of pixels in the two projected images can be used as the optimization target.

[0102] According to one embodiment of the present disclosure, when the number of pixels in each projected image is less than a preset threshold, or when the sum of the number of pixels in two projected images is less than a preset threshold, it can also be considered that the minimum number of pixels is reached.

[0103] When the number of pixels is minimized, the points in the two projected images are approximate straight lines. At this time, the adjustment angles of the two projected images relative to their respective original projected images are used as calibration parameters of the lidar point cloud.

[0104] Specifically, the ground point set can be projected onto a 256x256 image. Due to the orthogonal projection, overlapping ground pixels are automatically merged. This provides a criterion: when the number of pixels occupied by the image obtained by posture adjustment is minimized, the horizontal position angle is as close as possible to the true horizontal position angle. At this point, the total angle adjustment of the ground point set is the final optimal optimization parameter, which is used as the calibration parameter for the 3D point cloud.

[0105] Figure 10 The following shows an implementation method for determining ground points according to an embodiment of the present disclosure. Figure 10 Detailed description.

[0106] In step S121 : for a data point in the frame of point cloud data, at least two first adjacent data points are obtained.

[0107] Figure 10 The method shown can be implemented for each point or part of the points in the point cloud. For a point in the point cloud, at least two adjacent data points are obtained in step S121, which are called first adjacent data points. The first adjacent data points are Figure 3 The adjacent points in the point cloud are not necessarily the adjacent points in the actual physical world. Figure 3 For point P2-2 in the example, its adjacent data points can be points P1-2 and P3-2, or points P2-1 and P2-3, or points P1-1 and P3-3, or points P1-3 and P3-1, or points P1-1 and P3-1. In another embodiment of the present invention, more than two adjacent data points can be selected, for example, points P1-2, P3-2, and P2-1 can be selected, or points P2-1, P2-3, P1-1, and P3-3 can be selected. These are all within the scope of the present invention.

[0108] In step S122: determining whether the data point is a ground point based on the one data point and the at least two first adjacent data points.

[0109] In an embodiment of the present disclosure, when it is necessary to determine whether a point is a ground point, the judgment is not simply made based on the height of the point, but rather the point is determined to be a ground point based on the point and at least two adjacent points. For example, for point P2-2, the two first adjacent data points P1-2 and point P3-2 are combined to determine whether point P2-2 is a ground point.

[0110] According to a preferred embodiment of the present disclosure, the at least two first adjacent data points may include adjacent data points generated by adjacent channels in the same round of detection. Therefore, in some embodiments of the present invention, step S121 of selecting specific adjacent data points may include the following sub-steps:

[0111] Since any data point in the point cloud is acquired by a certain channel in a certain round of detection, the channel corresponding to the data point can be determined first;

[0112] selecting a channel adjacent to the channel corresponding to the data point; and

[0113] The at least two first adjacent data points are selected from the points generated by the adjacent channels.

[0114] Still taking point P2-2 as an example, first determine that the channel corresponding to point P2-2 is channel 2. Then select adjacent channels, such as channel 1, channel 3, or channel 1 and channel 3. Then, select the at least two first adjacent data points from the points generated by the adjacent channels. Preferably, the at least two first adjacent data points include points generated by adjacent channels in the same round of detection, such as point P1-2 and point P3-2, point P1-1 and point P1-2, point P1-2 and point P1-3, point P3-1 and point P3-3, etc.

[0115] According to an embodiment of the present disclosure, in step S103, whether the data point is a ground point can be determined by determining whether the orientation of a polygon formed by the data point and the at least two first adjacent data points is horizontal. For example, when the angle between the orientation of the polygon and the horizontal plane is less than a threshold, the orientation of the polygon is determined to be horizontal, and thus the point is determined to be a ground point.

[0116] After selecting at least two first adjacent data points, the specific data of the data point and the at least two first adjacent data points can be obtained. The specific data may include spatial coordinates, such as coordinates of an XYZ coordinate system, or polar coordinates. Based on the spatial coordinates of the data point and the at least two first adjacent data points, the spatial orientation of the connected polygon can be determined. The orientation of the polygon is then compared with a preset horizontal plane to determine the angle between the polygon and the horizontal plane. When the angle is equal to or close to 0°, it can be determined that the polygon is in a horizontal orientation, and the data point can be preliminarily determined to be a ground point. In addition, in order to maintain a certain tolerance, an angle threshold can be set, such as 10°, 20°, or 30°. Taking 30° as an example, when the angle between the polygon and the horizontal plane is less than 30°, the data point P2-2 can be preliminarily determined to be a ground point.

[0117] According to another embodiment of the present disclosure, in step S103, whether the point is a ground point can be determined based on whether the normal orientation of the polygon formed by the data point and the at least two first adjacent data points is vertical. For example, when the angle between the normal orientation and the vertical direction is less than a threshold (or the difference between the angle between the normal orientation and the horizontal plane and a right angle is less than a threshold), the orientation of the polygon is determined to be horizontal, and the point is then determined to be a ground point.

[0118] Figure 11In the figure, a spatial triangle formed by connecting point P2-2 and the two first adjacent points P1-2 and P3-2 is shown. On this basis, the normal line Normal of the spatial triangle can be calculated, and then it can be determined whether the normal line Normal is vertically oriented, for example, by determining whether the angle between the normal line Normal and the horizontal plane is equal to 90° or close to 90 degrees, it can be determined whether the point is a ground point. In addition, in order to maintain a certain tolerance, an angle threshold can be set, such as 10°, 20°, or 30°. Taking 30° as an example, when the angle between the normal line Normal and the vertical direction is less than 30° (or when the angle between the normal line Normal and the horizontal plane is greater than 60°), the data point P2-2 can be preliminarily determined to be a ground point.

[0119] The above data processing method can be used to preliminarily determine whether a data point in the point cloud is a ground point. To improve robustness, other adjacent data points can be used to re-verify whether the point is a ground point, thereby minimizing or eliminating the possibility of misjudgment.

[0120] During verification, a second adjacent data point adjacent to the point may be selected, wherein the second adjacent data point does not completely overlap with the first adjacent data point, and is preferably a completely different adjacent data point. Preferably, the second adjacent data point is selected from points generated by the channel corresponding to the data point. For example, for point P2-2, if the first adjacent points selected in step S1023 are points P1-2 and 2-2, then during verification, adjacent data points P2-1 or P2-3, also generated by the second channel, may be selected, or both adjacent data points P2-1 and P2-2 may be selected.

[0121] Next, for the second adjacent data point, similarly, by executing the above steps S121 and S122 , it is determined whether the second adjacent data point is a ground point.

[0122] When the second adjacent data point is a ground point, verify that the data point is a ground point.

[0123] In this way, when point P2-2 is initially determined to be a ground point, further verification of whether point P2-2 is indeed a ground point can be performed through other adjacent data points P2-1, point P2-3, or point 2-1 and point 2-3, which can minimize or eliminate the possibility of misjudgment.

[0124] According to an embodiment of the present disclosure, after a point is determined to be a ground point, it can be annotated in the point cloud data so that it can be easily and promptly determined whether the point is a ground point during subsequent data processing. Alternatively, after verifying that the point is a ground point, it can be annotated in the point cloud data.

[0125] Typically, depending on the LiDAR's installation position on a vehicle, some beams (e.g., beams 33-64) can scan the ground, while beams 1-32 typically scan horizontally and slightly upward, avoiding the ground. Therefore, according to embodiments of the present disclosure, the data processing method is only performed on the point cloud data generated by a predetermined portion of the detection channels.

[0126] In existing laser radars, a microprocessor or a central processing unit (CPU) is usually used to determine whether a point in a point cloud is a ground point. The amount of point cloud data of the laser radar is huge, generating millions of points of data per second. The CPU uses a serial processing method, which takes a long time. When the processing speed is increased, the hardware cost increases. In a preferred embodiment of the present invention, a graphics processing unit (GPU) is used instead of a microprocessor or a central processing unit to process the point cloud data, determine whether a point in the point cloud is a ground point, and determine the calibration parameters of the laser radar point cloud to adjust the point cloud. The image processing unit is relatively simple in structure, with a large number of computing units, which is far higher than the central processing unit. GPU is particularly good at processing image data. Every pixel on the image needs to be processed, and the process and method of processing each pixel are very similar.

[0127] Figure 12 FIG. 1 shows a schematic diagram of the logical structure of a graphics processing unit (GPU). Figure 12 As shown, the graphics processing unit GPU includes a large number of computing units ALU (logical operation unit), which can perform parallel calculations on each data point in the point cloud in parallel, with high processing efficiency.

[0128] Therefore, in the embodiments of the present disclosure, by adopting the graphics processing unit GPU for processing, the algorithm can be processed in parallel through parallel computing, which can effectively shorten the processing time by several times or even dozens of times, greatly accelerate the algorithm processing speed, and thus bring about a huge performance improvement.

[0129] According to one embodiment of the present disclosure, in order to facilitate processing using a graphics processing unit, Figure 3 Each point in the image includes four floating-point data, which can be expressed as X, Y, Z, and A. X, Y, and Z represent the three-dimensional position coordinates of the object scanned by the lidar in space. Data A has no specific meaning and is used as a fill value for the convenience of processing by the graphics processing unit. It can be set to zero or any value, and this disclosure does not limit it.

[0130] The embodiments of the present disclosure provide a technical solution for identifying ground points in point clouds. Unlike conventional CPU-based solutions for extracting ground points, this solution can utilize a GPU to accelerate the extraction process, reducing pre-processing time. Furthermore, by employing a secondary verification scheme, the robustness of ground point confirmation is improved.

[0131] Figure 13 A laser detection system 20 according to the present disclosure is shown, comprising a laser radar 21 and an image processing unit 23. The laser radar 21 is configured to detect the surrounding environment and generate point cloud data, and the image processing unit 23 communicates with the laser radar 21 to obtain the point cloud data and is configured to execute the data processing method 10 described above.

[0132] like Figure 13 As shown, the laser radar 21 is configured to emit a detection laser beam L for detecting the surrounding environment, and simultaneously receive an echo L'. Based on the echo L', it calculates obstacle information, such as distance and / or reflectivity, to generate a laser radar point cloud. The graphics processing unit 23 can acquire the point cloud data from the laser radar 21 and execute the aforementioned data processing method 10 to determine whether all or some of the points in the point cloud are ground points.

[0133] The graphics processing unit 23 can be external to the LiDAR, such as a chip mounted on a vehicle. According to one embodiment of the present disclosure, the graphics processing unit 23 can also be integrated into the LiDAR 21, so that the LiDAR 21 has sufficient data processing capabilities, reducing the performance requirements of external data processing.

[0134] According to an embodiment of the present disclosure, the laser radar 21 includes multiple detection channels, and the graphics processing unit 23 is configured to execute the data processing method 10 for the point cloud data generated by a preset part of the detection channels, for example, only executing the data processing method 10 for the point cloud data generated by those detection channels facing the ground.

[0135] According to an embodiment of the present disclosure, the laser radar is configured to represent each point with four floating-point numbers, that is, each point includes four floating-point data, which can be expressed as X, Y, Z, and A, wherein X, Y, and Z represent the three-dimensional position coordinates in space scanned by the laser radar. Data A does not have a specific meaning and is used as a fill value for the convenience of processing by the graphics processing unit. It can be set to zero or to any value, and the present disclosure does not limit it here.

[0136] In the embodiments of the present disclosure, a GPU is used to extract a set of ground points and then quickly calibrate the point cloud ground level, aligning the radar 3D point cloud ground level with the ground level of the physical world. According to the embodiments of the present disclosure, ground points in the 3D point cloud can be quickly acquired, and then the GPU is used to draw the ground points. The projection of the ground points on a preset plane is used to obtain the calibration parameters of the lidar point cloud, thereby calibrating the lidar point cloud to a level close to the world coordinate system and substantially aligning it with the horizontal plane in the world coordinate system.

[0137] Finally, it should be noted that the above description is merely a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features therein. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present disclosure shall be included within the scope of protection of the present disclosure.

Claims

1. A laser radar data processing method, comprising: S11: Get one frame of point cloud data from the laser radar; S12: Determine a set of ground points in the frame of point cloud data; and S13: Determine calibration parameters of the lidar point cloud based on the set of ground points.

2. The data processing method according to claim 1, wherein step S13 comprises: Projecting the ground point onto a preset plane; The calibration parameters of the lidar point cloud are determined according to the projection points of the ground points on the preset plane.

3. The data processing method according to claim 2, wherein the preset plane includes a longitudinal plane and a transverse plane, and the calibration parameters of the lidar point cloud include an angle relative to the longitudinal plane and an angle relative to the transverse plane.

4. The data processing method according to claim 3, wherein step S13 further comprises: The projection points of the ground points on the longitudinal plane and the transverse plane are fitted respectively to determine the calibration parameters of the laser radar point cloud.

5. The data processing method according to claim 3, wherein the step S13 further comprises: Repeat the following process to determine the calibration parameters of the LiDAR point cloud: Adjusting the spatial orientation of the set of ground points at a preset angular step size; Projecting the adjusted set of ground points onto the longitudinal plane and the transverse plane; determining the number of pixels in the projected image; When the number of pixels is the smallest, the angle adjustment amount of the set of ground points is used as the calibration parameter of the lidar point cloud.

6. The data processing method as claimed in claim 2, wherein the step of projecting the ground point onto a preset plane is achieved by orthogonally projecting the ground point onto an orthogonal view on the preset plane.

7. The data processing method according to claim 2, wherein the preset plane comprises a first plane and a second plane that are orthogonal to each other, and wherein the step of determining the calibration parameters of the lidar point cloud comprises: Calibration parameters of the point cloud in the first plane and calibration parameters of the point cloud in the second plane are determined according to projection points of the ground points on the first plane and the second plane, respectively.

8. The data processing method according to claim 7, further comprising: Step S14: adjusting the lidar point cloud according to the calibration parameters.

9. The data processing method according to any one of claims 1 to 8, further comprising determining the ground point by: S121: For a data point in the frame of point cloud data, obtaining at least two first adjacent data points; and S122: Determine whether the data point is a ground point based on the data point and the at least two first adjacent data points.

10. The data processing method according to claim 9, wherein the laser radar includes multiple detection channels, and step S121 includes: determining a channel corresponding to the data point; Selecting a channel adjacent to the channel corresponding to the data point; and The at least two first adjacent data points are selected from the points generated by the adjacent channels.

11. The data processing method according to claim 9, wherein the step S122 comprises: Determine whether the orientation of a polygon formed by the data point and the at least two first adjacent data points is horizontal. 12 . The data processing method according to claim 11 , wherein when the angle between the orientation of the polygon and a horizontal plane is smaller than a threshold, the orientation of the polygon is determined to be horizontal.

13. The data processing method according to claim 9, wherein the step S122 comprises: Determine whether the normal orientation of the polygon formed by the data point and the at least two first adjacent data points is vertical. 14 . The data processing method according to claim 13 , wherein when the angle between the normal orientation and the vertical direction is smaller than a threshold, the orientation of the polygon is determined to be horizontal.

15. The data processing method according to claim 10, further comprising: When the data point is determined to be a ground point, selecting a second adjacent data point from the points generated by the channel corresponding to the data point; For the second adjacent data point, steps S121 and S122 are executed to determine whether the second adjacent data point is a ground point.

16. The data processing method according to claim 15, further comprising: When the second adjacent data point is a ground point, verify that the data point is a ground point.

17. The data processing method according to claim 16, further comprising: Points verified as ground points are marked in the point cloud data to form a set of ground points.

18. The data processing method according to any one of claims 1 to 17, wherein the data processing method is executed by a graphics processing unit.

19. A laser detection system comprising: a lidar configured to detect surroundings and generate point cloud data; A graphics processing unit communicates with the laser radar to obtain the point cloud data, and is configured to execute the data processing method according to any one of claims 1 to 18.

20. The laser detection system of claim 19, wherein the graphics processing unit is integrated into the laser radar.

21. The laser detection system of claim 19 or 20, wherein the laser radar is configured to represent each point with four floating point numbers, wherein three floating point numbers correspond to spatial position data respectively.

Images