Method for using radar point cloud data in large scale
By combining pre-fusion and post-fusion methods, using target detection algorithm and quaternary operation, the problems of high error rate and detection algorithm collapse in large-batch lidar scenarios are solved, and the stable fusion of large-batch radar point cloud data is achieved. and accurate identification.
Patent Information
- Application Number
- CN202411769375.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-04
- Publication Date
- 2025-01-17
AI Technical Summary
There are many existing lidar fusion solutions that have high misjudgment rates and crashes in large batch scenarios. The point cloud density after the fusion solution is too large, resulting in too much pressure on the algorithm. The point cloud of a single radar target of the post-fusion solution is In fewer cases, the recognition is unstable and the misjudgment rate is high.
A method combining pre-fusion and post-fusion is adopted to obtain point cloud data of multiple lidars in the detection area for splicing and fusion, use the target detection algorithm to establish a tracking target database, and perform initial and secondary judgments in the same coordinate system. , set a similarity threshold to judge the consistency of the target, combine quaternary operation and coordinate conversion matrix conversion to ensure that point clouds overlap at different position angles.
It effectively reduces the misjudgment rate, solves the algorithm crash and misjudgment problems, and realizes the stable fusion of large-scale radar point cloud data and the accuracy of identification results.
Smart Images
Figure CN119322327A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent traffic safety technology, and more specifically, to a method for using radar point cloud data in large quantities. Background Art
[0002] With the maturity of LiDAR technology and its popular application in autonomous driving and other daily life scenarios, a single sensor can hardly meet the application requirements of some complex scenarios, such as large traffic intersections or parking lots. Due to the limitations of detection distance and field of view, it is difficult for a single LiDAR to fully identify and perceive all traffic participants in the scene. Therefore, a fusion measurement solution of multiple LiDARs is needed to solve the problem of blind spots. Commonly used multi-radar fusion solutions include front fusion solution and post-fusion solution.
[0003] Pre-fusion solution: Use multiple radars, and transform the point cloud data of multiple radars into the same coordinate system through transformation matrix transformation to obtain fused point cloud data. Multiple radars can fill in the gaps and blind spots. For example, due to the detection distance limitation or occlusion, the radar at 0 degrees can only see part of the target point cloud at 180 degrees, resulting in incomplete point cloud. If the radar point cloud at 180 degrees is converted and merged into the entire scene, the point cloud of these targets at 180 degrees can be completed.
[0004] The drawbacks of this solution are: there are too many radars in the scene, and the density of the fused point cloud is too high, which puts tremendous pressure on the detection algorithm and graphics drawing. If the number is too large, it may even cause the detection algorithm (such as the public 3D target point cloud detection algorithm pointpillar) to stop working. Therefore, the front fusion solution can generally only fuse about 10 laser radars at most, which cannot meet the fusion scenario of a large number of radars (such as 50-100).
[0005] Post-fusion solution: The point cloud data of a single laser radar is output to the detection algorithm to identify the target object, and then the targets identified by multiple laser radars are converted to the same coordinate system through the transformation matrix for fusion. If the similarity of various attributes of the target (coordinates, angles, contours, etc.) is greater than a certain threshold, they are considered to be the same target and the post-fusion is completed.
[0006] The drawbacks of this solution are: when a single radar target point cloud is small, the identified target is unstable. When fusion is performed, the recognition similarity of multiple radars for the same target is too different, which will cause difficulty in fusion and produce some misidentified targets. The more radars there are, the more targets need to be similarity judged, which makes it easier to make misjudgments.
[0007] In view of this, the present invention proposes a method for using radar point cloud data in large quantities with low misjudgment rate and effective fusion. Summary of the invention
[0008] The purpose of the present invention is to provide a method for using radar point cloud data in large quantities with low misjudgment rate and effective fusion.
[0009] A method for using radar point cloud data in large quantities, characterized in that it comprises the following steps:
[0010] Step 1: Obtain point cloud data of laser radars arranged in groups within the detection area and perform splicing and fusion to obtain fused point cloud data of the detection area, wherein at least one group of laser radars is arranged in each detection area, and a group of laser radars includes multiple laser radars;
[0011] Step 2: Using the fused point cloud data of at least one group of laser radars obtained in step 1 through a target detection algorithm to obtain multiple detection target data of the corresponding radar in the detection area and establish a tracking target database;
[0012] Step 3: The detection target data obtained in step 2 is converted to the same coordinate system for initial judgment and fusion of multiple detection target data of the corresponding laser radar in the detection area. When the judgment threshold is met, it is judged as the same target and the newly generated detection target data is merged. When the judgment threshold is not met, it is judged as a different target and the multi-target data is maintained for continuous tracking;
[0013] Step 4: The new detection target data generated by the first determination in step 3 is re-determined with the detection target data not determined in step 3, and the corresponding data in the tracking target database is updated if it is determined to be the same target.
[0014] Furthermore, step one is specifically to obtain point cloud data of laser radars arranged in groups within the detection area, perform quaternion operation on the point cloud data of each laser radar to obtain parallel point cloud data, remove the parallel point cloud data outside the detection area, retain the parallel point cloud data in the detection area, and transform the retained parallel point cloud data into the same radar point cloud coordinate system through a coordinate transformation matrix to complete fusion to obtain fused point cloud data of the group of laser radars.
[0015] Furthermore, in step one, the point cloud coordinates in the parallel point cloud data include that the x-axis and y-axis are parallel to the ground, and the Y-axis is perpendicular to the ground. Since a single laser radar passes through a gantry at a certain angle to the ground (generally 15° to the left and 18° to the bottom), parallel point cloud data are obtained through quaternion calculation conversion. The parallel point cloud data is parallel to the road surface, and the vehicle's forward direction is parallel to the X-axis direction, which is convenient for ranging and subsequent fusion detection.
[0016] Furthermore, in step one, it also includes removing the parallel point cloud data of the non-detection area, restoring the Z-axis coordinate of the parallel point cloud data converted by quaternion operation to 0, and retaining the point cloud data within the boundary of the detection area.
[0017] Furthermore, in step one, the coordinate transformation matrix is specifically converted as follows: the point cloud of one of the radars in the point cloud data within the boundary of the detection area is retained as the reference point cloud, and the other radar point clouds are rotated and translated, and converted to the same radar point cloud coordinate system to complete the fusion to obtain the fused point cloud data of the group of lidars, so that different point clouds at different positions and angles overlap, and the same target object seen overlaps together, and is no longer multiple ghost images of the same target.
[0018] In some embodiments, step three is specifically as follows: the detection target data obtained in step two is converted into the same coordinate system for initial judgment and fusion of multiple detection target data of the corresponding laser radar in the detection area. When the judgment threshold is one of the overlap greater than 50%, the length, width and height similarity greater than 90%, and the forward angle less than 10°, it is judged to be the same target when the judgment threshold is met. When the initial judgment meets the judgment threshold, it is judged to be the same target, and the newly generated detection target data is merged. When it does not meet the judgment threshold, it is judged to be a different target, and the multi-target data is maintained for continuous tracking.
[0019] In some embodiments, step four is specifically as follows: when the secondary judgment meets the judgment threshold, the judgment threshold is one of the overlap less than 50%, the length, width and height similarity greater than 90%, and the forward angle less than 10°, it is judged to be the same target, and the newly detected target data generated by the first judgment is replaced with the corresponding data in the tracking target database.
[0020] Beneficial effects of the invention: The invention proposes a method for using radar point cloud data in large quantities, acquiring point cloud data of laser radars arranged in groups within a detection area and performing splicing and fusion to obtain fused point cloud data of the detection area, and then obtaining multiple detection target data of the corresponding radar in the detection area through a target detection algorithm and establishing a tracking target database, and combining front fusion and post fusion to form the final recognition result, effectively solving the problems of algorithm collapse and misjudgment. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 The present invention provides a flowchart of a method for using radar point cloud data in large quantities.
[0022] Figure 2 This is a specific flow chart of a method for using radar point cloud data in large quantities in this application.
[0023] Figure 3 This is a pre-fusion flowchart of a method for using radar point cloud data in large quantities in this application.
[0024] Figure 4 This is a post-fusion flowchart of a method for using radar point cloud data in large quantities in this application.
[0025] Main component symbols
[0026] The following specific implementation manner will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION
[0027] The following examples are described to assist the understanding of the present application, and the examples are not and should not be interpreted in any way as limiting the scope of protection of the present application.
[0028] In the following description, those skilled in the art will recognize that throughout this discussion, components may be described as separate functional units (which may include sub-units), but those skilled in the art will recognize that various components or portions thereof may be divided into separate components or may be integrated together (including within a single system or component).
[0029] At the same time, the connections between components or systems are not intended to be limited to direct connections, rather, the data between these components may be modified, reformatted, or otherwise changed by intermediate components. In addition, additional or fewer connections may be used. It should also be noted that the terms "coupled," "connected," or "input" should be understood to include direct connections, indirect connections through one or more intermediate devices, and wireless connections.
[0030] Embodiment 1:
[0031] like Figure 1-2 As shown, a method for using radar point cloud data in large quantities, characterized in that it includes the following steps:
[0032] Step 1: Obtain point cloud data of laser radars arranged in groups within the detection area and perform splicing and fusion to obtain fused point cloud data of the detection area, wherein at least one group of laser radars is arranged in each detection area, and a group of laser radars includes multiple laser radars;
[0033] Step 2: Using the fused point cloud data of at least one group of laser radars obtained in step 1 through a target detection algorithm to obtain multiple detection target data of the corresponding radar in the detection area and establish a tracking target database;
[0034] Step 3: The detection target data obtained in step 2 is converted to the same coordinate system for initial judgment and fusion of multiple detection target data of the corresponding laser radar in the detection area. When the judgment threshold is met, it is judged as the same target and the newly generated detection target data is merged. When the judgment threshold is not met, it is judged as a different target and the multi-target data is maintained for continuous tracking;
[0035] Step 4: The new detection target data generated by the first determination in step 3 is re-determined with the detection target data not determined in step 3, and the corresponding data in the tracking target database is updated if it is determined to be the same target.
[0036] like Figure 3 As shown, step one specifically includes obtaining point cloud data of laser radars arranged in a group within the detection area, performing quaternion operation on the point cloud data of each laser radar to obtain parallel point cloud data, removing the parallel point cloud data outside the detection area, retaining the parallel point cloud data in the detection area, and converting the retained parallel point cloud data into the same radar point cloud coordinate system through a coordinate conversion matrix to complete fusion and obtain fused point cloud data of the group of laser radars;
[0037] The point cloud coordinates in the parallel point cloud data have x-axis and y-axis parallel to the ground, and the Y-axis is perpendicular to the ground. Since a single laser radar passes through a gantry at a certain angle to the ground (generally 15° to the left and 18° to the bottom), the parallel point cloud data is obtained through quaternion calculation conversion. The parallel point cloud data is parallel to the road surface, and the vehicle's forward direction is parallel to the X-axis direction, which is convenient for ranging and subsequent fusion detection.
[0038] The parallel point cloud data in the non-detection area is removed, the Z-axis coordinate of the parallel point cloud data converted by quaternion operation is restored to 0, and the point cloud data outside the boundary of the detection area is retained.
[0039] In the point cloud fusion algorithm, matrix transformation is used to transform the coordinates of all radar point clouds in the same group of radars into the coordinate system of one of the radars in the group. The conversion method is: use the three-dimensional homogeneous matrix Q to represent the transformation matrix. Each radar point cloud has a separate transformation matrix Q, represented by Q1, Q2, Q3..., and the representation of Q is Where R is the rotation transformation, t is the translation transformation distance column vector; the rotation matrix R is a 3X3 matrix in the form of Each value in the matrix represents a rotation coefficient; the values in the transformation matrix Q need to be generated by various point cloud registration tools. Common registration algorithms include ICP and NDT algorithms. The registration tool used in this application is self-developed. For a three-dimensional point (x, y, z) in the point cloud, it is represented as a column vector V, V = [x, y, z, 1] T , where the superscript T represents the transposition of the row vector. Use matrix multiplication to generate a new three-dimensional point (x1, y1, z1), V1 = [x1, y1, z1, 1], V1 = Q * V. The new three-dimensional point V1 after matrix multiplication is the position coordinate of point V in the new coordinate system. Then put the new coordinates of all points into a new point cloud group and send it to the target detection algorithm for target perception and recognition to obtain the detection target data.
[0040] like Figure 4As shown, step three is specifically that the detection target data obtained in step two is converted to perform an initial judgment in the same coordinate system and fuse multiple detection target data of the corresponding laser radar in the detection area. For example, a group of laser radars is divided into laser radar 1 and laser radar 2. When target A is detected in the detection target data of laser radar 1, target A is detected in the detection target data of laser radar 2 at the same time. At this time, target A of laser radar 2 is converted into target AA through the coordinate system. When the judgment threshold meets one of the overlap degree less than 50%, the length, width and height similarity greater than 90%, and the forward angle less than 10°, it meets the judgment threshold and is judged as the same target. When the initial judgment meets the judgment threshold, it is judged as the same target, and the newly generated detection target data target AAA is merged. When it does not meet the judgment threshold, it is judged as a different target and is maintained as multi-target data for continuous tracking.
[0041] Step 4 is to perform a secondary judgment on the target AAA obtained in step 3 and the target A. When the judgment threshold is met, the judgment threshold is one of the following: the overlap is less than 50%, the length, width and height similarity is greater than 90%, and the forward angle is less than 10°, it is judged to be the same target, and the target AAA of the newly detected target data generated by the first judgment is replaced with the corresponding data target A in the tracking target database.
[0042] Although the present application has disclosed multiple aspects and embodiments, other aspects and embodiments will be obvious to those skilled in the art, and several modifications and improvements may be made without departing from the concept of the present application, all of which belong to the protection scope of the present application. The multiple aspects and embodiments disclosed in the present application are only for illustration and are not intended to limit the present application. The actual protection scope of the present application shall be subject to the claims.
Claims
1. A method for using radar point cloud data in large quantities, characterized in that: The following steps are included: Step 1: Obtain point cloud data of laser radars arranged in groups within the detection area and perform splicing and fusion to obtain fused point cloud data of the detection area, wherein at least one group of laser radars is arranged in each detection area, and a group of laser radars includes multiple laser radars; Step 2: Using the fused point cloud data of at least one group of laser radars obtained in step 1 through a target detection algorithm to obtain multiple detection target data of the corresponding radar in the detection area and establish a tracking target database; Step 3: The detection target data obtained in step 2 is converted to the same coordinate system for initial judgment and fusion of multiple detection target data of the corresponding laser radar in the detection area. When the judgment threshold is met, it is judged as the same target and the newly generated detection target data is merged. When the judgment threshold is not met, it is judged as a different target and the multi-target data is maintained for continuous tracking; Step 4: The new detection target data generated by the first determination in step 3 is re-determined with the detection target data not determined in step 3, and the corresponding data in the tracking target database is updated if it is determined to be the same target.
2. The method for using radar point cloud data in large quantities as claimed in claim 1, characterized in that: Specifically, step one is to obtain point cloud data of laser radars arranged in groups within the detection area, perform quaternion operation on the point cloud data of each laser radar to obtain parallel point cloud data, remove the parallel point cloud data outside the detection area, retain the parallel point cloud data in the detection area, and convert the retained parallel point cloud data into the same radar point cloud coordinate system through a coordinate transformation matrix to complete the fusion and obtain the fused point cloud data of the group of laser radars.
3. The method for using radar point cloud data in large quantities as claimed in claim 1, characterized in that: In step one, the point cloud coordinates in the parallel point cloud data include x-axis and y-axis which are parallel to the ground, and the Y-axis which is perpendicular to the ground. Since a single laser radar passes through a gantry at an angle to the ground, parallel point cloud data are obtained through quaternion calculation conversion. The parallel point cloud data is parallel to the road surface, and the vehicle's forward direction is parallel to the X-axis direction, which is convenient for ranging and subsequent fusion detection.
4. The method for using radar point cloud data in large quantities as claimed in claim 1, characterized in that: In step one, it also includes removing the parallel point cloud data in the non-detection area, restoring the Z-axis coordinates of the parallel point cloud data converted by quaternion operation to 0, and retaining the point cloud data outside the boundary of the detection area.
5. The method for using radar point cloud data in large quantities as claimed in claim 1, characterized in that: In step one, the coordinate transformation matrix is specifically: the point cloud of one of the radars in the point cloud data within the boundary of the detection area is retained as the reference point cloud, and the other radar point clouds are rotated and translated and converted to the same radar point cloud coordinate system to complete the fusion and obtain the fused point cloud data of the group of lidars.
6. The method for using radar point cloud data in large quantities as claimed in claim 1, characterized in that: Specifically, step three is that the detection target data obtained in step two is converted to the same coordinate system for initial judgment and fusion of multiple detection target data of the corresponding laser radar in the detection area. When the judgment threshold is one of the overlap less than 50%, the length, width and height similarity greater than 90%, and the forward angle less than 10°, it is judged to be the same target when it meets the judgment threshold. When the initial judgment meets the judgment threshold, it is judged to be the same target, and the newly generated detection target data is merged. When it does not meet the judgment threshold, it is judged to be a different target and the multi-target data is maintained for continuous tracking.
7. The method for using radar point cloud data in large quantities as claimed in claim 1, characterized in that: Step 4 is specifically as follows: when the secondary judgment meets the judgment threshold, the judgment threshold is one of the following: the overlap is less than 50%, the length, width and height similarity is greater than 90%, and the forward angle is less than 10°, it is judged to be the same target, and the newly detected target data generated by the first judgment is replaced with the corresponding data in the tracking target database.