3d target matching method based on trajectory intersection-to-union ratio
By using a 3D target matching method based on trajectory intersection-union ratio (IOU) and optimizing the matching process with GIOU and T-IOU algorithms, the problem of low matching accuracy and insufficient robustness of traditional IOU index in autonomous driving scenarios is solved, achieving more accurate and coherent target trajectory generation and reducing equipment costs.
Patent Information
- Application Number
- CN202411421033.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-12
- Publication Date
- 2025-02-07
AI Technical Summary
In 3D object detection in autonomous driving scenarios, traditional IOU metrics have low matching accuracy, insufficient robustness, and high cost. They cannot effectively distinguish the alignment of the target boxes, resulting in missing or incorrect matching results, especially in vehicle and pedestrian detection.
A 3D target matching method based on trajectory intersection-union ratio is adopted, which includes point cloud data detection, global coordinate transformation, preliminary GIOU matching, and trajectory reallocation by T-IOU. The correlation between the detection box and the trajectory is recalculated through preliminary GIOU matching and T-IOU to optimize the matching results.
It improves the accuracy and consistency of 3D target matching, is suitable for linearly moving objects in traffic scenarios, reduces the equipment acquisition frequency and data quality requirements, and reduces equipment costs.
Smart Images
Figure CN119399501A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of 3D target matching, in particular to a 3D target matching method based on trajectory intersection-and-union ratio. Background Art
[0002] In 3D target detection, the data collected at different times are still independent of each other after pre-labeling. It is necessary to further use matching methods to correlate and match the results at different times to form a complete target object trajectory. Therefore, different matching strategies also directly affect the matching accuracy. The current mainstream method is to match by calculating the IOU value. However, in 3D target detection in autonomous driving scenarios, the traditional IOU indicator matching effect is not ideal, and there are still some problems in actual use:
[0003] 1. Low matching accuracy: In 3D detection of traffic scenes, due to the fast movement speed of vehicles and the small detection result box of pedestrians, it is easy to cause no obvious IOU correlation between the detection results of the same target at different times, thereby ignoring the correspondence between the two, resulting in missing or wrong matching and low detection accuracy.
[0004] 2. Lack of robustness: The IOU metric cannot distinguish the alignment of two target boxes and is easily affected by noise and erroneous detection results. At the same time, in 3D target detection, the shape and orientation of objects such as people and cars are linear, and there is an obvious spatial relationship between the previous and next frames.
[0005] 3. High cost: The use of the IOU indicator requires that the time interval between the front and rear frames of the collected data is small to ensure that there is an obvious intersection between the front and rear detection frames. This requires that the acquisition frequency and data quality of sensor equipment such as lidar are high, resulting in high equipment and acquisition costs. For this reason, a 3D target matching method based on trajectory intersection and union ratio is designed. Summary of the invention
[0006] The purpose of the present invention is to solve the above problems and design a 3D target matching method based on trajectory intersection and union ratio.
[0007] To achieve the above purpose, the technical solution of the present invention is a 3D target matching method based on trajectory intersection and union ratio, comprising the following steps:
[0008] Step 1: Point cloud data detection;
[0009] Step 2: Global coordinate transformation;
[0010] Step 3: GIOU preliminary matching;
[0011] Step 4: T-IOU redistribution trajectory.
[0012] As a further description of the present technical solution, in the step one, the main body of the point cloud data detection is the large model of the point cloud detection algorithm, and the action object of the point cloud data detection is the point cloud data. Through the point cloud data detection, the target object frame information of the model detection is obtained, and the point cloud data detection obtains the detection results of each frame for subsequent matching and tracking.
[0013] As a further description of the present technical solution, the detailed process of step one is as follows: obtaining the original point cloud data, which is usually obtained by laser radar scanning, and includes the three-dimensional coordinates xyz of the point, reflection intensity, and collection time point information; through the 3D point cloud detection large model, the 3D box information of objects such as vehicles and pedestrians contained in the point cloud can be preliminarily identified from the point cloud data.
[0014] As a further description of the present technical solution, in the step 2, the subject of the global coordinate transformation is the acquisition posture information of each frame data, the action object of the global coordinate transformation is the pre-detection target frame, the global coordinate transformation obtains the position coordinates of each frame in the same coordinate system, and the global coordinate transformation splices the detection frames at different times into continuous detection frames of the entire acquisition section in the same coordinate system, thereby calculating the complete target trajectory.
[0015] As a further description of the present technical solution, the detailed process of step two is as follows: point cloud data is usually collected continuously by the vehicle-mounted radar at certain time intervals during driving. Due to the movement of the vehicle and the bumps in driving, the vehicle-mounted radar usually has different postures at different times; for each frame of point cloud data, while collecting the point cloud, the data will synchronously record the radar pose information at that time, and use this information to convert the detection results of each frame.
[0016] As a further description of the present technical solution, in the step three, the subject of the GIOU preliminary matching is the GIOU algorithm, and the action object of the GIOU preliminary matching is the 3D detection box; the GIOU preliminary matching obtains a preliminary matching result, and the GIOU preliminary matching obtains a preliminary target object trajectory for subsequent trajectory optimization.
[0017] As a further description of the technical solution, the detailed process of step three is as follows: calculate the GIOU index of each detection box of the previous and next frames, set a threshold, regard the previous and next frame results that meet the threshold as the same target object, and save them as a trajectory. If there is no matching result for a certain trajectory, use Kalman filtering to predict a position and reduce the confidence of the position. When the confidence is lower than the threshold, truncate the trajectory; if there are multiple matching results for a certain trajectory, save this result as a disputed match and re-match it in the following steps.
[0018] As a further description of the present technical solution, in the step 4, the main body of the T-IOU reallocation trajectory is the T-IOU algorithm, the action object of the T-IOU reallocation trajectory is the trajectory obtained by preliminary matching, the T-IOU reallocation trajectory obtains a more accurate target object trajectory, and the fine-grained target object trajectory is obtained by the T-IOU reallocation trajectory.
[0019] As a further description of the technical solution, the detailed process of step four is as follows: For controversial results in GIOU, the T-IOU indicator is used to recalculate the correlation between the detection box and the trajectory. Compared with G-IOU which only uses the results of the previous and next frames, T-IOU combines the entire trajectory with the current frame results for judgment, which can more accurately reflect the correlation between the detection box and the trajectory.
[0020] Its beneficial effect lies in that, starting from actual data and applications, this technical solution designs a 3D frame matching method combining GIOU indicator and TIOU indicator for the characteristics of linear movement of objects such as people and vehicles in traffic scenes and smooth trajectories, so that the trajectory generated by matching is more accurate and coherent; this method first uses the GIOU indicator for matching. Compared with the traditional IOU indicator, this indicator takes into account the non-overlapping part between the two target frames and can reflect the alignment of the two, which is suitable for objects moving linearly in traffic scenes; for controversial detection frames that can simultaneously match multiple adjacent trajectories during the GIOU matching process, T-IOU is used for re-matching so that the target object can be matched to the correct trajectory. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 is a flow chart of the present invention;
[0022] Figure 2 It is the intersection graph of the to-be-matched frame and the trajectory of the present invention. DETAILED DESCRIPTION
[0023] The present invention will be described in detail below in conjunction with the accompanying drawings. Figure 1 As shown, a 3D target matching method based on trajectory intersection and union ratio includes the following steps:
[0024] Step 1: point cloud data detection; in step 1, the main body of point cloud data detection is the point cloud detection algorithm model, and the action object of point cloud data detection is point cloud data. Through point cloud data detection, the target object frame information of model detection is obtained, and the point cloud data detection obtains the detection results of each frame for subsequent matching and tracking.
[0025] The detailed process of step one is as follows: obtain the original point cloud data, which is usually obtained by laser radar scanning, and includes the three-dimensional coordinates xyz, reflection intensity, and acquisition time point information of the point; through the 3D point cloud detection large model, the 3D frame information of objects such as vehicles and pedestrians contained in the point cloud can be preliminarily identified from the point cloud data.
[0026] The original point cloud data is usually a pcd file or a bin file, which contains the three-dimensional coordinate values of all points scanned by the radar, point cloud intensity and other features; then, these data are input into the algorithm model, and the model determines whether these points belong to the target to be detected based on the shape of the point cluster, the intensity of the points and other information. For the detected targets, the algorithm gives the category attributes of the target's 3D box, the coordinates of the center point, the length, width, height and yaw angle.
[0027] Step 2: global coordinate conversion; in step 2, the subject of the global coordinate conversion is the acquisition posture information of each frame data, the action object of the global coordinate conversion is the pre-detection target frame, the global coordinate conversion obtains the position coordinates of each frame in the same coordinate system, and the global coordinate conversion splices the detection frames at different times into continuous detection frames of the entire acquisition section in the same coordinate system, thereby calculating the complete target trajectory.
[0028] The detailed process of step 2 is as follows: point cloud data is usually collected continuously by the vehicle-mounted radar at certain time intervals during driving. Due to the movement of the vehicle and the bumps in driving, the vehicle-mounted radar usually has different postures at different times; for each frame of point cloud data, while collecting the point cloud, the data will synchronously record the radar pose information at that time, and use this information to convert the detection results of each frame.
[0029] If the attitude angles φ, θ and γ of a certain frame of radar correspond to the angles generated by yaw, pitch and roll respectively, and P0 is the coordinate of the radar in the ground coordinate system, then the coordinates of the 3D point cloud detection frame in the ground coordinate system are as shown in the formula:
[0030]
[0031] in,
[0032]
[0033] Step 3: GIOU preliminary matching; in step 3, the subject of GIOU preliminary matching is the GIOU algorithm, and the action object of GIOU preliminary matching is the 3D detection box; GIOU preliminary matching obtains preliminary matching results, and GIOU preliminary matching obtains preliminary target object trajectory for subsequent trajectory optimization.
[0034] The detailed process of step three is as follows: calculate the GIOU index of each detection frame of the previous and next frames, set a threshold, regard the previous and next frame results that meet the threshold as the same target object, and save them as a trajectory. If there is no matching result for a certain trajectory, use Kalman filtering to predict a position and reduce the confidence of the position. When the confidence is lower than the threshold, truncate the trajectory; if there are multiple matching results for a certain trajectory, save this result as a disputed match and rematch it in the following steps.
[0035] In 3D target tracking in traffic scenes, the target object usually moves on the road surface, so the height of the 3D detection frame is set to 0 to project it onto a 2D plane for matching to reduce the amount of calculation. The GIOU calculation method for two target frames on the plane is shown as follows:
[0036]
[0037] Among them, Ac is the minimum enclosed area of detection boxes A and B.
[0038] The GIoU matrix is calculated for all detection frames of two adjacent frames, and a GIoU threshold is set. When the GIoU value between two detection frames is greater than the given threshold, the two detection frames are considered to be matched successfully, that is, they are the same target object. If a detection frame has no matching detection frame in the next frame, the Kalman filter is used to predict its corresponding position in the next frame and save it. At the same time, the confidence of the current frame is attenuated by a coefficient of 0.9 as the confidence of the next frame; if a detection frame has multiple matching trajectories, the frame is recorded as a controversial state and enters the next step of matching.
[0039] Step 4: T-IOU reallocation trajectory. In step 4, the main body of the T-IOU reallocation trajectory is the T-IOU algorithm. The action object of the T-IOU reallocation trajectory is the trajectory obtained by preliminary matching. The T-IOU reallocation trajectory obtains a more accurate target trajectory. The T-IOU reallocation trajectory is used to obtain a fine-grained target trajectory.
[0040] The detailed process of step four is as follows: For controversial results in GIOU, the T-IOU indicator is used to recalculate the correlation between the detection box and the trajectory. Compared with G-IOU which only uses the results of the previous and next frames, T-IOU combines the entire trajectory with the current frame results for judgment, which can more accurately reflect the correlation between the detection box and the trajectory.
[0041] For the detection boxes that are controversial during the GIoU matching process, they are brought into the corresponding tracks and T-IoU is calculated, as shown in the formula:
[0042]
[0043] Among them, Sa and Sb are the time step indicators of tracks Ta and Tb respectively, and Bi is the i-th box in the overlapping part of the two tracks. It can be seen from the formula that TIoU is actually the average IoU of each pair of boxes divided by the union length of the two tracks. Matching is performed according to the maximum value of TIoU to make the detection box more consistent with the track characteristics.
[0044] like Figure 2 As shown in the figure, at time t+1, the to-be-matched box b1 has intersections with both trajectories, which is recorded as a controversial match. Since the GIOU value calculated by b1 and trajectory T2 is larger, if the traditional matching method is used to match directly according to the size of the GIOU value, the to-be-matched box b1 will successfully match with trajectory T2, and an incorrect matching result will be obtained at this time; however, if the overall T-IOU is calculated to combine the single frame with the overall trajectory, the T-IOU value of the to-be-matched box b1 and trajectory 1 is larger at this time, and the box can be correctly matched with the corresponding trajectory; at the same time, for undisputed matching situations, T-IOU will not change the original correct matching result based on GIOU, so the overall matching effect is better than the traditional method.
[0045] This technical solution starts from actual data and applications. Aiming at the characteristics of linear movement of objects such as people and vehicles in traffic scenes and smooth trajectories, a 3D box matching method combining GIOU indicators and TIOU indicators is designed to make the matching generated trajectory more accurate and coherent. This method first uses the GIOU indicator for matching. Compared with the traditional IOU indicator, this indicator takes into account the non-overlapping part between the two target boxes and can reflect the alignment of the two. It is suitable for objects moving linearly in traffic scenes. For controversial detection boxes that can simultaneously match multiple adjacent trajectories during the GIOU matching process, T-IOU is used for re-matching so that the target object can be matched to the correct trajectory.
[0046] The above technical solutions only reflect the preferred technical solutions of the technical solutions of the present invention. Some changes that may be made to certain parts thereof by technicians in this technical field all reflect the principles of the present invention and fall within the protection scope of the present invention.
Claims
1. A 3D target matching method based on trajectory intersection and union ratio, characterized in that: The following steps are involved: Step 1: Point cloud data detection; Step 2: Global coordinate transformation; Step 3: GIOU preliminary matching; Step 4: T-IOU redistribution trajectory.
2. A 3D target matching method based on trajectory intersection and union ratio according to claim 1, characterized in that: In the step one, the subject of point cloud data detection is the large model of point cloud detection algorithm, and the action object of point cloud data detection is point cloud data. Through point cloud data detection, the target object frame information of model detection is obtained, and the point cloud data detection obtains the detection results of each frame for subsequent matching and tracking.
3. The 3D target matching method based on trajectory intersection and union ratio according to claim 1, characterized in that: The detailed process of step one is as follows: obtain the original point cloud data, which is usually obtained by laser radar scanning, and includes the three-dimensional coordinates xyz, reflection intensity, and acquisition time point information of the point; through the 3D point cloud detection large model, the 3D frame information of objects such as vehicles and pedestrians contained in the point cloud can be preliminarily identified from the point cloud data.
4. The 3D target matching method based on trajectory intersection and union ratio according to claim 1, characterized in that: In the step 2, the subject of the global coordinate transformation is the acquisition posture information of each frame data, the action object of the global coordinate transformation is the pre-detection target frame, the global coordinate transformation obtains the position coordinates of each frame in the same coordinate system, and the global coordinate transformation splices the detection frames at different times into continuous detection frames of the entire acquisition section in the same coordinate system, thereby calculating the complete target trajectory.
5. The 3D target matching method based on trajectory intersection and union ratio according to claim 1, characterized in that: The detailed process of step 2 is as follows: point cloud data is usually collected continuously by the vehicle-mounted radar at certain time intervals during driving. Due to the movement of the vehicle and the bumps in driving, the vehicle-mounted radar usually has different postures at different times; for each frame of point cloud data, while collecting the point cloud, the data will synchronously record the radar pose information at that time, and use this information to convert the detection results of each frame.
6. The 3D target matching method based on trajectory intersection and union ratio according to claim 1, characterized in that: In the step three, the subject of GIOU preliminary matching is the GIOU algorithm, and the action object of GIOU preliminary matching is the 3D detection box; GIOU preliminary matching obtains preliminary matching results, and GIOU preliminary matching obtains preliminary target object trajectory for subsequent trajectory optimization.
7. The 3D target matching method based on trajectory intersection and union ratio according to claim 1, characterized in that: The detailed process of step three is as follows: calculate the GIOU index of each detection frame of the previous and next frames, set a threshold, regard the previous and next frame results that meet the threshold as the same target object, and save them as a trajectory. If there is no matching result for a certain trajectory, use Kalman filtering to predict a position and reduce the confidence of the position. When the confidence is lower than the threshold, truncate the trajectory; if there are multiple matching results for a certain trajectory, save this result as a disputed match and rematch it in the following steps.
8. The 3D target matching method based on trajectory intersection and union ratio according to claim 1, characterized in that: In the step 4, the main body of the T-IOU reallocation trajectory is the T-IOU algorithm, the action object of the T-IOU reallocation trajectory is the trajectory obtained by preliminary matching, the T-IOU reallocation trajectory obtains a more accurate target trajectory, and the T-IOU reallocation trajectory obtains a fine-grained target trajectory.
9. The 3D target matching method based on trajectory intersection and union ratio according to claim 1, characterized in that: The detailed process of step four is as follows: For controversial results in GIOU, the T-IOU indicator is used to recalculate the correlation between the detection box and the trajectory. Compared with G-IOU which only uses the results of the previous and next frames, T-IOU combines the entire trajectory with the current frame results for judgment, which can more accurately reflect the correlation between the detection box and the trajectory.