3D target trajectory automatic labeling method for point cloud denoising and trajectory verification

Abstract

The application discloses a 3D target trajectory automatic labeling method for point cloud denoising and trajectory verification, first carries out data enhancement on continuous point cloud data, combines past point cloud frames with current point cloud frames by using time information, obtains various point cloud frame combination data, then adopts a 3D target detection algorithm to obtain a 3D target detection result of the point cloud frame combination data for weighted fusion, adjusts and filters by using a point density perception voxel network, carries out trajectory tracking based on a data observation and association strategy, generates an initial 3D target trajectory set in a positive direction and an initial 3D target trajectory set in a reverse direction, then obtains the 3D target trajectory set in the positive direction and the 3D target trajectory set in the reverse direction through trajectory legality verification, and carries out weighted fusion processing to obtain the 3D target trajectory set; the method has the advantages that the 3D target trajectory automatic labeling can be completely and accurately completed while the labeling efficiency is high and the labeling cost is low.

Description

Technical Field

[0001] This invention relates to a method for automatic annotation of 3D target trajectories, and more particularly to a method for automatic annotation of 3D target trajectories for point cloud denoising and trajectory verification. Background Technology

[0002] In the current wave of global technological innovation, autonomous driving technology is changing the existing landscape of the automotive industry and is rapidly becoming a core competitive technology in the automotive field. In recent years, national and governmental governments have attached great importance to the application and development of artificial intelligence technology in the automotive sector, and have introduced relevant policies to promote the development of autonomous driving technology.

[0003] Autonomous driving technology aims to create self-driving cars capable of accurately perceiving pedestrians, vehicles, and lane markings on the road, ultimately making autonomous decisions. Currently, we rely on large amounts of autonomous driving datasets and highly efficient deep learning algorithms to train accurate and stable autonomous driving algorithms. 3D target trajectory sets, as an important component of autonomous driving datasets, play a crucial role in the field. In autonomous driving scenarios, 3D target trajectories are presented through continuous point cloud data. The trajectory in continuous point cloud data is essentially a series of points that map the path of an object as it changes over time, reflecting spatial and temporal information. The annotation of 3D target trajectories involves annotating the paths or motion patterns of various objects in the driving environment. The annotation results directly affect the accuracy of the autonomous driving algorithm's prediction of target trajectories, thus influencing the perception and decision-making of autonomous vehicles.

[0004] 3D target trajectory annotation methods are mainly divided into manual annotation and automatic annotation. Manual annotation is time-consuming, labor-intensive, inefficient, and costly. Automatic annotation uses automatic annotation algorithms, and most existing automatic annotation algorithms follow a paradigm that combines detection (implemented by 3D target detection algorithms) and tracking (implemented by tracking algorithms). Although automatic annotation is time-saving, labor-saving, efficient, and cost-effective, the accuracy and speed of 3D target detection algorithms are constantly improving. When tracking algorithms are combined with 3D target detection algorithms, they are easily affected by noise generated during the detection phase of the 3D target detection algorithm. This reduces the accuracy of the tracking algorithm in estimating the motion state parameters of occluded or distant objects, causing the annotated trajectory to deviate from the actual motion trajectory of the 3D target. Furthermore, unverified trajectories can lead to fragmented trajectories, false positive trajectories, and ID switching, ultimately resulting in automatic annotation failing to generate sufficiently complete and accurate target trajectories. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a method for automatic annotation of 3D target trajectory that can complete point cloud denoising and trajectory verification with high annotation efficiency and low annotation cost.

[0006] The technical solution adopted by this invention to solve the above-mentioned technical problems is: an automatic annotation method for 3D target trajectories for point cloud denoising and trajectory verification, comprising the following steps:

[0007] Step 1: Perform data augmentation on the continuous point cloud data to obtain the augmented continuous point cloud data;

[0008] Step 2: Using time information, combine past point cloud frames with the current point cloud frame in X ways to obtain X types of point cloud frame combination data, where X is an integer greater than or equal to 2; and use a 3D target detection algorithm to obtain the 3D target detection results of the X types of point cloud frame combination data.

[0009] Step 3: Perform weighted fusion of the X 3D object detection results to obtain the initial 3D object detection result;

[0010] Step 4: Adjust and filter the initial 3D target detection results using a point density-sensing voxel network to obtain the adjusted 3D target detection results;

[0011] Step 5: Based on data observation and association strategies, perform trajectory tracking on the adjusted 3D target detection results to generate a forward initial 3D target trajectory set and a reverse initial 3D target trajectory set, wherein the forward initial 3D target trajectory set and the reverse initial 3D target trajectory set are each composed of multiple 3D target trajectories;

[0012] Step 6: Perform trajectory validity verification on each 3D target trajectory in the two initial 3D target trajectory sets, namely the forward initial 3D target trajectory set and the reverse initial 3D target trajectory set (if a 3D target trajectory fails the verification, it is considered a false positive trajectory). The 3D target trajectories that pass the validity verification in the forward initial 3D target trajectory set are used to form the forward 3D target trajectory set, and the 3D target trajectories that pass the validity verification in the reverse initial 3D target trajectory set are used to form the reverse 3D target trajectory set.

[0013] Step 7: Perform weighted fusion processing on the forward-oriented 3D target trajectory set and the reverse-oriented 3D target trajectory set to obtain the final 3D target trajectory set. The 3D target trajectory is automatically labeled.

[0014] Compared with existing technologies, the advantages of this invention lie in its introduction of multi-frame point cloud combination, utilizing data from the current frame and multiple past frames to form point cloud frame combination data, providing more spatial and temporal information, enhancing the detection capability of distant or occluded targets. Furthermore, a weighted fusion method is employed to improve detection accuracy and reduce false detection rate. A point density-aware voxel network is used to adjust and filter the initial 3D target detection results, obtaining more accurate 3D target bounding boxes, reducing noise in the 3D target detection stage, and improving the accuracy of 3D target detection results. Data observation and association strategies further filter point cloud noise in the 3D target detection results, improving the accuracy of spatial position and motion parameter state estimation for occluded or distant objects, thereby avoiding fragmented trajectories caused by deviations between the subsequently labeled trajectory and the actual motion trajectory of the target. Through forward and reverse target trajectory... The trajectory can filter out low-quality matches, ensuring that the trajectory is highly consistent with the real motion trajectory, and further filtering point cloud noise. By verifying the legality of the 3D target trajectory, the dependence on fixed frame number and threshold is reduced, effectively reducing false positive trajectories and improving the accuracy of 3D target trajectory tracking. Finally, the final 3D target trajectory is obtained by weighted fusion of the forward 3D target trajectory set and the reverse 3D target trajectory set, avoiding the problem that the 3D target disappears in a certain period of time and cannot form a complete trajectory. Moreover, the reverse 3D target trajectory set can also effectively eliminate the influence of errors caused by historical states on the forward 3D target trajectory set, improving the efficiency and accuracy of automatic annotation of 3D target trajectory. Thus, this invention can complete the automatic annotation of 3D target trajectory completely and accurately while having high annotation efficiency and low annotation cost.

[0015] Furthermore, the specific process of performing data augmentation on the continuous point cloud data in step 1 to obtain the augmented continuous point cloud data is as follows:

[0016] Step S1.1: Denote the continuous point cloud data as P, and denote the i-th point cloud frame in the continuous point cloud data P as Pi. i In the continuous point cloud data, the i-th point cloud frame P i For the data obtained from the i-th scan of the lidar, the i-th point cloud frame P in the continuous point cloud data P is... i The time of being scanned by lidar is recorded as Where i = 1, 2, ..., I, I is the total number of point cloud frames in the continuous point cloud data P;

[0017] Step S1.2: Extract the i-th point cloud frame P from the continuous point cloud data P. i The timestamp is T i T is calculated using formula (1). i :

[0018]

[0019] Where, Δt m It is the i-th point cloud frame P in continuous point cloud data i With the m-th point cloud frame P m The time difference of being scanned by the lidar, m = 1, 2, ..., i; by obtaining the timestamp of each point cloud frame in the continuous point cloud data, the time of each point cloud frame in the continuous point cloud data can be distinguished, which helps the subsequent algorithm to locate 3D targets and achieve velocity prediction;

[0020] Step S1.3: From the ground truth data P of the continuous point cloud data P gt N true values ​​are selected from the data, where N is greater than or equal to 5 and less than or equal to 10. The N true values ​​include V1 true values ​​of vehicles, V2 true values ​​of pedestrians, and V3 true values ​​of cyclists, where V1, V2, and V3 are integers greater than or equal to 0. After performing translation and rotation operations on all N true values, they are randomly added to the i-th point cloud frame P of the continuous point cloud data P. i In the continuous point cloud data P, for the i-th point cloud frame P i Enhancement is performed by taking the i-th point cloud frame P of the continuous point cloud data P. i The data before and after the increase is denoted as P. aug i, P aug i represents the enhanced continuous point cloud data P. aug The i-th point cloud frame is used to enrich the 3D target information of the point cloud data;

[0021] Furthermore, the xth type of point cloud frame combination data obtained in step 2 is denoted as... x = 1, 2, ..., X, for the enhanced continuous point cloud data P aug Perform the xth combination to obtain the xth point cloud frame combination data. The specific process is as follows: from the enhanced continuous point cloud data P aug The first point cloud frame P aug 1. Start by selecting a point cloud frame each time. Determine if there are x-1 more point cloud frames after the selected point cloud frame. If there are, combine the selected point cloud frame with the following x-1 point cloud frames to form a point cloud frame combination of the xth method. If there are no points cloud frames, the xth method combination ends. All point cloud frame combinations of the xth method constitute the xth point cloud frame combination data.

[0022] Furthermore, in step 3, the X 3D target detection results are weighted and fused using formula (2) to obtain the initial 3D target detection result D. final :

[0023] D final =WBF(D1,D2,D3,…,D)X (2)

[0024] Where WBF(·) represents the weighted fusion function, D x The x-th point cloud frame combination data obtained using a 3D object detection algorithm The 3D target detection results are obtained by weighting the position and confidence of different detection boxes to improve the detection accuracy of 3D targets and reduce the false detection rate. A more accurate initial 3D target detection result is obtained from the 3D target detection results of X point cloud frame combination data.

[0025] Furthermore, in step 4, a point density-sensing voxel network thinning function is used to refine the initial 3D target detection result D after fusion in step 3. final Adjusting the detection bounding box in the D array yields more accurate 3D target detection results. refinal Among them, the 3D target detection result D refinal It consists of the 3D target detection results of I point cloud frames of continuous point cloud data P.

[0026] Furthermore, in step 5, the adjusted 3D target detection result D is adjusted based on data observation and association strategies. refinal The specific process of performing trajectory tracking and generating a positive initial set of 3D target trajectories is as follows:

[0027] Step S6.1: Set the data observation threshold to α. conf To avoid system overload, the validity of the observation results will be evaluated during the trajectory legality verification phase. Data observation is a 3D target recognition step, the main purpose of which is to allow the observation of low-detection-score objects at a distance and with partial occlusion, while preventing a large number of low-confidence 3D targets from overloading the system, where 0.1≤α conf ≤0.3;

[0028] Step S6.2: Convert the 3D target detection results D refinal The i-th point cloud frame P i The 3D object detection result is denoted as 3D object detection results The number of 3D targets is denoted as O. i 3D target detection results D Pi The o i Each 3D target is denoted as Among them o i =1, 2, 3, ..., O i 3D target detection results D refinal 3D object detection results of the first point cloud frame P1 in the data. The timestamps of all 3D targets are equal to the timestamp of the first point cloud frame P1. This applies to the 3D target detection result D. refinal 3D object detection results of the first point cloud frame P1 in the data. All 3D targets are initialized with trajectories to obtain their 3D target trajectories. At this point, the target detection result D is obtained. refinal 3D object detection results of the first point cloud frame P1 in the data. Each 3D target's 3D trajectory is an initial trajectory, and the target detection results D are... refinal In the 3D object detection results of each point cloud frame, the number of successful matches for each 3D object trajectory is initialized to 0, and the number of failed matches is initialized to 0. The 3D object detection results D... refinal 3D object detection results of the first point cloud frame P1 in the data. The 3D target trajectories of all 3D targets constitute the 0th generation 3D target trajectory set T0, where This represents the o1-th 3D target trajectory in the 0th generation 3D target trajectory set T0, i.e., the 3D target detection result D. refinal 3D object detection results of the first point cloud frame P1 in the data. The 3D trajectory of the o1th 3D target;

[0029] Step S6.3: Set the iteration count variable r, the maximum number of iterations R, and the matching threshold n. min Threshold IoU thres and GIoU thres Where R = I⁻¹, n min =3,n max =5, 0.25≤IoU thres ≤0.35, 0.1≤GIoU thres If r is less than or equal to 0.2, initialize r to 1.

[0030] Step S6.4: Perform the r-th iteration to obtain the r-th generation 3D target trajectory set T r The specific process is as follows:

[0031] Step S6.4.1: Set the 3D target trajectory T of the (r-1)th generation r-1 The number of 3D target trajectories is denoted as L. r-1 Set the variable to l r and q r+1 , will l r Initialize to 1;

[0032] Step S6.4.2: q r+1 Initialize to 1;

[0033] Step S6.4.3: For the (r-1)th generation 3D target trajectory set T r-1 The first one r 3D target trajectory The updated 3D target trajectory is obtained and used as the r-th generation 3D target trajectory set T. r The l r 3D target trajectory Added to the r-th generation 3D target trajectory set T r The specific process is as follows:

[0034] Step S6.4.3.1: Determine the 3D target detection result The qth r+1 3D target Is the confidence score greater than α? conf If it is greater than α conf If so, proceed directly to step S6.4.3.2; otherwise, determine q. r+1 Is the current value equal to O? r+1 If q r+1 The current value is not equal to 0. r+1 Then q is used. r+1 The sum of the current value plus 1 updates q r+1 The value of q is then returned to step S6.4.3.1. If q r+1 The current value is equal to O. r+1 Then first set the 3D target trajectory T of the (r-1)th generation r-1 The first one r 3D target trajectory As the r-th generation 3D target trajectory set T r The l r 3D target trajectory Added to the r-th generation 3D target trajectory set T r In, and make the 3D target trajectory The number of successful matches equals the 3D target trajectory The current value of the number of successful matches, 3D target trajectory The number of matching failures equals the 3D target trajectory The sum of the current value of the number of failed matches plus 1 is then used to determine l. r Is the current value equal to L? r-1 If l r The current value is not equal to L r-1 Then first use l r The sum of the current value plus 1 updates l r The value of l is then used to return to step S6.4.2. If l r The current value is equal to L. r-1 Then proceed to step S6.4.3.7;

[0035] Step S6.4.3.2: Use a 3D Kalman filter (3DKF) to process the (r-1)th generation 3D target trajectory set T r-1 The first one r 3D target trajectory Make a prediction to obtain the corresponding 3D target state, and record it as...

[0036] Step S6.4.3.3: Calculate the 3D target detection results using formulas (3) and (4) respectively. The qth r+1 3D target Intersection over Union (IoU) of middle bounding boxes and generalized intersection

[0037]

[0038] in, This represents the 3D object detection result. The qth r+1 3D target With 3D target state The volume of the intersection Indicates 3D object detection results The qth r+1 3D target With 3D target state The union volume, This indicates that the results include 3D object detection. The qth r+1 3D target With 3D target state The minimum enclosed volume;

[0039] Step S6.4.3.4: If Less than IoU thres or Less than GIoU thres This indicates the 3D object detection result. The qth r+1 3D target and 3D target state Matching failed; 3D object detection results will be returned. The qth r+1 3D target For 3D targets that fail to match, step S6.4.3.6 is executed at this time;

[0040] if Greater than or equal to IoUthres or Greater than or equal to GIoU thres This indicates the 3D object detection result. The qth r+1 3D target and 3D target state Matching successful, 3D object detection results will be displayed. The qth r+1 3D target As the 3D target that has been successfully matched, proceed to step 6.4.3.5;

[0041] Step S6.4.3.5: Set the 3D target detection results... The qth r+1 3D target The timestamp is equal to the (r+1)th point cloud frame P r+1 Timestamps and 3D object detection results The qth r+1 3D target The 3D Kalman filter is updated to obtain the 3D target trajectory, which is the set of 3D target trajectories T for the r-th generation. r The l r 3D target trajectory And make the 3D target trajectory The number of successful matches is equal to the set of 3D target trajectories T of generation r-1. r-1 The first one r 3D target trajectory The sum of the current number of successful matches plus 1, 3D target trajectory The number of matching failures is equal to the set of 3D target trajectories T of generation r-1. r-1 The first one r 3D target trajectory The number of failed matches, to determine l r Is the current value equal to L? r-1 If l r The current value is equal to L. r-1 Then first use l r The sum of the current value plus 1 updates l r The value of l is then used to return to step S6.4.2. If l r The current value is equal to L. r-1 Then proceed to step S6.4.3.7;

[0042] Step S6.4.3.6: Determine q r+1 Is the current value equal to O? r+1 If q r+1 The current value is not equal to 0. r+1 Then q is used.r+1 The sum of the current value plus 1 updates q r+1 The value of q is then returned to step S6.4.3.1. If q r+1 The current value is equal to O. r+1 Then first set the 3D target trajectory T of the (r-1)th generation r-1 The first one r 3D target trajectory As the r-th generation 3D target trajectory set T r The l r 3D target trajectory Added to the r-th generation 3D target trajectory set T r In, and make the 3D target trajectory The number of successful matches equals the 3D target trajectory The current value of the number of successful matches, 3D target trajectory The number of matching failures equals the 3D target trajectory The sum of the current value of the number of failed matches plus 1 is then used to determine l. r Is the current value equal to L? r-1 If l r The current value is not equal to L r-1 Then use l r The sum of the current value plus 1 updates l r The value of l is then used to return to step S6.4.2. If l r The current value is equal to L. r-1 Then proceed to step S6.4.3.7;

[0043] Step S6.4.3.7: Analyze the 3D target detection results. Each 3D target that fails to match is evaluated individually. If the 3D target detection result is... The confidence score of a 3D target that failed to match is higher than α. conf Then the timestamp of the 3D target is equal to the (r+1)th point cloud frame P. r+1 The timestamp is used to initialize the trajectory of the 3D target, resulting in an initialized trajectory as the 3D target trajectory of the 3D target. This 3D target trajectory is then used as the set of 3D target trajectories for the r-th generation. r A 3D target trajectory is added to the r-th generation 3D target trajectory set T r The Lth r-1 After identifying a 3D target trajectory, the number of successful matches and the number of failed matches for that 3D target trajectory are initialized to 0.

[0044] Step S6.4.3.8: For the currently obtained r-th generation 3D target trajectory set T rEach of the 3D target trajectories in the set is judged individually. If the set of 3D target trajectories in the r-th generation is T... r The number of successful matches of a certain 3D target trajectory is less than a threshold n min And the number of failed matches is greater than the threshold n max Then the 3D target trajectory will be removed from the set of 3D target trajectories T of generation r. r Delete it if necessary, otherwise do not process it.

[0045] Step S6.5: Determine if the current value of r is equal to R. If not, update the value of r by adding 1 to the current value of r, and then return to step S6.4 for the next iteration. If it is equal to R, then the currently obtained R-th generation 3D target trajectory set T is... R That is, the initial set of 3D target trajectories T after positive transformation. forward .

[0046] Furthermore, in step 5, the adjusted 3D target detection result D is adjusted based on data observation and association strategies. refinal The specific process of performing trajectory tracking and generating a reversed initial set of 3D target trajectories is as follows:

[0047] Step S6.6: Let the 3D object detection result D refinal The i-th point cloud frame P I 3D object detection results The timestamps of all 3D objects are equal to the i-th point cloud frame. The timestamp for the 3D object detection result D refinal The i-th point cloud frame P in I 3D object detection results All 3D targets are initialized with trajectories to obtain their 3D target trajectories. At this point, the target detection result D is obtained. refinal The i-th point cloud frame P I 3D object detection results Each 3D target's 3D trajectory is an initial trajectory, and the target detection results D are... refinal In the 3D object detection results of each point cloud frame, the number of successful matches for each 3D object trajectory is initialized to 0, and the number of failed matches is initialized to 0. The 3D object detection results D... refinal The i-th point cloud frame P I 3D object detection results The 3D target trajectories of all 3D targets constitute the 0th generation 3D target trajectory set T0', where This represents the o-th trajectories in the 0th generation 3D target trajectory set T0. I A 3D target trajectory, i.e., the 3D target detection result D refinalThe i-th point cloud frame P I 3D object detection results The o I 3D target trajectory of a 3D target;

[0048] Step S6.7: Set the iteration count variable r', the maximum iteration count R, and the matching threshold n. min Threshold IoU thres and GIoU thres Where R = I⁻¹, n min =3,n max =5, 0.25≤IoU thres ≤0.35, 0.1≤GIoU thres If ≤0.2, initialize r' to 1;

[0049] Step S6.8: Perform the r'-th iteration to obtain the r'-th generation 3D target trajectory set T". The specific process is as follows:

[0050] Step S6.8.1: Set the 3D target trajectory set T' of the r'-1th generation. r'-1 The number of 3D target trajectories is denoted as L' r'-1 Set the variable to l' r' and q' I-r' , will l' r' Initialize to 1;

[0051] Step S6.8.2: q' I-r' Initialize to 1;

[0052] Step S6.8.3: For the r'-1th generation 3D target trajectory set T' r'-1 The l' r' 3D target trajectory The updated 3D target trajectory is obtained and used as the l-th 3D target trajectory in the r'-th generation 3D target trajectory set T”. The specific process of adding it to the 3D target trajectory set T of generation r' is as follows:

[0053] Step S6.8.3.1: Determine the 3D target detection result The q' I-r' 3D target Is the confidence score greater than α? conf If it is greater than α conf If yes, proceed directly to step S6.8.3.2; otherwise, determine q'. I-r' Is the current value equal to O? I-r' If q' I-r' The current value is not equal to 0. I-r' Then q' I-r'The sum of the current value plus 1 updates q' I-r' The value of q' is then returned to step S6.8.3.1. If q' I-r' The current value is equal to O. I-r' Then first set the 3D target trajectory T' of the r'-1 generation. r'-1 The l' r' 3D target trajectory As the lth 3D target trajectory of the r'th generation 3D target trajectory set T",

[0054] Added to the 3D target trajectory set T' of generation r' r' In, and make the 3D target trajectory The number of successful matches equals the 3D target trajectory The current value of the number of successful matches, 3D target trajectory The number of matching failures equals the 3D target trajectory The sum of the current value of the number of failed matches plus 1 is then used to determine l'. r' Is the current value equal to L'? r'-1 If l' r' The current value is not equal to L' r'-1 Then l' is used first. r' The sum of the current value plus 1 updates L' r'-1 The value of l' is then determined, and the process returns to step S6.8.2. If l' r' The current value is equal to L' r'-1 Then proceed to step S6.8.3.7;

[0055] Step S6.8.3.2: Use a 3D Kalman filter to process the set of 3D target trajectories T' of the r'-1th generation. r'-1 The l' r' 3D target trajectory Make a prediction to obtain the corresponding 3D target state, and record it as...

[0056] Step S6.8.3.3: Calculate the 3D target detection results using formulas (5) and (6) respectively. The q' I-r' 3D target Intersection over Union (IoU) of middle bounding boxes and generalized intersection

[0057]

[0058] in, This represents the 3D object detection result. The q' I-r' 3D target With 3D target state The volume of the intersection Indicates 3D object detection results The q' I-r' 3D target With 3D target state The union volume, This indicates that the results include 3D object detection. The q' I-r' 3D target With 3D target state The minimum enclosed volume;

[0059] Step S6.8.3.4: If Less than IoU thres or Less than GIoU thres This indicates the 3D object detection result. The q' I-r' 3D target and 3D target state Matching failed; 3D object detection results will be returned. The q' I-r' 3D target For 3D targets that fail to match, step S6.8.3.6 is executed at this time;

[0060] if Greater than or equal to IoU thres or Greater than or equal to GIoU thres This indicates the 3D object detection result. The q' I-r' 3D target and 3D target state Matching successful, 3D object detection results will be displayed. The q' I-r' 3D target As the 3D target that has been successfully matched, proceed to step 6.8.3.5;

[0061] Step S6.8.3.5: Set the 3D target detection results... The q' I-r' 3D target The timestamp is equal to the I-r'th point cloud frame P I-r' Timestamps and 3D object detection results The q' I-r' 3D target The 3D Kalman filter is updated to obtain the 3D target trajectory, which is the set of 3D target trajectories T for the r'th generation. r’ The l” r’ 3D target trajectory And make the 3D target trajectory The number of successful matches is equal to the set of 3D target trajectories T' of generation r'-1. r'-1 The l' r' 3D target trajectory The sum of the current number of successful matches plus 1, 3D target trajectory of

[0062] The number of matching failures is equal to the set of 3D target trajectories T' of generation r'-1. r'-1 The l' r' 3D target trajectory Number of match failures

[0063] The current value of the number, determine l' r' Is the current value equal to L'? r'-1 If l' r' The current value is equal to L' r'-1 Then l' is used first. r' The sum of the current value plus 1 updates L' r'-1 The value of l' is then determined, and the process returns to step S6.8.2. If l' r' The current value is equal to L' r'-1 Then proceed to step S6.8.3.7;

[0064] Step S6.8.3.6: Determine q' I-r' Is the current value equal to O? I-r' If q' I-r' The current value is not equal to 0. I-r' Then q' I-r' The sum of the current value plus 1 updates q' I-r' The value of q' is then returned to step S6.8.3.1. If q' I-r' The current value is equal to

[0065] O I-r' Then first set the 3D target trajectory T' of the r'-1 generation. r'-1 The l' r' 3D target trajectory As the r' generation of 3D

[0066] Trajectory set T' r' The l' r' 3D target trajectory Added to the 3D target trajectory set T' of generation r' r' In the middle, and make 3D

[0067] Target trajectory The number of successful matches equals the 3D target trajectory The current value of the number of successful matches, 3D target trajectory

[0068] trace The number of matching failures equals the 3D target trajectory The sum of the current value of the number of failed matches plus 1 is then used to determine l'. r' Is the current value equal to L'? r'-1 If l' r' The current value is not equal to L' r'-1 Then l' is used first. r' The sum of the current value plus 1 updates L' r'-1 The value of l' is then determined, and the process returns to step S6.8.2. If l' r' The current value is equal to L' r'-1 Then proceed to step S6.8.3.7;

[0069] Step S6.8.3.7: Analyze the 3D target detection results. Each 3D target that fails to match is evaluated individually. If the 3D target detection result is... The confidence score of a 3D target that failed to match is higher than α. conf Then the timestamp of the 3D target is equal to the I-r'th point cloud frame P. I-r' The timestamp is then used to initialize the trajectory of the 3D target, resulting in an initialized trajectory as the 3D target trajectory of the 3D target. This 3D target trajectory is then used as the set of 3D target trajectories for the r'th generation T'. r' A 3D target trajectory is added to the 3D target trajectory set T' of generation r'. r' The L' r'-1 After identifying a 3D target trajectory, the number of successful matches and the number of failed matches for that 3D target trajectory are initialized to 0.

[0070] Step S6.8.3.8: Perform the following steps on all 3D target trajectories in the currently obtained r'-th generation 3D target trajectory set T”:

[0071] Determine if the set of 3D target trajectories in generation r' is T' r' The number of successful matches of a certain 3D target trajectory is less than a threshold n min And the number of failed matches is greater than the threshold n max Then the 3D target trajectory will be removed from the 3D target trajectory set T' of generation r'. r' Delete it if necessary, otherwise do not process it.

[0072] Step S6.9: Determine if the current value of r' is equal to R. If not, update the value of r' by adding 1 to the current value of r', and then return to step S6.8 for the next iteration. If it is equal to R, then the currently obtained R-th generation 3D target trajectory set T' R That is, the inversely converted initial 3D target trajectory set T backward .

[0073] Furthermore, in step 6, the initial 3D target trajectory set T is positively oriented. forward The specific process for verifying the legality of each 3D target trajectory is as follows:

[0074] Step S7.1: Convert the initial 3D target trajectory set T into a forward-oriented form. forward The number of 3D target trajectories is denoted as L. forward Let T' be the set of positively oriented 3D target trajectories. forward Legality threshold α legality Threshold α cov , where 0.5 ≤ α legality ≤0.6, 1.5≤α cov If ≤2.0, initialize h to 1;

[0075] Step S7.2: Convert the forwarded initial 3D target trajectory set T forward Traj, the h-th 3D target trajectory h The number of 3D targets is denoted as O' h Set variable q” h The h-th 3D target trajectory traj h The first 3D target Deterministic score Initialize to 0, and set q” h Initialize to 2;

[0076] Step S7.3: Calculate the h-th 3D target trajectory traj h The middle q” h 3D target Deterministic score The specific process is as follows:

[0077] Step S7.3.1: Transfer the h-th 3D target trajectory traj h The middle q” h 3D target timestamp minus q” h -1 3D target The difference obtained from the timestamps is used as the 3D target. With 3D targets The time difference, denoted as d h;

[0078] Step S7.3.2: Make the 3D target The confidence decay term is in, Representing 3D targets The confidence score; e represents the base of the natural logarithm, and * is the multiplication operator;

[0079] When 3D target Confidence score detected at time t The higher the 3D target The higher the legitimacy of the added trajectory, the more exponential decay is introduced, since false positive trajectories are typically unstable and transient. This reduces the legitimacy of trajectories that have not been observed for a long time.

[0080] Step S7.3.3: Make the 3D target The missing penalty item is Trajectory missing time d h The longer the trajectory, the more likely the 3D target has left, thus reducing the validity of the trajectory. The detection score is high (i.e. If the value is relatively large, the impact of the missing penalty is reduced to prevent accidental deletion of distant or partially occluded 3D targets.

[0081] Step S7.3.4: Calculate the 3D target using formula (7). Deterministic score

[0082] In this step, 3D targets are combined. Historical certainty score By utilizing historical trajectory information, the system becomes more tolerant of temporarily lost real 3D targets;

[0083] Step S7.3.5: Determine the 3D target Deterministic score Is it greater than the legality threshold α? legality If it is greater than , then proceed to step S7.4; otherwise, proceed to step S7.3.6.

[0084] Step S7.3.6: Transfer the h-th 3D target trajectory traj h "middle front q" h The 3D targets are combined into a new 3D target trajectory. Using a 3D Kalman filter to analyze the trajectory of a 3D target Prediction is performed to obtain the h-th 3D target trajectory traj. h The q”h 3D target State estimation variance and 3D target in the x-direction The variance of the state estimate in the y-direction is: The h-th 3D target trajectory traj h The q” h 3D target The variance of the state estimate in the x-direction is denoted as . The h-th 3D target trajectory traj h The q” h 3D target The variance of the state estimate in the y-direction is denoted as...

[0085] Step S7.3.7: Determine Is it greater than α? cov or Is it greater than α? cov ,if Greater than α cov or Greater than α cov If the condition is met, proceed to step S7.5; otherwise, proceed to step S7.3.8.

[0086] Step S7.3.8: Determine q” h Is the current value equal to O'? h If not equal, then use q. h The sum of the current value plus 1 updates q” h The value is determined, and then the process returns to step S7.3. If the value is equal to the given value, then step S7.4 is executed.

[0087] Step S7.4: Convert the forwarded initial 3D target trajectory set T forward Traj, the h-th 3D target trajectory h As the 3D target trajectory set T' forward A 3D target trajectory is added to the 3D target trajectory set T' forward middle;

[0088] Step S7.5: Determine if the current value of h is equal to L. forward If it is not equal, then update the value of h by adding 1 to the current value of h, and then return to execute step S7.2; if it is equal, then proceed to step S7.6.

[0089] Step S7.6: Forwarding the initial 3D target trajectory set T forward All 3D target trajectories undergo trajectory validity verification, resulting in a positive 3D target trajectory set T'. forward .

[0090] Furthermore, in step 6, the reversed initial 3D target trajectory set T backward The specific process for verifying the legality of each 3D target trajectory is as follows:

[0091] Step S7.7: Convert the reversed initial 3D target trajectory set T backward The number of 3D target trajectories is denoted as L. backward Let T' be the set of reversed 3D target trajectories. backward Legality threshold α legality Threshold α cov , variable h', where 0.5≤α legality ≤0.6, 1.5≤α cov If the value is less than or equal to 2.0, initialize h' to 1.

[0092] Step S7.8: Convert the reversed initial 3D target trajectory set T backward The h'th 3D target trajectory traj' h' The number of 3D targets is denoted as O". h' Set variable q”' h' The h'th 3D target trajectory traj' h' The first 3D target Deterministic score Initialize to 0, and set q”' h' Initialize to 2;

[0093] Step S7.9: Calculate the h'-th 3D target trajectory traj' h' The middle q”' h' 3D target Deterministic score The specific process is as follows:

[0094] Step S7.9.1: Transfer the h'-th 3D target trajectory traj' h' The middle q”' h' 3D target timestamp minus q”' h' -1 3D target The difference obtained from the timestamps is used as the 3D target. With 3D targets The time difference is denoted as d' h' ;

[0095] Step S7.9.2: Make the 3D target The confidence decay term is in, Representing 3D targets The confidence score; e represents the base of the natural logarithm, and * is the multiplication operator;

[0096] Step S7.9.3: Make the 3D target The missing penalty item is

[0097] Step S7.9.4: Calculate the 3D target using formula (8). Deterministic score

[0098] Step S7.9.5: Determine the 3D target Deterministic score Is it greater than the legality threshold α? legality If it is greater than , then proceed to step S7.10; otherwise, proceed to step S7.9.6.

[0099] Step S7.9.6: Transfer the h'-th 3D target trajectory traj' h' "middle front q"' h' The 3D targets are combined into a new 3D target trajectory. Using a 3D Kalman filter to analyze the trajectory of a 3D target Prediction is performed to obtain the h'-th 3D target trajectory traj'. h' The q”' h' 3D target State estimation variance and 3D target in the x-direction The state estimation variance in the y-direction is represented by the h'-th 3D target trajectory traj'. h' The q”' h' 3D target The variance of the state estimate in the x-direction is denoted as . The h'th 3D target trajectory traj' h' The q”' h' 3D target The variance of the state estimate in the y-direction is denoted as...

[0100] Step S7.9.7: Determine Is it greater than α? cov or Is it greater than α? cov ,if Greater than α cov or Greater than α cov If the condition is met, proceed to step S7.11; otherwise, proceed to step S7.9.8.

[0101] Step S7.9.8: Determine q”' h'Is the current value equal to 0? h' If not equal, then use q”' h' The sum of the current value plus 1 updates q”' h' The value is determined, and then the process returns to step S7.9. If the value is equal to the given value, then step S7.10 is executed.

[0102] Step S7.10: Convert the reversed initial 3D target trajectory set T backward The h'th 3D target trajectory traj' h' As the 3D target trajectory set T' backward A 3D target trajectory is added to the 3D target trajectory set T' backward middle;

[0103] Step S7.11: Determine if the current value of h' is equal to L. backward If it is not equal, then update the value of h' by adding 1 to the current value of h', and then return to execute step S7.8; if it is equal, then proceed to step S7.12.

[0104] Step S7.12: Reverse the initial 3D target trajectory set T backward After the trajectory validity verification of all 3D target trajectories is completed, the reversed 3D target trajectory set T' is obtained. backward . Attached Figure Description

[0105] Figure 1 This is a flowchart of the automatic 3D target trajectory annotation method for point cloud denoising and trajectory verification according to the present invention;

[0106] Figure 2 This is a flowchart illustrating how the present invention generates an initial set of 3D target trajectories by tracking the adjusted 3D target detection results based on data observation and association strategies.

[0107] Figure 3 This is a flowchart illustrating the trajectory validity verification process for each 3D target trajectory in the initial 3D target trajectory set according to the present invention. Detailed Implementation

[0108] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0109] Example 1: As Figure 1 As shown, an automatic 3D target trajectory annotation method for point cloud denoising and trajectory verification includes the following steps:

[0110] Step 1: Perform data augmentation on the continuous point cloud data to obtain the augmented continuous point cloud data;

[0111] Step 2: Using time information, combine past point cloud frames with the current point cloud frame in X ways to obtain X types of point cloud frame combination data, where X is an integer greater than or equal to 2; and use a 3D target detection algorithm to obtain the 3D target detection results of the X types of point cloud frame combination data.

[0112] Step 3: Perform weighted fusion of the X 3D object detection results to obtain the initial 3D object detection result;

[0113] Step 4: Adjust and filter the initial 3D target detection results using a point density-aware voxel network to obtain the adjusted 3D target detection results. In this process, using a point density-aware voxel network to adjust and filter the initial 3D target detection results can obtain more accurate 3D target bounding boxes, reduce noise in the 3D target detection stage, and improve the accuracy of the 3D target detection results.

[0114] Step 5: Based on data observation and association strategies, perform trajectory tracking on the adjusted 3D target detection results to generate a forward initial 3D target trajectory set and a reverse initial 3D target trajectory set. Each of the forward and reverse initial 3D target trajectory sets consists of multiple 3D target trajectories. The data observation and association strategies can further filter point cloud noise in the 3D target detection results, improve the accuracy of spatial position and motion parameter state estimation of occluded or distant objects, and thus avoid fragmented trajectories caused by deviations between the subsequently labeled trajectory and the actual motion trajectory of the target.

[0115] Step 6: Perform trajectory validity verification on each 3D target trajectory in the two initial 3D target trajectory sets, namely the forward initial 3D target trajectory set and the reverse initial 3D target trajectory set (if a 3D target trajectory fails the verification, it is considered a false positive trajectory). The 3D target trajectories that pass the validity verification in the forward initial 3D target trajectory set are used to form the forward 3D target trajectory set, and the 3D target trajectories that pass the validity verification in the reverse initial 3D target trajectory set are used to form the reverse 3D target trajectory set.

[0116] Step 7: Perform weighted fusion processing on the forward-oriented 3D target trajectory set and the reverse-oriented 3D target trajectory set to obtain the final 3D target trajectory set. The 3D target trajectory is automatically labeled.

[0117] Traditional 3D target detection is susceptible to noise interference, leading to false positives or false negatives, especially at long distances or when targets are partially occluded. In such cases, the detected 3D targets can cause subsequent trajectory tracking to deviate from the actual trajectory. This embodiment's automatic 3D target trajectory annotation method introduces multi-frame point cloud combination, utilizing data from the current frame and multiple past frames to construct a point cloud frame combination, providing more spatial and temporal information and enhancing the detection capability for distant or occluded targets. It also employs a weighted fusion of detection boxes, weighting the position and confidence of different detection boxes to improve detection accuracy and reduce false positives. A point density-aware voxel network is used to adjust and filter the initial 3D target detection results, analyzing changes in point cloud density to further optimize the detection boxes, making the 3D target boundaries more accurate and effectively reducing noise generated during the detection stage, resulting in more precise 3D target bounding boxes. Regarding trajectory tracking, existing algorithms suffer from low accuracy in estimating motion state parameters when targets are partially occluded or at long distances, causing trajectories to deviate from the actual path. This embodiment's automatic 3D target trajectory annotation method tracks the adjusted 3D target detection results based on data observation and association strategies. It filters out low-quality matches by using forward and reversed target trajectories, ensuring a high degree of consistency between the trajectory and the actual motion trajectory, further filtering point cloud noise. Due to detector instability, subsequent trajectory tracking may receive false 3D target detection results, generating false positive trajectories. Traditional methods typically use a 3D target confidence threshold to filter out low-confidence targets, but this method may lose distant or partially occluded true 3D targets. This embodiment's automatic 3D target trajectory annotation method proposes a trajectory validity verification method. It uses historical trajectory information and state estimation uncertainty to terminate the trajectory, reducing dependence on fixed frame numbers and thresholds, effectively reducing false positive trajectories, and improving the accuracy of 3D target trajectory tracking. Finally, the weighted fusion of the forward and reversed 3D target trajectory sets yields the final 3D target trajectory set, avoiding the problem of 3D targets disappearing within a certain time period and failing to form a complete trajectory. Furthermore, the reversed trajectory effectively eliminates the influence of historical state errors on the forward trajectory, improving the efficiency and accuracy of automatic 3D target trajectory annotation. Therefore, the automatic 3D target trajectory annotation method in this embodiment improves the speed of 3D target trajectory annotation through an efficient process and accurately completes the automatic annotation of 3D target trajectory. It can fully meet the annotation requirements of large-scale datasets and greatly reduce annotation time and labor costs.

[0118] Example 2: This example is basically the same as Example 1, except that in this example, the specific process of performing data augmentation on the continuous point cloud data in step 1 to obtain the augmented continuous point cloud data is as follows:

[0119] Step S1.1: Denote the continuous point cloud data as P, and denote the i-th point cloud frame in the continuous point cloud data P as Pi. i In the continuous point cloud data, the i-th point cloud frame P i For the data obtained from the i-th scan of the lidar, the i-th point cloud frame P in the continuous point cloud data P is... i The time of being scanned by lidar is recorded as Where i = 1, 2, ..., I, I is the total number of point cloud frames in the continuous point cloud data P;

[0120] Step S1.2: Extract the i-th point cloud frame P from the continuous point cloud data P. i The timestamp is T i T is calculated using formula (1). i :

[0121]

[0122] Where, Δt m It is the i-th point cloud frame P in continuous point cloud data i With the m-th point cloud frame P m The time difference between being scanned by the lidar, m = 1, 2, ..., i;

[0123] Step S1.3: From the ground truth data P of the continuous point cloud data P gt N true values ​​are selected from the data, where N is greater than or equal to 5 and less than or equal to 10. The N true values ​​include V1 true values ​​of vehicles, V2 true values ​​of pedestrians, and V3 true values ​​of cyclists, where V1, V2, and V3 are integers greater than or equal to 0. After performing translation and rotation operations on all N true values, they are randomly added to the i-th point cloud frame P of the continuous point cloud data P. i In the continuous point cloud data P, for the i-th point cloud frame P i Enhancement is performed by taking the i-th point cloud frame P of the continuous point cloud data P. i The data before and after the increase is denoted as P. aug i, P aug i represents the enhanced continuous point cloud data P. aug The i-th point cloud frame.

[0124] In this embodiment, by obtaining the timestamp of each point cloud frame in the continuous point cloud data, the time of each point cloud frame in the continuous point cloud data can be distinguished, which helps the subsequent algorithm to locate 3D targets and achieve velocity prediction; at the same time, by enhancing the point cloud frames, the 3D target information of the point cloud data is enriched.

[0125] Example 3: This example is basically the same as Example 2, except that: in this example, the xth type of point cloud frame combination data obtained in step 2 is denoted as x = 1, 2, ..., X, for the enhanced continuous point cloud data P aug Perform the xth combination to obtain the xth point cloud frame combination data. The specific process is as follows: from the enhanced continuous point cloud data P aug The first point cloud frame P aug 1. Start by selecting a point cloud frame each time. Determine if there are x-1 more point cloud frames after the selected point cloud frame. If there are, combine the selected point cloud frame with the following x-1 point cloud frames to form a point cloud frame combination of the xth method. If there are no points cloud frames, the xth method combination ends. All point cloud frame combinations of the xth method constitute the xth point cloud frame combination data.

[0126] Example 4: This example is basically the same as Example 3, except that in this example, in step 3, formula (2) is used to weight and fuse the X 3D target detection results to obtain the initial 3D target detection result D. final :

[0127] D final =WBF(D1,D2,D3,…,D) X (2)

[0128] Where WBF(·) represents the weighted fusion function, D x The x-th point cloud frame combination data obtained using a 3D object detection algorithm 3D target detection results.

[0129] In this embodiment, by weighting the position and confidence of different detection boxes, the detection accuracy of 3D targets is improved and the false detection rate is reduced. A more accurate initial 3D target detection result is obtained from the 3D target detection results of X point cloud frame combination data.

[0130] Example 5: This example is basically the same as Example 4, except that in this example, the point density-sensing voxel network thinning function is used in step 4 to refine the initial 3D target detection result D after fusion in step 3. final Adjusting the detection bounding box in the D array yields more accurate 3D target detection results. refinal Among them, the 3D target detection result D refinal It consists of the 3D target detection results of I point cloud frames of continuous point cloud data P.

[0131] Example 6: This example is basically the same as Example 5, except that in this example, the adjusted 3D target detection result D is adjusted based on data observation and association strategies in step 5. refinal The specific process of performing trajectory tracking and generating a positive initial set of 3D target trajectories is as follows:

[0132] Step S6.1: Set the data observation threshold to α. conf To avoid system overload, the validity of the observation results will be evaluated during the trajectory legality verification phase. Data observation is a 3D target recognition step, the main purpose of which is to allow the observation of low-detection-score objects at a distance and with partial occlusion, while preventing a large number of low-confidence 3D targets from overloading the system, where 0.1≤α conf ≤0.3;

[0133] Step S6.2: Convert the 3D target detection results D refinal The i-th point cloud frame P i The 3D object detection result is denoted as 3D object detection results The number of 3D targets is denoted as O. i 3D target detection results The o i Each 3D target is denoted as Among them o i =1, 2, 3, ..., O i 3D target detection results D refinal 3D object detection results of the first point cloud frame P1 in the data. The timestamps of all 3D targets are equal to the timestamp of the first point cloud frame P1. This applies to the 3D target detection result D. refinal 3D object detection results of the first point cloud frame P1 in the data. All 3D targets are initialized with trajectories to obtain their 3D target trajectories. At this point, the target detection result D is obtained. refinal 3D object detection results of the first point cloud frame P1 in the data. Each 3D target's 3D trajectory is an initial trajectory, and the target detection results D are... refinal In the 3D object detection results of each point cloud frame, the number of successful matches for each 3D object trajectory is initialized to 0, and the number of failed matches is initialized to 0. The 3D object detection results D... refinal The 3D object detection result D of the first point cloud frame P1 in the data. P1 The 3D target trajectories of all 3D targets constitute the 0th generation 3D target trajectory set T0, where This represents the o1-th 3D target trajectory in the 0th generation 3D target trajectory set T0, i.e., the 3D target detection result D. refinal 3D object detection results of the first point cloud frame P1 in the data. The 3D trajectory of the o1th 3D target;

[0134] Step S6.3: Set the iteration count variable r, the maximum number of iterations R, and the matching threshold n.min Threshold IoU thres and GIoU thres Where R = I⁻¹, n min =3,n max =5, 0.25≤IoU thres ≤0.35, 0.1≤GIoU thres If r is less than or equal to 0.2, initialize r to 1.

[0135] Step S6.4: Perform the r-th iteration to obtain the r-th generation 3D target trajectory set T r The specific process is as follows:

[0136] Step S6.4.1: Set the 3D target trajectory T of the (r-1)th generation r-1 The number of 3D target trajectories is denoted as L. r-1 Set the variable to l r and q r+1 , will l r Initialize to 1;

[0137] Step S6.4.2: q r+1 Initialize to 1;

[0138] Step S6.4.3: For the (r-1)th generation 3D target trajectory set T r-1 The first one r 3D target trajectory The updated 3D target trajectory is obtained and used as the r-th generation 3D target trajectory set T. r The l r 3D target trajectory Added to the r-th generation 3D target trajectory set T r The specific process is as follows:

[0139] Step S6.4.3.1: Determine the 3D target detection result The qth r+1 3D target Is the confidence score greater than α? conf If it is greater than α conf If so, proceed directly to step S6.4.3.2; otherwise, determine q. r+1 Is the current value equal to O? r+1 If q r+1 The current value is not equal to 0. r+1 Then q is used. r+1 The sum of the current value plus 1 updates q r+1 The value of q is then returned to step S6.4.3.1. If q r+1 The current value is equal to O. r+1 Then first set the 3D target trajectory T of the (r-1)th generationr-1 The first one r 3D target trajectory As the r-th generation 3D target trajectory set T r The l r 3D target trajectory Added to the r-th generation 3D target trajectory set T r In, and make the 3D target trajectory The number of successful matches equals the 3D target trajectory The current value of the number of successful matches, 3D target trajectory The number of matching failures equals the 3D target trajectory The sum of the current value of the number of failed matches plus 1 is then used to determine l. r Is the current value equal to L? r-1 If l r The current value is not equal to L r-1 Then first use l r The sum of the current value plus 1 updates l r The value of l is then used to return to step S6.4.2. If l r The current value is equal to L. r-1 Then proceed to step S6.4.3.7;

[0140] Step S6.4.3.2: Use a 3D Kalman filter (3DKF) to process the (r-1)th generation 3D target trajectory set T r-1 The first one r 3D target trajectory Make a prediction to obtain the corresponding 3D target state, and record it as...

[0141] Step S6.4.3.3: Calculate the 3D target detection results using formulas (3) and (4) respectively. The qth r+1 3D target Intersection over Union (IoU) of middle bounding boxes and generalized intersection

[0142]

[0143] in, This represents the 3D object detection result. The qth r+1 3D target With 3D target state The volume of the intersection Indicates 3D object detection results The qth r+1 3D target With 3D target state The union volume, This indicates that the results include 3D object detection. The qth r+1 3D target With 3D target state The minimum enclosed volume;

[0144] Step S6.4.3.4: If Less than IoU thres or Less than GIoU thres This indicates the 3D object detection result. The qth r+1 3D target and 3D target state Matching failed; 3D object detection results will be returned. The qth r+1 3D target For 3D targets that fail to match, step S6.4.3.6 is executed at this time;

[0145] if Greater than or equal to IoU thres or Greater than or equal to GIoU thres This indicates the 3D object detection result. The qth r+1 3D target and 3D target state Matching successful, 3D object detection results will be displayed. The qth r+1 3D target As the 3D target that has been successfully matched, proceed to step 6.4.3.5;

[0146] Step S6.4.3.5: Set the 3D target detection results... The qth r+1 3D target The timestamp is equal to the (r+1)th point cloud frame P r+1 Timestamps and 3D object detection results The qth r+1 3D target The 3D Kalman filter is updated to obtain the 3D target trajectory, which is the set of 3D target trajectories T for the r-th generation. r The l r 3D target trajectory And make the 3D target trajectory The number of successful matches is equal to the set of 3D target trajectories T of generation r-1. r-1The first one r 3D target trajectory The sum of the current number of successful matches plus 1, 3D target trajectory The number of matching failures is equal to the set of 3D target trajectories T of generation r-1. r-1 The first one r 3D target trajectory The number of failed matches, to determine l r Is the current value equal to L? r-1 If l r The current value is equal to L. r-1 Then first use l r The sum of the current value plus 1 updates l r The value of l is then used to return to step S6.4.2. If l r The current value is equal to L. r-1 Then proceed to step S6.4.3.7;

[0147] Step S6.4.3.6: Determine q r+1 Is the current value equal to O? r+1 If q r+1 The current value is not equal to 0. r+1 Then q is used. r+1 The sum of the current value plus 1 updates q r+1 The value of q is then returned to step S6.4.3.1. If q r+1 The current value is equal to O. r+1 Then first set the 3D target trajectory T of the (r-1)th generation r-1 The first one r 3D target trajectory As the r-th generation 3D target trajectory set T r The l r 3D target trajectory Added to the r-th generation 3D target trajectory set T r In, and make the 3D target trajectory The number of successful matches equals the 3D target trajectory The current value of the number of successful matches, 3D target trajectory The number of matching failures equals the 3D target trajectory The sum of the current value of the number of failed matches plus 1 is then used to determine l. r Is the current value equal to L? r-1 If l r The current value is not equal to L r-1 Then use l r The sum of the current value plus 1 updates l r The value of l is then used to return to step S6.4.2. If l r The current value is equal to L.r-1 Then proceed to step S6.4.3.7;

[0148] Step S6.4.3.7: Analyze the 3D target detection results. Each 3D target that fails to match is evaluated individually. If the 3D target detection result is... The confidence score of a 3D target that failed to match is higher than α. conf Then the timestamp of the 3D target is equal to the (r+1)th point cloud frame P. r+1 The timestamp is used to initialize the trajectory of the 3D target, resulting in an initialized trajectory as the 3D target trajectory of the 3D target. This 3D target trajectory is then used as the set of 3D target trajectories for the r-th generation. r A 3D target trajectory is added to the r-th generation 3D target trajectory set T r The Lth r-1 After identifying a 3D target trajectory, the number of successful matches and the number of failed matches for that 3D target trajectory are initialized to 0.

[0149] Step S6.4.3.8: For the currently obtained r-th generation 3D target trajectory set T r Each of the 3D target trajectories in the set is judged individually. If the set of 3D target trajectories in the r-th generation is T... r The number of successful matches of a certain 3D target trajectory is less than a threshold n min And the number of failed matches is greater than the threshold n max Then the 3D target trajectory will be removed from the set of 3D target trajectories T of generation r. r Delete it if necessary, otherwise do not process it.

[0150] Step S6.5: Determine if the current value of r is equal to R. If not, update the value of r by adding 1 to the current value of r, and then return to step S6.4 for the next iteration. If it is equal to R, then the currently obtained R-th generation 3D target trajectory set T is... R That is, the initial set of 3D target trajectories T after positive transformation. forward .

[0151] In this embodiment, step 5 involves adjusting the 3D target detection result D based on data observation and association strategies. refinal The specific process of performing trajectory tracking and generating a reversed initial set of 3D target trajectories is as follows:

[0152] Step S6.6: Let the 3D object detection result D refinal The i-th point cloud frame P I 3D object detection results The timestamps of all 3D objects are equal to the i-th point cloud frame. The timestamp for the 3D object detection result D refinal The i-th point cloud frame P I 3D object detection results All 3D targets are initialized with trajectories to obtain their 3D target trajectories. At this point, the target detection result D is obtained. refinal The i-th point cloud frame P I 3D object detection results Each 3D target's 3D trajectory is an initial trajectory, and the target detection results D are... refinal In the 3D object detection results of each point cloud frame, the number of successful matches for each 3D object trajectory is initialized to 0, and the number of failed matches is initialized to 0. The 3D object detection results D... refinal The i-th point cloud frame P I 3D object detection results The 3D target trajectories of all 3D targets constitute the 0th generation 3D target trajectory set T0', where This represents the o-th trajectories in the 0th generation 3D target trajectory set T0. I A 3D target trajectory, i.e., the 3D target detection result D refinal The i-th point cloud frame P I 3D object detection results The o I 3D target trajectory of a 3D target;

[0153] Step S6.7: Set the iteration count variable r', the maximum iteration count R, and the matching threshold n. min Threshold IoU thres and GIoU thres Where R = I⁻¹, n min =3,n max =5, 0.25≤IoU thres ≤0.35, 0.1≤GIoU thres If ≤0.2, initialize r' to 1;

[0154] Step S6.8: Perform the r'-th iteration to obtain the r'-th generation 3D target trajectory set T". The specific process is as follows:

[0155] Step S6.8.1: Set the 3D target trajectory set T' of the r'-1th generation. r'-1 The number of 3D target trajectories is denoted as L' r'-1 Set the variable to l' r' and q' I-r' , will l' r' Initialize to 1;

[0156] Step S6.8.2: q'I-r' Initialize to 1;

[0157] Step S6.8.3: For the r'-1th generation 3D target trajectory set T' r'-1 The l' r' 3D target trajectory traj' l r ” - 1 The updated 3D target trajectory is obtained and used as the set of 3D target trajectories T' for the r'th generation. r' The l' r' 3D target trajectory traj' l r ” The specific process of adding it to the 3D target trajectory set T of generation r' is as follows:

[0158] Step S6.8.3.1: Determine the 3D target detection result The q' I-r' 3D target Is the confidence score greater than α? conf If it is greater than α conf If yes, proceed directly to step S6.8.3.2; otherwise, determine q'. I-r' Is the current value equal to O? I-r' If q' I-r' The current value is not equal to 0. I-r' Then q' I-r' The sum of the current value plus 1 updates q' I-r' The value of q' is then returned to step S6.8.3.1. If q' I-r' The current value is equal to O. I-r' Then first set the 3D target trajectory T' of the r'-1 generation. r'-1 The l' r' indivual

[0159] 3D target trajectory As the r'th generation 3D target trajectory set T' r' The l' r' 3D target trajectory Add to r'

[0160] 3D target trajectory set T' r' In, and make the 3D target trajectory The number of successful matches equals the 3D target trajectory of

[0161] Current value of the number of successful matches, 3D target trajectory The number of matching failures equals the 3D target trajectory Matching

[0162] Sum the current value of 1 for the number of failures, then check l'. r' Is the current value equal to L'? r'-1 If l' r' The current value is not equal to L' r'-1 Then l' is used first. r' The sum of the current value plus 1 updates L' r'-1 The value of l' is then determined, and the process returns to step S6.8.2. If l' r' The current value is equal to L' r'-1 Then proceed to step S6.8.3.7;

[0163] Step S6.8.3.2: Use a 3D Kalman filter to process the set of 3D target trajectories T' of the r'-1th generation. r'-1 The l' r' 3D target trajectory Make a prediction to obtain the corresponding 3D target state, and record it as...

[0164] Step S6.8.3.3: Calculate the 3D target detection results using formulas (5) and (6) respectively. The q' I-r' 3D target Intersection over Union (IoU) of middle bounding boxes and generalized intersection

[0165]

[0166] in, This represents the 3D object detection result. The q' I-r' 3D target With 3D target state The volume of the intersection Indicates 3D object detection results The q' I-r' 3D target With 3D target state The union volume, This indicates that the results include 3D object detection. The q' I-r' 3D target With 3D target state The minimum enclosed volume;

[0167] Step S6.8.3.4: If Less than IoU thres or Less than GIoU thres This indicates the 3D object detection result. The q' I-r' 3D target and 3D target state Matching failed; 3D object detection results will be returned. The q' I-r' 3D target For 3D targets that fail to match, step S6.8.3.6 is executed at this time;

[0168] if Greater than or equal to IoU thres or Greater than or equal to GIoU thres This indicates the 3D object detection result. The q' I-r' 3D target and 3D target state Matching successful, 3D object detection results will be displayed. The q' I-r' 3D target As the 3D target that has been successfully matched, proceed to step 6.8.3.5;

[0169] Step S6.8.3.5: Set the 3D target detection results... The q' I-r' 3D target The timestamp is equal to the I-r'th point cloud frame P I-r' Timestamps and 3D object detection results The q' I-r' 3D target The 3D Kalman filter is updated to obtain the 3D target trajectory, which is the l-th 3D target trajectory in the r'-th generation 3D target trajectory set T”. And make the 3D target trajectory The number of successful matches is equal to the set of 3D target trajectories T' of generation r'-1. r'-1 The l' r' 3D target trajectory The sum of the current number of successful matches plus 1, 3D target trajectory The number of matching failures is equal to the set of 3D target trajectories T' of generation r'-1. r'-1 The l' r' 3D target trajectory The current value of the number of failed matches is used to determine l'. r' Is the current value equal to L'? r'-1 If l' r' The current value is equal to L' r'-1 Then l' is used first. r' The sum of the current value plus 1 updates L'r'-1 The value of l' is then determined, and the process returns to step S6.8.2. If l' r' The current value is equal to L' r'-1 Then proceed to step S6.8.3.7;

[0170] Step S6.8.3.6: Determine q' I-r' Is the current value equal to O? I-r' If q' I-r' The current value is not equal to 0. I-r' Then q' I-r' The sum of the current value plus 1 updates q' I-r' The value of q' is then returned to step S6.8.3.1. If q' I-r' The current value is equal to O. I-r' Then first set the 3D target trajectory T' of the r'-1 generation. r'-1 The l' r' 3D target trajectory As the r'th generation 3D target trajectory set T' r' The l' r' 3D target trajectory Added to the 3D target trajectory set T' of generation r' r' In, and make the 3D target trajectory The number of successful matches equals the 3D target trajectory The current value of the number of successful matches, 3D target trajectory The number of successful matches equals the 3D target trajectory The sum of the current number of successful matches plus 1 is used to determine l'. r' Is the current value equal to L'? r'-1 If l' r' The current value is not equal to L' r'-1 Then l' is used first. r' The sum of the current value plus 1 updates L' r'-1 The value of l' is then determined, and the process returns to step S6.8.2. If l' r' The current value is equal to L' r'-1 Then proceed to step S6.8.3.7;

[0171] Step S6.8.3.7: Analyze the 3D target detection results. Each 3D target that fails to match is evaluated individually. If the 3D target detection result is... The confidence score of a 3D target that failed to match is higher than α. conf Then the timestamp of the 3D target is equal to the I-r'th point cloud frame P. I-r'The timestamp is then used to initialize the trajectory of the 3D target, resulting in an initialized trajectory as the 3D target trajectory of the 3D target. This 3D target trajectory is then used as the set of 3D target trajectories for the r'th generation T'. r' A 3D target trajectory is added to the 3D target trajectory set T' of generation r'. r' The L' r'-1 After identifying a 3D target trajectory, the number of successful matches and the number of failed matches for that 3D target trajectory are initialized to 0.

[0172] Step S6.8.3.8: For the currently obtained r'-th generation 3D target trajectory set T” r’ Each of the 3D target trajectories in the set is judged individually. If the set of 3D target trajectories in the r'th generation is T' r' The number of successful matches of a certain 3D target trajectory is less than a threshold n min And the number of failed matches is greater than the threshold n max Then the 3D target trajectory will be removed from the 3D target trajectory set T' of generation r'. r' Delete it if necessary, otherwise do not process it.

[0173] Step S6.9: Determine if the current value of r' is equal to R. If not, update the value of r' by adding 1 to the current value of r', and then return to step S6.8 for the next iteration. If it is equal to R, then the currently obtained R-th generation 3D target trajectory set T' R That is, the inversely converted initial 3D target trajectory set T backward .

[0174] In this embodiment, step 6 involves forwarding the initial 3D target trajectory set T. forward The specific process for verifying the legality of each 3D target trajectory is as follows:

[0175] Step S7.1: Convert the initial 3D target trajectory set T into a forward-oriented form. forward The number of 3D target trajectories is denoted as L. forward Let T' be the set of positively oriented 3D target trajectories. forward Legality threshold α legality Threshold α cov , where 0.5 ≤ α legality ≤0.6, 1.5≤α cov If ≤2.0, initialize h to 1;

[0176] Step S7.2: Convert the forwarded initial 3D target trajectory set T forward Traj, the h-th 3D target trajectory h The number of 3D targets is denoted as O' hSet variable q” h The h-th 3D target trajectory traj h The first 3D target Deterministic score Initialize to 0, and set q” h Initialize to 2;

[0177] Step S7.3: Calculate the h-th 3D target trajectory traj h The middle q” h 3D target Deterministic score The specific process is as follows:

[0178] Step S7.3.1: Transfer the h-th 3D target trajectory traj h The middle q” h 3D target timestamp minus q” h -1 3D target The difference obtained from the timestamps is used as the 3D target. With 3D targets The time difference, denoted as d h ;

[0179] Step S7.3.2: Make the 3D target The confidence decay term is in, Representing 3D targets The confidence score; e represents the base of the natural logarithm, and * is the multiplication operator;

[0180] When 3D target Confidence score detected at time t The higher the 3D target The higher the legitimacy of the added trajectory, the more exponential decay is introduced, since false positive trajectories are typically unstable and transient. This reduces the legitimacy of trajectories that have not been observed for a long time.

[0181] Step S7.3.3: Make the 3D target The missing penalty item is Trajectory missing time d h The longer the trajectory, the more likely the 3D target has left, thus reducing the validity of the trajectory. The detection score is high (i.e. If the value is relatively large, the impact of the missing penalty is reduced to prevent accidental deletion of distant or partially occluded 3D targets.

[0182] Step S7.3.4: Calculate the 3D target using formula (7). Deterministic score

[0183] In this step, 3D targets are combined. Historical certainty score By utilizing historical trajectory information, the system becomes more tolerant of temporarily lost real 3D targets;

[0184] Step S7.3.5: Determine the 3D target Deterministic score Is it greater than the legality threshold α? legality If it is greater than , then proceed to step S7.4; otherwise, proceed to step S7.3.6.

[0185] Step S7.3.6: Transfer the h-th 3D target trajectory traj h "middle front q" h The 3D targets are combined into a new 3D target trajectory. Using a 3D Kalman filter to analyze the trajectory of a 3D target Prediction is performed to obtain the h-th 3D target trajectory traj. h The q” h 3D target State estimation variance and 3D target in the x-direction The variance of the state estimate in the y-direction is: The h-th 3D target trajectory traj h The q” h 3D target The variance of the state estimate in the x-direction is denoted as . The h-th 3D target trajectory traj h The q” h 3D target The variance of the state estimate in the y-direction is denoted as...

[0186] Step S7.3.7: Determine Is it greater than α? cov or Is it greater than α? cov ,if Greater than α cov or Greater than α cov If the condition is met, proceed to step S7.5; otherwise, proceed to step S7.3.8.

[0187] Step S7.3.8: Determine q” h Is the current value equal to O'? h If not equal, then use q. h The sum of the current value plus 1 updates q”h The value is determined, and then the process returns to step S7.3. If the value is equal to the given value, then step S7.4 is executed.

[0188] Step S7.4: Convert the forwarded initial 3D target trajectory set T forward Traj, the h-th 3D target trajectory h As the 3D target trajectory set T' forward A 3D target trajectory is added to the 3D target trajectory set T' forward middle;

[0189] Step S7.5: Determine if the current value of h is equal to L. forward If it is not equal, then update the value of h by adding 1 to the current value of h, and then return to execute step S7.2; if it is equal, then proceed to step S7.6.

[0190] Step S7.6: Forwarding the initial 3D target trajectory set T forward All 3D target trajectories undergo trajectory validity verification, resulting in a positive 3D target trajectory set T'. forward .

[0191] In this embodiment, step 6 involves reversing the initial 3D target trajectory set T. backward The specific process for verifying the legality of each 3D target trajectory is as follows:

[0192] Step S7.7: Convert the reversed initial 3D target trajectory set T backward The number of 3D target trajectories is denoted as L. backward Let T' be the set of reversed 3D target trajectories. backward Legality threshold α legality Threshold α cov , variable h', where 0.5≤α legality ≤0.6, 1.5≤α cov If the value is less than or equal to 2.0, initialize h' to 1.

[0193] Step S7.8: Convert the reversed initial 3D target trajectory set T backward The h'th 3D target trajectory traj' h' The number of 3D targets is denoted as O". h' Set variable q”' h' The h'th 3D target trajectory traj' h' The first 3D target Deterministic score Initialize to 0, and set q”' h' Initialize to 2;

[0194] Step S7.9: Calculate the h'-th 3D target trajectory traj' h' The middle q”' h' 3D target det”'h q”’h' Deterministic score The specific process is as follows:

[0195] Step S7.9.1: Transfer the h'-th 3D target trajectory traj' h' The middle q”' h' 3D target timestamp minus q”' h' -1 3D target The difference obtained from the timestamps is used as the 3D target. With 3D targets The time difference is denoted as d' h' ;

[0196] Step S7.9.2: Make the 3D target The confidence decay term is in, Representing 3D targets The confidence score; e represents the base of the natural logarithm, and * is the multiplication operator;

[0197] Step S7.9.3: Make the 3D target The missing penalty item is

[0198] Step S7.9.4: Calculate the 3D target using formula (8). Deterministic score

[0199] Step S7.9.5: Determine the 3D target Deterministic score Is it greater than the legality threshold α? legality If it is greater than , then proceed to step S7.10; otherwise, proceed to step S7.9.6.

[0200] Step S7.9.6: Transfer the h'-th 3D target trajectory traj' h' "middle front q"' h' The 3D targets are combined into a new 3D target trajectory. Using a 3D Kalman filter to analyze the trajectory of a 3D target Prediction is performed to obtain the h'-th 3D target trajectory traj'. h' The q”' h' 3D target State estimation variance and 3D target in the x-direction The state estimation variance in the y-direction is represented by the h'-th 3D target trajectory traj'. h' The q”' h' 3D target The variance of the state estimate in the x-direction is denoted as . The h'th 3D target trajectory traj' h' The q”' h' 3D target The variance of the state estimate in the y-direction is denoted as...

[0201] Step S7.9.7: Determine Is it greater than α? cov or Is it greater than α? cov ,if Greater than α cov or Greater than α cov If the condition is met, proceed to step S7.11; otherwise, proceed to step S7.9.8.

[0202] Step S7.9.8: Determine q”' h' Is the current value equal to 0? h' If not equal, then use q”' h' The sum of the current value plus 1 updates q”' h' The value is determined, and then the process returns to step S7.9. If the value is equal to the given value, then step S7.10 is executed.

[0203] Step S7.10: Convert the reversed initial 3D target trajectory set T backward The h'th 3D target trajectory traj' h' As the 3D target trajectory set T' backward A 3D target trajectory is added to the 3D target trajectory set T' backward middle;

[0204] Step S7.11: Determine if the current value of h' is equal to L. backward If it is not equal, then update the value of h' by adding 1 to the current value of h', and then return to execute step S7.8; if it is equal, then proceed to step S7.12.

[0205] Step S7.12: Reverse the initial 3D target trajectory set T backward After the trajectory validity verification of all 3D target trajectories is completed, the reversed 3D target trajectory set T' is obtained. backward .

[0206] In this embodiment, the legality verification of 3D target trajectories is mainly achieved by calculating the confidence decay, missing value penalty, and historical association of 3D targets in the trajectory to obtain a deterministic score. The exponential decay strategy of 3D target confidence means that the higher the confidence score of a 3D target, the greater the legality of its corresponding 3D target trajectory. For unstable, short-duration, and fragmented false positive trajectories, this method can rapidly reduce the legality of trajectories that have not been observed for a long time. At the same time, the historical deterministic score of the trajectory is added to enhance the tolerance for real targets that are occluded, far away, or briefly lost, effectively improving the accuracy and robustness of automatic labeling of 3D target trajectories, while significantly suppressing the automatic labeling of false positive trajectories.

[0207] To verify the performance of the automatic 3D target trajectory annotation method for point cloud denoising and trajectory verification of this invention, we conducted experiments on the Waymo dataset (https: / / waymo.com / open / download / #perception-dataset) released by Waymo. The Waymo Open dataset is one of the largest and most diverse autonomous driving datasets to date, containing 798 training point cloud sequences, 202 validation point cloud sequences, and 150 test point cloud sequences. Each point cloud sequence (i.e., continuous point cloud data) contains approximately 200 frames (i.e., 200 point cloud frames) and provides annotation information for 3D targets from a 360-degree full-view perspective. Table 1 presents a performance comparison of the automatic 3D target trajectory annotation method of the present invention for point cloud denoising and trajectory verification with other state-of-the-art methods (CenterPoint disclosed in the literature "Center-based 3d object detection and tracking", SimpleTrack disclosed in the literature "impletrack: Understanding and ret-hinking 3d multi-object tracking", ImmotralTrack disclosed in the literature "Immortal tracker: Tracklet never dies", and TrajectoryFormer disclosed in the literature "TrajectoryFormer: 3DObject Tracking Transformer with Predictive Trajectory Hypotheses") on a validation set consisting of 202 validation point cloud sequences of the Waymo Open dataset.

[0208] Table 1. Experimental results of the method of this invention and other methods on the validation set of the Waymo Open dataset (unit: %)

[0209]

[0210] Table 1 shows that the MOTA (Multiple Object Tracking Accuracy) is an important indicator for verifying the performance of automatic 3D target trajectory annotation. FP (False Positives) represents the percentage of false detections, Miss (False Negatives) represents the percentage of missed detections, and Mismatch (ID Switch) represents the percentage of ID switches. As can be seen from Table 1, the automatic 3D target trajectory annotation method of this invention, which combines point cloud denoising and trajectory verification, significantly outperforms other state-of-the-art methods in 3D target trajectory annotation accuracy for Vehicle, Pedestrian, and Cyclist categories. This further verifies the effectiveness of the automatic 3D target trajectory annotation method of this invention in annotating complete 3D target trajectories. In addition to its high accuracy in 3D target annotation, the automatic 3D target trajectory annotation method of point cloud denoising and trajectory verification of this invention also reduces the proportion of false detections, missed detections and ID switching in 3D target trajectories. This further proves that the verified 3D target trajectory can avoid problems such as fragmented trajectory, false positive trajectory and ID switching to a certain extent. Thus, the automatic 3D target trajectory annotation method of point cloud denoising and trajectory verification of this invention can automatically annotate complete and accurate 3D target trajectories.

Claims

1. A method for automatic annotation of 3D target trajectories for point cloud denoising and trajectory verification, characterized in that... Includes the following steps: Step 1: Perform data augmentation on the continuous point cloud data to obtain augmented continuous point cloud data; Step 2: Use time information to compare past point cloud frames with the current point cloud frame in the enhanced continuous point cloud data. X Combining these methods, we can obtain... X Combined point cloud frame data, X The integer is greater than or equal to 2; and a 3D object detection algorithm is used to obtain... X 3D target detection results based on point cloud frame combination data; Step 3, X The 3D object detection results are weighted and fused to obtain the initial 3D object detection result; Step 4: Adjust and filter the initial 3D target detection results using a point density-sensing voxel network to obtain the adjusted 3D target detection results; Step 5: Based on data observation and association strategies, perform trajectory tracking on the adjusted 3D target detection results to generate a forward initial 3D target trajectory set and a reverse initial 3D target trajectory set, wherein the forward initial 3D target trajectory set and the reverse initial 3D target trajectory set are each composed of multiple 3D target trajectories; Step 6: Perform trajectory validity verification on each 3D target trajectory in the two initial 3D target trajectory sets, namely the forward initial 3D target trajectory set and the reverse initial 3D target trajectory set. The forward 3D target trajectory set is formed by using the 3D target trajectories that have passed the validity verification in the forward initial 3D target trajectory set, and the reverse 3D target trajectory set is formed by using the 3D target trajectories that have passed the validity verification in the reverse initial 3D target trajectory set. Step 7: Perform weighted fusion processing on the forward-oriented 3D target trajectory set and the reverse-oriented 3D target trajectory set to obtain the final 3D target trajectory set. The 3D target trajectory is automatically labeled. In step 6, the initial set of positively oriented 3D target trajectories is... The specific process for verifying the legality of each 3D target trajectory is as follows: Step S7.1: Convert the initial set of forwarded 3D target trajectories The number of 3D target trajectories is denoted as Set the set of positively oriented 3D target trajectories as Legality threshold Threshold , variable h, where, , Initialize h to 1; Step S7.2: Convert the initial set of forwarded 3D target trajectories The trajectory of the h-th 3D target The number of 3D targets is denoted as Set variables The trajectory of the h-th 3D target The first 3D target Deterministic score Initialize to 0, Initialize to 2; Step S7.3: Calculate the trajectory of the h-th 3D target. The Middle 3D target Deterministic score The specific process is as follows: Step S7.3.1: Transfer the trajectory of the h-th 3D target. The Middle 3D target timestamp minus the -1 3D target The difference obtained from the timestamps is used as the 3D target. With 3D targets The time difference is denoted as ; Step S7.3.2: Make the 3D target The confidence decay term is ,in, Representing 3D targets Confidence score; Denotes the base of the natural logarithm. This is the multiplication operator; Step S7.3.3: Make the 3D target The missing penalty item is ; Step S7.3.4: Calculate the 3D target using formula (7). Deterministic score : （7） Step S7.3.5: Determine the 3D target Deterministic score Is it greater than the legality threshold? If it is greater than , then proceed to step S7.4; otherwise, proceed to step S7.3.

6. Step S7.3.6: Transfer the trajectory of the h-th 3D target. Center front The 3D targets are combined into a new 3D target trajectory. Using a 3D Kalman filter to analyze the trajectory of a 3D target Prediction is performed to obtain the trajectory of the h-th 3D target. The 3D target exist Orientation state estimation variance and 3D target exist The variance of the state estimate of the direction is The trajectory of the h-th 3D target The 3D target exist The variance of the state estimate of the direction is denoted as The trajectory of the h-th 3D target The 3D target exist The variance of the state estimate of the direction is denoted as ; Step S7.3.7: Determine Is it greater than or Is it greater than ,if Greater than or Greater than If the condition is met, proceed to step S7.5; otherwise, proceed to step S7.3.

8. Step S7.3.8: Determine Is the current value equal to If they are not equal, then use The sum of the current value plus 1 is updated. The value is determined, and then the process returns to step S7.

3. If the value is equal to the given value, then step S7.4 is executed. Step S7.4: Convert the initial set of forwarded 3D target trajectories The trajectory of the h-th 3D target As a set of 3D target trajectories Add a 3D target trajectory to the 3D target trajectory set middle; Step S7.5: Determine if the current value of h is equal to If it is not equal, then update the value of h by adding 1 to the current value of h, and then return to execute step S7.2; if it is equal, then proceed to step S7.

6. Step S7.6: Forwarding the initial 3D target trajectory set All 3D target trajectories undergo trajectory validity verification, resulting in a set of positive 3D target trajectories. ; For the reversed initial 3D target trajectory set The method for verifying the validity of each 3D target trajectory in the process, and the method for performing trajectory validity verification on the initial set of positively oriented 3D target trajectories. The method for verifying the legality of each 3D target trajectory is the same.

2. The automatic annotation method for 3D target trajectory for point cloud denoising and trajectory verification according to claim 1, characterized in that... The specific process of performing data augmentation on the continuous point cloud data in step 1 to obtain the augmented continuous point cloud data is as follows: Step S1.1: Record the continuous point cloud data as Continuous point cloud data The Middle i Each point cloud frame is recorded as Among them, the first continuous point cloud data i Point cloud frame For lidar i The data obtained from the second scan will be continuous point cloud data. The Middle i Point cloud frame The time of being scanned by lidar is recorded as ,in, i =1, 2, ..., I, where I is continuous point cloud data. The total number of midpoint cloud frames; Step S1.2: Transfer continuous point cloud data The Middle i Point cloud frame The timestamp is The result is obtained by formula (1). : （1） in, It is the first in continuous point cloud data i Point cloud frame With the m Point cloud frame The time difference between being scanned by lidar m =1, 2, ..., i ; Step S1.3: From continuous point cloud data True data N real values ​​are selected from the data, where N is greater than or equal to 5 and less than or equal to 10. The N real values ​​include V1 real values ​​of vehicles, V2 real values ​​of pedestrians and V3 real values ​​of cyclists, where V1, V2 and V3 are integers greater than or equal to 0. Will N The actual values ​​were randomly added to the continuous point cloud data after being translated and rotated. The i Point cloud frame In the context of continuous point cloud data The i Point cloud frame Enhance the continuous point cloud data The i Point cloud frame The data before and after the increase are denoted as , For enhanced continuous point cloud data The i A point cloud frame.

3. The automatic annotation method for 3D target trajectory for point cloud denoising and trajectory verification according to claim 2, characterized in that... The first one obtained in step 2 x The combination of point cloud frames is denoted as , x =1, 2, ..., X For the enhanced continuous point cloud data Proceed to the first x Combining the methods in this way, we obtain the first... x Seed cloud frame combination data The specific process is as follows: From enhanced continuous point cloud data The first point cloud frame Initially, each time a point cloud frame is selected, it is determined whether the point cloud exists after that frame. x -1 point cloud frame; if it exists, then append that point cloud frame to the subsequent... x -1 point cloud frames are combined to form the first x A combination of point cloud frames in one of the following ways; if it does not exist, then the first... x The first combination ends, the second x All point cloud frames combined in this way constitute the first... x Combined data of point cloud frames.

4. The method for automatic annotation of 3D target trajectories for point cloud denoising and trajectory verification according to claim 1, characterized in that... In step 3, formula (2) is used to... The 3D object detection results are weighted and fused to obtain the initial 3D object detection result. : （2） in, This represents the weighted fusion function. The first result obtained using a 3D object detection algorithm x Seed cloud frame combination data 3D target detection results.

5. The method for automatic annotation of 3D target trajectories for point cloud denoising and trajectory verification according to claim 1, characterized in that... In step 4, a point density-sensing voxel network thinning function is used to refine the initial 3D target detection results fused in step 3. Adjusting the detection bounding box in the bounding box yields more accurate 3D target detection results. Among them, 3D object detection results From continuous point cloud data The 3D target detection results are composed of I point cloud frames.

6. The automatic annotation method for 3D target trajectory for point cloud denoising and trajectory verification according to claim 1, characterized in that... In step 5, the adjusted 3D target detection results are analyzed based on data observation and association strategies. The specific process of performing trajectory tracking and generating a positive initial set of 3D target trajectories is as follows: Step S6.1: Set the threshold for data observation to... ,in, ; Step S6.2: 3D target detection results The first in i Point cloud frame The 3D object detection result is denoted as 3D target detection results The number of 3D targets is denoted as 3D target detection results The Each 3D target is denoted as ,in =1, 2, 3, ... 3D target detection results The first point cloud frame in 3D object detection results The timestamps of all 3D objects are equal to the first point cloud frame. Timestamps for 3D object detection results The first point cloud frame in 3D object detection results All 3D targets are initialized with trajectories to obtain their 3D target trajectories. The target detection results are then obtained. The first point cloud frame in 3D object detection results Each 3D target's 3D target trajectory is an initial trajectory, and the target detection results are... In the 3D object detection results of each point cloud frame, the number of successful matches for each 3D object trajectory is initialized to 0, and the number of failed matches is initialized to 0. The 3D object detection results are then used. The first point cloud frame in 3D object detection results The 3D target trajectories of all 3D targets constitute the 0th generation 3D target trajectory set. ,in , Represents the set of 0th generation 3D target trajectories The Middle The 3D target trajectory, i.e., the 3D target detection result. The first point cloud frame in 3D object detection results The 3D target trajectory of a 3D target; Step S6.3: Set the iteration count variable r, the maximum iteration count R, and the matching threshold. Threshold and Where R = I-1, =3, =5, , Initialize r to 1; Step S6.4: Perform the r-th iteration to obtain the r-th generation 3D target trajectory set. The specific process is as follows: Step S6.4.1: Set the 3D target trajectory of the (r-1)th generation The number of 3D target trajectories is denoted as Set the variable as and ,Will Initialize to 1; Step S6.4.2: [The sentence is incomplete and requires more context to be translated accurately.] Initialize to 1; Step S6.4.3: For the (r-1)th generation 3D target trajectory set The Middle 3D target trajectory The updated 3D target trajectory is obtained and used as the first... r 3D target trajectory set The 3D target trajectory Add to the r 3D target trajectory set The specific process is as follows: Step S6.4.3.1: Determine the 3D target detection result The 3D target Is the confidence score greater than If greater than If so, proceed directly to step S6.4.3.2; otherwise, determine... Is the current value equal to ,if The current value is not equal to Then adopt The sum of the current value plus 1 is updated. If the value is obtained, then return to step S6.4.3.

1. The current value is equal to ， First, set the 3D target trajectory of generation r-1. The Middle 3D target trajectory As the first r 3D target trajectory set The 3D target trajectory Add to the r 3D target trajectory set In, and make the 3D target trajectory The number of successful matches equals the 3D target trajectory The current value of the number of successful matches, 3D target trajectory The number of matching failures equals the 3D target trajectory The sum of the current value of the number of failed matches plus 1 is then used for judgment. Is the current value equal to ,if The current value is not equal to Then use first The sum of the current value plus 1 is updated. The value is then determined, and the process returns to step S6.4.

2. If... The current value is equal to ， Then proceed to step S6.4.3.7; Step S6.4.3.2: Use a 3D Kalman filter (3DKF) to process the (r-1)th generation 3D target trajectory set. The Middle 3D target trajectory Make a prediction to obtain the corresponding 3D target state, and record it as... ; Step S6.4.3.3: Calculate the 3D target detection results using formulas (3) and (4) respectively. The 3D target Intersection over Union (IoU) of middle bounding boxes and generalized intersection : （3） （4） in, This represents the 3D object detection result. The 3D target With 3D target state The volume of the intersection Indicates 3D object detection results The 3D target With 3D target state The union volume, This indicates that the results include 3D object detection. The 3D target With 3D target state The minimum enclosed volume; Step S6.4.3.4: If Less than or Less than This indicates the 3D object detection result. The 3D target and 3D target state Matching failed; 3D object detection results will be returned. The 3D target For 3D targets that fail to match, step S6.4.3.6 is executed at this time; if Greater than or equal to or Greater than or equal to This indicates the 3D object detection result. The 3D target and 3D target state Matching successful, 3D object detection results will be displayed. The 3D target As the 3D target that has been successfully matched, proceed to step 6.4.3.5; Step S6.4.3.5: Set the 3D target detection results... The 3D target The timestamp equals the Point cloud frame Timestamps and 3D object detection results The 3D target The 3D Kalman filter is updated to obtain the 3D target trajectory, which is the trajectory of the first target. r 3D target trajectory set The 3D target trajectory And make the 3D target trajectory The number of successful matches is equal to the set of 3D target trajectories of generation r-1. The Middle 3D target trajectory The sum of the current number of successful matches plus 1, 3D target trajectory The number of matching failures is equal to the set of 3D target trajectories in generation r-1. The Middle 3D target trajectory The current value of the number of failed matches is used to determine... Is the current value equal to ,if The current value is equal to Then use first The sum of the current value plus 1 is updated. The value is then determined, and the process returns to step S6.4.

2. If... The current value is equal to ， Then proceed to step S6.4.3.7; Step S6.4.3.6: Determine Is the current value equal to ,if The current value is not equal to Then adopt The sum of the current value plus 1 is updated. If the value is obtained, then return to step S6.4.3.

1. The current value is equal to ， First, set the 3D target trajectory of generation r-1. The Middle 3D target trajectory As the first r 3D target trajectory set The 3D target trajectory Add to the r 3D target trajectory set In, and make the 3D target trajectory The number of successful matches equals the 3D target trajectory The current value of the number of successful matches, 3D target trajectory The number of matching failures equals the 3D target trajectory The sum of the current value of the number of failed matches plus 1 is then used for judgment. Is the current value equal to ,if The current value is not equal to Then adopt The sum of the current value plus 1 is updated. The value is then determined, and the process returns to step S6.4.

2. If... The current value is equal to ， Then proceed to step S6.4.3.7; Step S6.4.3.7: Analyze the 3D target detection results. Each 3D target that fails to match is evaluated individually. If the 3D target detection result is... The confidence score of a 3D target that failed to match is higher than Then the timestamp of the 3D target is equal to the timestamp of the first 3D target. Point cloud frame The timestamp is then used to initialize the trajectory of the 3D target, resulting in an initialized trajectory that serves as the 3D target trajectory. This 3D target trajectory is then used as the first... r 3D target trajectory set Add a 3D target trajectory to the first r 3D target trajectory set The After identifying a 3D target trajectory, the number of successful matches and the number of failed matches for that 3D target trajectory are initialized to 0. Step S6.4.3.8: For the currently obtained r-th generation 3D target trajectory set Each of the 3D target trajectories in the set is judged individually. If the set of 3D target trajectories in the r-th generation is... The number of successful matches for a certain 3D target trajectory is less than a threshold. And the number of failed matches exceeds the threshold. Then the 3D target trajectory will be transferred from the first... r 3D target trajectory set Delete it if necessary, otherwise do not process it. Step S6.5: Determine r Is the current value equal to R If they are not equal, then use r The sum of the current value plus 1 is updated. r The value is then determined, and the process returns to step S6.4 for the next iteration. If it equals... R Then the currently obtained R-th generation 3D target trajectory set That is, the set of initial 3D target trajectories for positive transformation. .

7. The automatic annotation method for 3D target trajectory for point cloud denoising and trajectory verification according to claim 6, characterized in that... In step 5, the adjusted 3D target detection results are analyzed based on data observation and association strategies. The specific process of performing trajectory tracking and generating a reversed initial set of 3D target trajectories is as follows: Step S6.6: Set the 3D object detection results The first in Point cloud frame 3D object detection results The timestamps of all 3D objects are equal to the timestamp of the first 3D object. Point cloud frame Timestamps for 3D object detection results The first in Point cloud frame 3D object detection results All 3D targets are initialized with trajectories to obtain their 3D target trajectories. The target detection results are then obtained. The first in Point cloud frame 3D object detection results Each 3D target's 3D target trajectory is an initial trajectory, and the target detection results are... In the 3D object detection results of each point cloud frame, the number of successful matches for each 3D object trajectory is initialized to 0, and the number of failed matches is initialized to 0. The 3D object detection results are then used. The first in Point cloud frame 3D object detection results The 3D target trajectories of all 3D targets constitute the 0th generation 3D target trajectory set. ,in , Represents the set of 0th generation 3D target trajectories The Middle The 3D target trajectory, i.e., the 3D target detection result. The first in Point cloud frame 3D object detection results The 3D target trajectory of a 3D target; Step S6.7: Set the iteration count variable Maximum number of iterations R, matching threshold Threshold and Where R = I-1, =3, =5, , ,Will Initialize to 1; Step S6.8: Proceed to the next step In the nth iteration, we obtain the nth 3D target trajectory set The specific process is as follows: Step S6.8.1: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation 3D target trajectory set The number of 3D target trajectories is denoted as Set the variable as and ,Will Initialize to 1; Step S6.8.2: [The sentence is incomplete and requires more context to be translated accurately.] Initialize to 1; Step S6.8.3: For the first 3D target trajectory set The Middle 3D target trajectory The updated 3D target trajectory is obtained and used as the first... 3D target trajectory set The 3D target trajectory Add to the 3D target trajectory set The specific process is as follows: Step S6.8.3.1: Determine the 3D target detection result The 3D target Is the confidence score greater than If greater than If so, proceed directly to step S6.8.3.2; otherwise, determine... Is the current value equal to ,if The current value is not equal to Then adopt The sum of the current value plus 1 is updated. If the value is obtained, then return to step S6.8.3.

1. The current value is equal to ， Then first the first 3D target trajectory set The Middle 3D target trajectory As the first 3D target trajectory set The 3D target trajectory Add to the 3D target trajectory set In, and make the 3D target trajectory The number of successful matches equals the 3D target trajectory The current value of the number of successful matches, 3D target trajectory The number of matching failures equals the 3D target trajectory The sum of the current value of the number of failed matches plus 1 is then used for judgment. Is the current value equal to ,if The current value is not equal to Then use first The sum of the current value plus 1 is updated. The value is then determined, and the process returns to step S6.8.

2. If... The current value is equal to ， Then proceed to step S6.8.3.7; Step S6.8.3.2: Use a 3D Kalman filter to apply the first... 3D target trajectory set The Middle 3D target trajectory Make a prediction to obtain the corresponding 3D target state, and record it as... ; Step S6.8.3.3: Calculate the 3D target detection results using formulas (5) and (6) respectively. The 3D target Intersection over Union (IoU) of middle bounding boxes and generalized intersection : （5） （6） in, This represents the 3D object detection result. The 3D target With 3D target state The volume of the intersection Indicates 3D object detection results The 3D target With 3D target state The union volume, This indicates that the results include 3D object detection. The 3D target With 3D target state The minimum enclosed volume; Step S6.8.3.4: If Less than or Less than This indicates the 3D object detection result. The 3D target and 3D target state Matching failed; 3D object detection results will be returned. The 3D target For 3D targets that fail to match, step S6.8.3.6 is executed at this time; if Greater than or equal to or Greater than or equal to This indicates the 3D object detection result. The 3D target and 3D target state Matching successful, 3D object detection results will be displayed. The 3D target As the 3D target that has been successfully matched, proceed to step 6.8.3.5; Step S6.8.3.5: Set the 3D target detection results... The 3D target The timestamp equals the Point cloud frame Timestamps and 3D object detection results The 3D target The 3D Kalman filter is updated to obtain the 3D target trajectory, which is the trajectory of the first target. 3D target trajectory set The 3D target trajectory And make the 3D target trajectory The number of successful matches is equal to the number of times the first match was made. 3D target trajectory set The Middle 3D target trajectory The sum of the current number of successful matches plus 1, 3D target trajectory The number of failed matches is equal to the number of times the match failed. 3D target trajectory set The Middle 3D target trajectory The current value of the number of failed matches is used to determine... Is the current value equal to ,if The current value is equal to Then use first The sum of the current value plus 1 is updated. The value is then determined, and the process returns to step S6.8.

2. If... The current value is equal to ， Then proceed to step S6.8.3.7; Step S6.8.3.6: Determine Is the current value equal to ,if The current value is not equal to Then adopt The sum of the current value plus 1 is updated. If the value is obtained, then return to step S6.8.3.

1. The current value is equal to ， Then first the first 3D target trajectory set The Middle 3D target trajectory As the first 3D target trajectory set The 3D target trajectory Add to the 3D target trajectory set In, and make the 3D target trajectory The number of successful matches equals the 3D target trajectory The current value of the number of successful matches, 3D target trajectory The number of matching failures equals the 3D target trajectory The sum of the current value of the number of failed matches plus 1 is then used for judgment. Is the current value equal to ,if The current value is not equal to Then use first The sum of the current value plus 1 is updated. The value is then determined, and the process returns to step S6.8.

2. If... The current value is equal to ， Then proceed to step S6.8.3.7; Step S6.8.3.7: Analyze the 3D target detection results. Each 3D target that fails to match is evaluated individually. If the 3D target detection result is... The confidence score of a 3D target that failed to match is higher than Then the timestamp of the 3D target is equal to the timestamp of the first 3D target. Point cloud frame The timestamp is then used to initialize the trajectory of the 3D target, resulting in an initialized trajectory that serves as the 3D target trajectory. This 3D target trajectory is then used as the first... 3D target trajectory set Add a 3D target trajectory to the first 3D target trajectory set The After identifying a 3D target trajectory, the number of successful matches and the number of failed matches for that 3D target trajectory are initialized to 0. Step S6.8.3.8: For the currently obtained first... 3D target trajectory set The trajectory of each 3D target in the data is judged separately. If the first... 3D target trajectory set The number of successful matches for a certain 3D target trajectory is less than a threshold. And the number of failed matches exceeds the threshold. Then the 3D target trajectory will be transferred from the first... 3D target trajectory set Delete it if necessary, otherwise do not process it. Step S6.9: Determine Is the current value equal to R If they are not equal, then use The sum of the current value plus 1 is updated. The value is then determined, and the process returns to step S6.8 for the next iteration. If it equals... R Then the currently obtained R-th generation 3D target trajectory set That is, the set of inverted initial 3D target trajectories. .

Images