A trajectory management method based on long-term and short-term retention prediction trajectory

Abstract

The present invention discloses a trajectory management method based on long- and short-term retention of predicted trajectories, which relates to the field of 3D multi-target tracking, including: S1. In 3D multi-target tracking based on point cloud data detection, for the input current frame detection target, a predicted trajectory is obtained using a 3D Kalman filter algorithm; S2. Association matching is performed through a trajectory management module. If the match is successful, the predicted successful trajectory is updated; otherwise, the predicted trajectory and the detection target are managed using long and short time periods respectively, the current frame detection target is initialized as a potential predicted trajectory, and added to the dormant trajectory set; the detection target of the next frame is associated and matched until all targets are detected. The present invention retains a complete predicted trajectory for the observation target in a complex dynamic environment. When the target that temporarily leaves the field of view reappears, it can be successfully associated and matched, thereby reducing the number of target ID switches, reducing the error tracking rate, and increasing the accuracy of 3D point cloud multi-target tracking.

Description

Technical Field

[0001] The present invention relates to the field of 3D multi-target tracking, and more particularly to a trajectory management method based on long-term and short-term retention prediction trajectories. Background Art

[0002] Trajectory management is a crucial component of detection-based 3D multi-target tracking. This module is primarily responsible for generating new targets, continuously tracking targets, detecting target disappearance, handling occlusions, and associating tracks. In complex, dynamic environments with numerous targets, overlapping targets, or frequent occlusions, effective trajectory management can improve the performance and stability of point cloud multi-target tracking systems, reduce target ID switching, and ensure accurate target tracking.

[0003] In most detection-based 3D point cloud multi-target tracking, common trajectory management methods are as follows:

[0004] First, for successfully associated detected targets and predicted trajectories, the detected targets are used to update the predicted trajectories. Second, for unmatched detected targets, they are considered potential new trajectories and are confirmed as new target trajectories only after they have been detected for a certain number of consecutive frames (usually five). Predicted trajectories that fail to match are considered to be about to leave the field of view. After failing to match them for several consecutive frames (usually five frames), they are considered lost and deleted. Ideally, this method helps retain target trajectories that failed to match due to single-frame misses while effectively deleting trajectories that have left the field of view. However, in practical applications, due to complex dynamic environments with numerous targets, overlapping targets, and frequent occlusion, traditional trajectory management methods often lead to problems such as misassociation and premature trajectory termination. For example, a method and system for predicting the trajectory of surrounding vehicles based on the fusion of long and short-term motion trajectories can generate future motion trajectories, but it cannot handle occlusion scenarios lasting for dozens of frames, resulting in misassociation and premature trajectory termination. The specific issues of misassociation and premature trajectory termination are as follows:

[0005] 1. Wrong association

[0006] It usually refers to the situation where a tracked object is mistakenly associated with another target in 3D point cloud multi-target tracking. The traditional trajectory management method has a visualization diagram of the wrong association, such as Figure 10As shown in the figure, the black box represents the true value, the blue box represents the detection result, and the orange box represents the prediction result. In frame 56, both detection and prediction identify target ID 19. However, in frame 60, target ID 19 disappears (perhaps due to occlusion or temporary loss of field of view). At this point, the traditional trajectory management method cannot predict target ID 19 in a timely manner and instead mistakenly associates it with target ID 21. Therefore, when target ID 19 reappears in frame 64, its predicted value is target ID 21, that is, the target ID 19 value switches to ID 21, resulting in target tracking errors and reduced target tracking accuracy.

[0007] 2. Early termination of the trajectory

[0008] Usually refers to the situation in which the tracking system mistakenly terminates or deletes the trajectory of a target in 3D point cloud multi-target tracking, while the target has not actually left the field of view. This situation is usually caused by factors such as target occlusion, detection errors, or poor tracking algorithm performance. Premature trajectory termination will cause the tracking system to lose continuous tracking of the target, affecting the continuity and accuracy of tracking. Traditional trajectory management methods Visualization diagram of premature trajectory termination, such as Figure 11 As shown in Figure 2. The black box represents the true value, the blue box represents the detection result, and the green and red boxes represent the prediction results.

[0009] from Figure 11 As can be seen, after frame 56, target ID 19 disappears. Traditional trajectory management methods can predict the trajectory of target ID 19 from frames 58 to 60. However, after frame 62, these methods are unable to predict the trajectory of target ID 19. Therefore, when target ID 19 reappears in frame 68, it is tracked as a new target ID 21. This results in a target ID switch, causing tracking errors and reduced tracking accuracy.

[0010] In summary, traditional trajectory management methods apply the same short tracking cycle (within 5 frames) to all targets. This results in low-confidence targets (falsely detected targets) being tracked for multiple frames before being removed. High-confidence targets that are occluded across multiple frames (missed targets) may be removed prematurely, leading to tracking loss. This situation leads to frequent switching of target IDs, increasing the rate of false tracking and reducing the accuracy of multi-target tracking in 3D point clouds. To address this issue, this paper proposes a trajectory management method based on long and short tracking cycles. Summary of the Invention

[0011] An object of the present invention is to solve at least the above problems and / or disadvantages and to provide at least the advantages which will be described hereinafter.

[0012] In order to achieve these objectives and other advantages of the present invention, a trajectory management method based on long-term and short-term retention prediction trajectory is provided, comprising:

[0013] S1. In 3D multi-target tracking based on point cloud data detection, for the current frame detection target input by the 3D target detector , using the 3D Kalman filter algorithm to get the predicted trajectory X ;

[0014] S2, through the trajectory management module and X Perform association matching;

[0015] If the match is successful, Defined as detection success target , using 3D Kalman filter algorithm to predict successful trajectory Update trajectory status;

[0016] For detection targets that are not successfully matched , adopt short-term cycle management and Defined as unmatched success target , used to initialize the potential prediction trajectory ;

[0017] For the predicted trajectory that was not matched successfully X , which is defined as the unmatched successful prediction trajectory , and Adopt long-term cycle management to continuously maintain the corresponding forecast status;

[0018] S3. And the corresponding status Add to collection middle;

[0019] S4. 、 、 All of them are used as the detection targets of the next frame, and return to S2 to complete the association matching of each target until all targets are detected and output .

[0020] Preferably, in S1, the predicted trajectory X The acquisition process is:

[0021] S10, based on detection target Construct the following state vector set :

[0022]

[0023] In the above formula, They are the center position of the detection target at time t in the world coordinate system, is the orientation angle of the detection target at time t, is the length, width and height of the detection target at time t, Detect the velocity component of the target at time t;

[0024] S11. Predict the state vector set of the next frame based on the linear motion model :

[0025]

[0026] In the above formula, is the center position of the detection target in the world coordinate system at time t+1, is the velocity component of the detected target at time t+1, and , , , , , , It is the difference between the current frame timestamp t+1 and the previous frame timestamp t;

[0027] S12, use 3D Kalman filter algorithm to predict the trajectory X Characterized by:

[0028] .

[0029] Preferably, in S2, the predicted successful trajectory The update is done by:

[0030] S210: input detection target If the match is successful, the target is detected by the following 3D Kalman filter algorithm: The corresponding associated detection box To update:

[0031]

[0032] In the above formula, is the complete state output by the 3D Kalman filter algorithm step, and , is the Kalman filter update function;

[0033] S211, associate the detection frame with Assign to 、 Complete the update operation:

[0034]

[0035] S212, add 1 to the cumulative number of hits for which the trajectory successfully matches the detection frame;

[0036] S213, determine whether hits reaches the minimum number of hits M_hits If so, mark it as alive Status Otherwise, mark birth State, enter the next frame trajectory pool to execute S3.

[0037] Preferably, in S2, the long-term period management is a method of using 3D Kalman filtering to perform trajectory prediction:

[0038] S220, obtain the trajectory status after the previous frame update , to the trajectory state of the current frame Make predictions;

[0039] S221, based on the covariance of the previous frame, the covariance matrix of the current frame is calculated using the following formula: To update:

[0040]

[0041] In the above formula, F is included The Jacobian matrix of is the covariance of the previous frame, Q is the process noise covariance matrix;

[0042] S222, do not change the state of the trajectory, and and Save it for prediction of the next frame and participate in data association of the next frame.

[0043] Preferably, in S2, if and If the following formula is satisfied, Assign values ​​to potential new prediction trajectories , initialize the hit count to 0:

[0044]

[0045] In the above formula, 3DGIOU is a calculation and Similarity association algorithm, is a detection box of the current frame, is the predicted state of the i-th trajectory in the trajectory pool, is the trajectory pool collection, Indicates that all tracks in the track pool are correlated and compared.

[0046] The present invention has at least the following beneficial effects: Instead of deleting predicted trajectories between targets in the short term, as in traditional methods, the present invention proposes a trajectory management method based on retaining predicted trajectories over long and short time periods. This method retains a complete predicted trajectory for targets observed in complex dynamic environments. When a target that temporarily leaves the field of view reappears, it can be successfully associated and matched. This reduces the number of target ID switches, lowers the tracking error rate, and improves the accuracy of 3D point cloud multi-target tracking. Experiments were designed for quantitative and qualitative comparative analysis, verifying the effectiveness of the long and short time trajectory management method.

[0047] Other advantages, objectives and features of the present invention will be reflected in part from the following description and will be understood by those skilled in the art through study and practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a diagram showing the effect of using the improved trajectory management method of the present invention when the error association problem occurs;

[0049] Figure 2 This is a diagram showing the effect of using the improved trajectory management method of the present invention to improve the problem of early trajectory termination;

[0050] Figure 3 A comparison chart of the AMOTA evaluation index between the present invention and the traditional trajectory management method;

[0051] Figure 4 A comparison chart of the present invention and the traditional trajectory management method in terms of AMOTP evaluation indicators;

[0052] Figure 5 A comparison chart of the present invention and the traditional trajectory management method in terms of MOTA evaluation indicators;

[0053] Figure 6 A comparison chart of the IDS evaluation indicators between the present invention and the traditional trajectory management method;

[0054] Figure 7 A visual diagram of the traditional trajectory management method;

[0055] Figure 8 A visual diagram of the improved trajectory management method of the present invention;

[0056] Figure 9 This is a system flow chart of the present invention for multi-target tracking based on point cloud data in a complex dynamic environment;

[0057] Figure 10 Schematic diagram of error association of traditional trajectory management methods;

[0058] Figure 11Schematic diagram of early trajectory termination of traditional trajectory management methods;

[0059] Figure 12 This is a schematic diagram of campus target ID6 in frames 15, 21, and 27;

[0060] Figure 13 For the method of the present invention, the campus target ID6 is detected in the 15th, 21st and 27th frames;

[0061] Figure 14 Schematic diagram of campus target ID6 detection at frames 15, 21, and 27 using the traditional AB3DMOT method;

[0062] Figure 15 Schematic diagram of campus target ID6 detection in frames 15, 21, and 27 using the traditional Chiu method;

[0063] Figure 16 Schematic diagram of campus target ID6 detection at frames 15, 21, and 27 using the traditional CenterPoint method;

[0064] Figure 17 Schematic diagram of campus target ID6 detection at frames 15, 21, and 27 using the traditional CBMOT method;

[0065] Figure 18 Schematic diagram of campus target ID6 detected in frames 15, 21, and 27 using the traditional SimpleTrack method. DETAILED DESCRIPTION

[0066] The present invention will be described in further detail below in conjunction with the accompanying drawings so that those skilled in the art can implement the invention with reference to the description.

[0067] The present invention analyzes the traditional trajectory management method, which is based on deleting the predicted trajectory that fails to match successfully within a few frames, and proposes an improved trajectory management method, that is, a trajectory management method that retains the target predicted trajectory for a long time. Figure 9 As shown, the processing steps are as follows: for the input current frame, detect D t , first use 3DKF to predict all the trajectories in the trajectory pool, and then use the Hungarian algorithm to match and detect D tIf the match is successful with the predicted trajectory X (i.e., data association), Kalman update (position / size / speed) is performed, the trajectory hit count is +1, and the cumulative hit count of the trajectory is judged to be ≥M_hits (M_hits defaults to 1, if so, it is marked as alive state; otherwise, the birth state is maintained; and for the predicted trajectory that is not successfully matched (long-term management is used), the Kalman prediction value is used as the current state, and no hit count and state transition are performed; for the detection target that is not successfully matched (short-term management is used), it is initialized to a new trajectory: state vector Z=[D_pos, 0_vel] (position=detection value, speed=0), marked as birth state, hit count=0, and added to the trajectory pool; further, the alive state and the trajectory that is successfully matched in the current frame are filtered out from the trajectory pool, and the trajectory pool of the next frame = all current trajectories (including successful matching, failed matching, and newly initialized trajectories). The specific operation process is as follows:

[0068] S1, the 3D target detector (CenterPoint) outputs the current frame detection target , They are the center position of the detection target at time t in the world coordinate system, is the orientation angle of the detection target at time t, is the length, width, and height of the detection target at time t. The 3D Kalman filter algorithm is used to predict the seven dimensions of position, orientation angle, and size to form the trajectory prediction X. The prediction steps are as follows:

[0069] S10, state vector definition:

[0070]

[0071] Among them, are the center position of the detection target at time t in the world coordinate system, is the orientation angle of the detection target at time t, is the length, width and height of the detection target at time t, Detect the velocity component of the target at time t;

[0072] S11. Prediction step (predict)

[0073] Use the linear motion model to predict the next frame state:

[0074]

[0075]

[0076]

[0077]

[0078]

[0079]

[0080] in, t The target size and orientation angle at time +1 are t Remain unchanged at all times.

[0081] The predicted trajectory X is represented as:

[0082] .

[0083] S2, when the target is detected and predicted trajectory X When the association match is successful, the detection target is used Update predicted trajectory , and its update steps are:

[0084] For input detection target: , if the match is successful (i.e., it has a corresponding associated detection box) and predicted trajectory X :

[0085]

[0086]

[0087] In the above formula, is the complete state output by the 3D Kalman filter algorithm step, that is , is the Kalman filter update function, To match the successful prediction trajectory, To match the detection target successfully;

[0088] Add 1 to the trajectory hit count and get hits , where hits represents the cumulative number of frames in which the trajectory successfully matches the detection box.

[0089] Determine whether hits reaches M_ hits (i.e. if hits ≥M_ hits (M_ hits =1 default value)), if it is, it will be marked as alive state Otherwise, mark birth State, enter the next frame trajectory pool to execute S3.

[0090] In S2, if the matching is unsuccessful, the detection target that is not successfully matched is Perform initialization operation, and if the detection target is not successfully matched and predicted trajectory satisfy:

[0091]

[0092] Assign values ​​to potential new prediction trajectories In the above formula, 3DGIOU is an association algorithm, which is used to calculate the detection target and predicted trajectory similarity. A detection box of the current frame, The predicted state of the i-th trajectory in the trajectory pool, is the trajectory pool collection, Indicates that all tracks in the track pool are correlated and compared.

[0093] In S2, if the matching is unsuccessful, the predicted trajectory of the unsuccessful matching is ,adopting long-term cycle management and using 3D Kalman filter algorithm to continuously maintain its prediction state.

[0094] S210. Get the last valid status:

[0095]

[0096] The track state after the previous frame update.

[0097] S211. Use the 3D Kalman filter algorithm to perform linear motion model prediction on the current frame state:

[0098] 1. Matrix definition

[0099] 1) State transfer matrix F, F is a 10*10 matrix derived from the uniform linear motion model. Its structure is based on the state vector Get, that is:

[0100]

[0101] Among them, the position component Affected by speed, that is, the new position is the original position plus speed multiplied by the time difference, and the other components (towards the angle θ 、Dimensions (Length l, Width w, high h ), velocity component ( ) remain unchanged.

[0102] 2) Covariance matrix definition P , PThis is a 10×10 matrix, with the diagonal elements representing the variances of each state component and the off-diagonal elements representing the covariances between two state components. Let the initial covariance matrix be a diagonal matrix with the diagonal elements representing the initial variances of each state component. You can set the initial position variance to 1.0, the initial orientation variance to 0.1, the initial size variance to 0.2, and the initial velocity variance to 2.0. The covariance matrix will be continuously updated as the filtering process progresses.

[0103] 3) Process noise covariance matrix Q , Q is a 10×10 matrix, initialized as a diagonal matrix:

[0104]

[0105] in, , , , , , , , , , , 、 、 are the covariances of the position components, 、 、 are the covariances of the velocity components, 、 、 are the covariances of length, width, and height, respectively. is the covariance of the heading angle.

[0106] 2. Location Prediction

[0107]

[0108]

[0109]

[0110] In the above formula, It is the difference between the current frame timestamp t and the previous frame timestamp t-1.

[0111] 3. Speed ​​remains constant

[0112]

[0113]

[0114]

[0115] Heading angle , the dimensions (length, width, height) remain unchanged, that is: , , ;

[0116] The output is:

[0117] 4. Update the covariance matrix (reflecting the increased uncertainty)

[0118]

[0119] Where F is the state transfer matrix; is the covariance of the previous frame; Q is the process noise covariance matrix.

[0120] 5. Maintain track status

[0121] If the match is unsuccessful:

[0122] 1) The hit count of the trajectory does not increase;

[0123] 2) The status of the track (i.e. alive or birth) remains unchanged, and the track will not be deleted even if multiple frames of matching fail.

[0124] 6. Prepare for the next frame

[0125] The updated status and the covariance matrix Save it for prediction of the next frame. The track will continue to remain in the track pool and participate in the data association of the next frame.

[0126] S3, the trajectory set output by this frame is the "alive" state and the trajectory of successful matching , the next frame input trajectory set is the trajectory prediction of the successful match , unsuccessfully matched predicted trajectories and is initialized as a new predicted trajectory , that is, this step only outputs the trajectory that meets the conditions, that is, the trajectory set output by the current frame The trajectory of the "alive" state and successful matching , the next frame input trajectory set Trajectory prediction for successful matching , unsuccessfully matched predicted trajectories and is initialized as a new predicted trajectory ;

[0127]

[0128]

[0129] Among them, the current frame output trajectory The state must be alive and the current frame must be matched successfully, excluding the birth state and only predicting the maintained trajectory.

[0130] Through this trajectory management method, even in complex dynamic environments with many targets, overlapping targets, and frequent occlusions, the retained predicted trajectories will repeatedly predict their positions in future frames. The predicted trajectories can be retained for all targets that have been observed at least once. Once the target appears near the predicted trajectory again, it can be associated and matched with the corresponding predicted trajectory again, thereby reducing the frequent switching of target IDs, reducing the error tracking rate, and improving the accuracy of target tracking.

[0131] The code of the trajectory management algorithm is as follows:

[0132] Input: 3D object detection

[0133] Output: Output trajectory

[0134] Time step:

[0135] Track collection:

[0136]

[0137] It can be seen from the above code that in the algorithm of the trajectory management method proposed in the present invention, its input is the target frame detected by the 3D detector. , the output is a trajectory set , when the tracking is not completed, perform the following loop.

[0138] like Figure 1 As shown in the figure, the improved trajectory management method based on long and short time can avoid the problem of incorrect association in the traditional trajectory management method. Figure 1 In the figure, the black box represents the true value, the blue box represents the detection result, and the red box represents the prediction result. Figure 1 It can be seen intuitively that in the 56th frame, both detection and prediction identified target ID19; in the 60th frame, target ID19 disappeared (probably because it was blocked or temporarily left the field of view), but the trajectory of target ID19 can still be predicted using the method of the present invention, so when target ID19 appears again in the 64th frame, its predicted value is target ID19, and the target's identity ID does not switch, and target ID19 is continuously tracked.

[0139] like Figure 2 As shown in Figure 2, the problem of premature termination of trajectories in traditional trajectory management methods can also be avoided by the improved trajectory management method. Figure 2 In the figure, the black box represents the true value, the blue box represents the detection result, and the orange box represents the prediction result. Figure 2 As can be seen intuitively in the figure, target ID 19 disappears after frame 56. Although target ID 19 disappears in frame 60, our improved trajectory management method is able to predict its trajectory. Therefore, when target ID 19 reappears in frame 64, it is not tracked as a new target ID 21, as with traditional trajectory management methods. Instead, it remains as target ID 19, preventing a target ID switch and thus reducing tracking accuracy.

[0140] Experimental analysis is carried out below through Examples and Comparative Examples:

[0141] Example:

[0142] This method was implemented in a campus environment for actual testing, and a visual comparison was made between this method and other methods on campus data. From frame 15 to frame 27, target ID6 temporarily left the field of view due to occlusion in frame 21 and appeared in frame 27. Its image is shown in the figure below. Figure 12 As shown, Figure 12 (a) is the 15th frame, Figure 12 (b) is the 21st frame, Figure 12 (c) is the 27th frame.

[0143] The comparison results of this algorithm with other five algorithms are shown in the following table. Figures 13-18 As shown in the figure, from top to bottom are the method in this paper, A Baseline for 3D Multi-Object Tracking (AB3DMOT for short), Probabilistic 3dmulti-object tracking for autonomousdriving (Chiu for short), Center-based 3dobject detection and tracking (CenterPoint for short), Score refinement forconfidence-based 3D multi-object tracking (CBMOT for short), and Understanding andrethinking 3d multi-object tracking (SimpleTrack for short) method.

[0144] from Figures 13-18It can be seen that target ID6 disappears in the 21st frame, and AB3DMOT and Chiu are unable to predict the trajectory of the target in the 23rd frame. Therefore, when target ID6 appears again in the 27th frame, it is identified as target ID9. Although the CenterPoint, CBMOT, and SimpleTrack methods can predict the trajectory in the 23rd frame, target ID6 is predicted as ID12, ID16, and ID11. However, when the method in this paper is used, it can still be predicted as ID6 in the 23rd frame, so when target ID6 appears again in the 27th frame, it is still identified as target ID6. Therefore, the present invention does not cause the switching of target identity, thereby improving tracking accuracy.

[0145] In summary, in the complex dynamic environment of the campus environment, compared with other methods, the target ID switching times of the present invention are less and the tracking effect is better.

[0146] Verification example:

[0147] By conducting experiments on the traditional trajectory management method and the improved trajectory management method, quantitative and qualitative analysis is performed to verify the effect of the present invention;

[0148] This verification example uses the nuScenes dataset as a sample. This dataset contains a large amount of autonomous driving data, which contains a large number of objects, and objects overlap or occlude each other. Therefore, this paper conducts a comparison of indicators on the nuScenes dataset to verify the effectiveness of the improved trajectory management method selected in this paper.

[0149] 1. Quantitative Analysis

[0150] Table 1 shows a quantitative comparison of the two trajectory management methods on the nuScenes dataset. Table 1 shows that the improved trajectory management method achieves a 5.9% improvement in the AMOTA (Average Multi-Object Tracking Accuracy) evaluation metric compared to the traditional trajectory management method; a 6.9% reduction in the AMOTP (Average Multi-Object Tracking Precision) evaluation metric; a 0.5% increase in the MOTA (Multi-Object Tracking Accuracy) evaluation metric; and a 235-fold reduction in the IDS (Number of Identity Switches) evaluation metric. This verifies the effectiveness of the long- and short-term trajectory management method.

[0151] Table 1

[0152]

[0153] Among them, the evaluation indicators of AMOTA, AMOTP, MOTA, and IDS for seven categories, including bicycles, buses, cars, motorcycles, pedestrians, trailers, and trains, are compared. Figure 3 - Figure 6 shown.

[0154] from Figure 4 It can be seen that the improved trajectory management method has the lowest AMOTP index for the seven categories of bicycle, bus, car, motorcycle, pedestrian, trailer, and truck, and the overall effect is good. Figure 3 、 Figure 5 As can be seen from the above, the car and pedestrian categories are not much different from the traditional trajectory management method, but the other five categories have improved significantly. Figure 6 As can be seen from the figure, the improved trajectory management method is more effective than the traditional trajectory management method in terms of the number of target ID switches. The effect is more obvious for the car and pedestrian categories, with the target ID value reduced by 93 and 142 times respectively.

[0155] 2. Qualitative analysis

[0156] Since there are many targets in the car category in the nuScenes dataset and targets are easily occluded, the traditional trajectory management method and the improved trajectory management method are compared visually in the car category, as shown in the following figure. Figure 7 - Figure 8 As shown in Figure 5, acquisition is performed every 6 frames from frame 56 to frame 68. At frame 62, target ID 19 temporarily disappears or leaves the point cloud field of view due to target occlusion or overlap between targets, and does not reappear until frame 68.

[0157] Specifically, Figure 7 It is the traditional trajectory management method. Figure 8 It is an improved trajectory management method. The black box represents the true value, the blue box represents the detection result, the orange and red boxes represent the prediction results, and the dotted box contains the ID, detection and prediction results of the target in the frame. Figure 7 - Figure 8As can be seen from the visual comparison chart, the traditional trajectory management method cannot predict the trajectory of target ID19 when it disappears in the 62nd frame. Therefore, when target ID19 appears again in the 68th frame, it will be identified as another target with ID 21. And the target IDs following target ID19 have also changed successively, that is, the subsequent target ID21 and target ID23 have also switched to ID23 and ID25 respectively. Therefore, the traditional trajectory management method will cause the target ID to switch. However, when using the improved trajectory management method based on long and short time, when target ID19 disappears, its trajectory can still be predicted, so when target ID19 appears again, it is still identified as target ID19. Similarly, the subsequent target IDs are also identified one after another. Therefore, the improved trajectory management method does not cause the target identity to switch, thereby improving tracking accuracy.

[0158] The above solution is only an illustration of a preferred embodiment, but is not limited thereto. When implementing the present invention, appropriate replacements and / or modifications can be made according to user needs.

[0159] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and exemplary embodiments. They can be applied to a variety of fields suitable for the present invention. Further modifications will be readily apparent to those skilled in the art. Therefore, the present invention is not limited to the specific details and illustrations shown and described herein without departing from the general concept defined by the claims and their equivalents.

Claims

1. A trajectory management method based on long-term and short-term retention prediction trajectory, characterized in that: include: S1. In 3D multi-target tracking based on point cloud data detection, for the current frame detection target input by the 3D target detector , using the 3D Kalman filter algorithm to get the predicted trajectory X ; S2, through the trajectory management module and X Perform association matching; If the match is successful, Defined as detection success target , using 3D Kalman filter algorithm to predict successful trajectory Update trajectory status; For detection targets that are not successfully matched , adopt short-term cycle management and Defined as unmatched success target , used to initialize the potential prediction trajectory ; For the predicted trajectory that was not matched successfully X , which is defined as the unmatched successful prediction trajectory , and Adopt long-term cycle management to continuously maintain the corresponding forecast status; S3, And the corresponding status Add to collection middle; S4. 、 、 All of them are used as the detection targets of the next frame, and return to S2 to complete the association matching of each target until all targets are detected and output ; In S2, the successful trajectory is predicted The update is done by: S210: input detection target If the match is successful, the target is detected by the following 3D Kalman filter algorithm: The corresponding associated detection box To update: In the above formula, is the complete state output by the 3D Kalman filter algorithm step, and , is the Kalman filter update function; S211, associate the detection frame with Assign to 、 Complete the update operation: S212, add 1 to the cumulative number of hits for which the trajectory successfully matches the detection frame; S213, determine whether hits reaches the minimum number of hits M_hits If so, mark it as alive Status Otherwise, mark birth State, enter the next frame trajectory pool to execute S3.

2. The trajectory management method based on long-term and short-term retention prediction trajectory according to claim 1, characterized in that: In S1, the predicted trajectory X The acquisition process is: S10, based on detection target Construct the following state vector set : In the above formula, They are the center position of the detection target at time t in the world coordinate system, is the orientation angle of the detection target at time t, is the length, width and height of the detection target at time t, Detect the velocity component of the target at time t; S11. Predict the state vector set of the next frame based on the linear motion model : In the above formula, is the center position of the detection target in the world coordinate system at time t+1, is the velocity component of the detected target at time t+1, and , , , , , , It is the difference between the current frame timestamp t+1 and the previous frame timestamp t; S12, use 3D Kalman filter algorithm to predict the trajectory X Characterized by: 。 3. The trajectory management method based on long-term and short-term retention prediction trajectory according to claim 1, characterized in that: In S2, the long-term period management is to use 3D Kalman filtering to perform trajectory prediction: S220, obtain the trajectory state after the previous frame update , to the trajectory state of the current frame Make predictions; S221, based on the covariance of the previous frame, the covariance matrix of the current frame is calculated using the following formula: To update: In the above formula, F is included The Jacobian matrix of is the covariance of the previous frame, Q is the process noise covariance matrix; S222, do not change the state of the trajectory, and and Save it for prediction of the next frame and participate in data association of the next frame.

4. The trajectory management method based on long-term and short-term retention prediction trajectory according to claim 1, characterized in that: In S2, if and If the following formula is satisfied, Assign values ​​to potential new prediction trajectories , initialize the hit count to 0: In the above formula, 3DGIOU is a calculation and Similarity association algorithm, is a detection box of the current frame, is the predicted state of the i-th trajectory in the trajectory pool, is the trajectory pool collection, Indicates that all tracks in the track pool are correlated and compared.

Images