Three-dimensional multi-target tracking system and method based on damping window mechanism

Abstract

The invention relates to the technical field of computer vision, in particular to a three-dimensional multi-target tracking system and method based on a damping window mechanism, and aims to realize high-precision three-dimensional target detection by deeply fusing multi-modal data of a camera and a radar through a DeepFusion algorithm. Three-dimensional extended Kalman filtering (EKF) is adopted for target state prediction, so that the prediction accuracy is remarkably improved; a dynamic trajectory-oriented optimization algorithm is introduced, so that the problems of mismatching and missed matching in target association are effectively reduced; meanwhile, the system adopts a damping window mechanism for managing the life cycle of the target trajectory, so that the problem of premature termination of the trajectory caused by transient missing detection or false detection is avoided. The method can effectively improve the efficiency of multi-target tracking, has a high anti-interference capability, and is suitable for target tracking tasks in a complex environment.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and particularly relates to a three-dimensional multi-object tracking system and method based on a damping window mechanism. Background Art

[0002] Multi-object tracking is a core technology in many intelligent application fields such as automotive autonomous driving, traffic flow detection, security monitoring, robotics, and action recognition. With the maturity of three-dimensional information acquisition technology and the development of deep learning in three-dimensional object detection, three-dimensional multi-object tracking methods based on deep neural networks have gradually emerged. Three-dimensional multi-object tracking provides the trajectories of surrounding objects to assist robots or vehicles in performing more intelligent path planning and obstacle avoidance. Compared with other data forms, three-dimensional point clouds can provide rich geometric, shape, and scale information and achieve accurate object tracking in strong or weak light environments.

[0003] In the multi-object tracking task, accurately detecting and tracking targets (such as vehicles, pedestrians, etc.) is a key technology for achieving autonomous driving and intelligent monitoring. In recent years, object detection algorithms based on deep learning have made significant progress, especially the DeepFusion algorithm, which can effectively improve the detection accuracy of three-dimensional objects by fusing radar and camera data.

[0004] However, existing three-dimensional multi-object tracking methods still have deficiencies in data association, motion estimation, etc. Especially in scenes with dense targets and severe occlusions, the tracking performance significantly deteriorates. Traditional trajectory management methods (such as counting or confidence-based strategies) are prone to ignoring low-confidence targets, resulting in trajectory breaks or false terminations. In addition, using a single motion model for state prediction performs poorly when dealing with non-linear motion models and is prone to tracking error accumulation. Summary of the Invention

[0005] The purpose of the present invention is to overcome the defects of the prior art and propose a three-dimensional multi-object tracking method and system based on a damping window mechanism, which can improve the accuracy, robustness, and efficiency of object tracking.

[0006] To achieve the above purpose, the present invention adopts the following specific technical solutions:

[0007] The three-dimensional multi-object tracking system based on the damping window mechanism provided by the present invention includes:

[0008] A three-dimensional object detection algorithm module: adopting the DeepFusion three-dimensional object detection algorithm to obtain and fuse the point cloud data of the lidar and the color image data of the camera to achieve three-dimensional object detection;

[0009] The three-dimensional extended Kalman filter module includes a state prediction unit and a state update unit: the state prediction unit uses a constant velocity three-dimensional extended Kalman filter to approximate the inter-frame displacement of an object to predict the state of the target, and the state update unit is used to process the target motion prediction under a non-linear system to improve the accuracy and robustness of target tracking;

[0010] The dynamic trajectory-oriented data association algorithm module: According to the target detection results of the three-dimensional target detection algorithm module, combined with the target state predicted by the three-dimensional extended Kalman filter module, the predicted trajectory state is obtained, the matching cost between the target detection result and the predicted trajectory state is calculated, the Hungarian algorithm is used for optimal matching, and according to the preset matching threshold, the matching results with a matching cost lower than the matching threshold are retained for trajectory update and trajectory sorting;

[0011] The trajectory management module based on the damping window mechanism: calculates the trajectory score based on the matching results and the predicted trajectory state, adopts the damping window mechanism, compares the trajectory score with the preset activity threshold, retains the trajectories with a trajectory score not lower than the preset value and marks them as active for continuous tracking, and updates the trajectory state.

[0012] Further, in the three-dimensional extended Kalman filter module, the state of the predicted target is specifically as follows:

[0013]

[0014] Among them, F t is the state transition matrix, which describes that the target state has transferred from time t-1 to time t, ω t-1 is the process noise, and X t is the state predicted at time t based on time t-1;

[0015] Update the state covariance matrix:

[0016]

[0017] Among them, Q is the process noise covariance matrix;

[0018] In the three-dimensional extended Kalman filter module, when the detection is successfully associated with the trajectory, the observation value Z t is used to update the state;

[0019]

[0020]

[0021] P t =(I-K t H t )P t|t-1 ;

[0022] Among them, H t is the observation matrix, R is the observation covariance matrix, and P t is the updated state covariance, and K t is the Kalman gain.

[0023] Furthermore, in the dynamic trajectory-oriented data association algorithm module, the matching cost C ij between the detection result of the target and the predicted trajectory state is as follows:

[0024] C ij = cost(d i , T j );

[0025] Use the Hungarian algorithm to find the optimal match between detections and trajectories:

[0026]

[0027] where x ij is a binary variable indicating whether the detection d i matches the trajectory T j .

[0028] Furthermore, calculate the trajectory score in the trajectory management module based on the damping window mechanism:

[0029]

[0030] where φ i is the association factor, 1 for success and 0 for failure; f(Δt i ) is the time decay function; Δt i is the difference between the current time and the historical time;

[0031] f(Δt i ) The time decay function needs to satisfy:

[0032] f(Δt i ) > 0, and the weight is always positive; The influence of historical data decays with time; The weight of detections closer to the current time is greater; f(0) = c, c > 0, and the weight of detections at the current time is a constant;

[0033] The time decay function is an exponential decay function:

[0034]

[0035] λ is the decay coefficient;

[0036] If s(t) ≥ θ active , the trajectory is marked as active, retained, and continued to be tracked;

[0037] If θ tentative ≤ s(t) < θ active , the trajectory is marked as tentative and the termination is postponed;

[0038] If s(t) < θ tentative , the trajectory terminates;

[0039] Among them, θ active is the active threshold, and θ tentative is the tentative threshold.

[0040] The present invention also provides a three-dimensional multi-target tracking method based on a damping window mechanism, which applies the above-mentioned three-dimensional multi-target tracking system based on a damping window mechanism, and includes the following steps:

[0041] S1. Adopt the DeepFusion three-dimensional object detection algorithm to obtain point cloud data from the lidar, obtain color image data from the camera, synchronize and calibrate the point cloud data and the color image data, extract features from the point cloud data and the color image and fuse them, and perform object detection based on the fused features;

[0042] S2. Adopt a constant velocity three-dimensional extended Kalman filter to approximate the inter-frame displacement of the object to predict the state of the target;

[0043] S3. According to the target detection result obtained in step S1, combine the target state predicted in step S2 to obtain the predicted trajectory state, calculate the matching cost between the target detection result and the predicted trajectory state, use the Hungarian algorithm for optimal matching, and according to the preset matching threshold, retain the matching results with a matching cost lower than the matching threshold for trajectory update and trajectory arrangement;

[0044] S4. Calculate the trajectory score based on the matching result and the predicted trajectory state, adopt the damping window mechanism, compare the trajectory score with the preset active threshold, retain the trajectories with a trajectory score not lower than the preset value and mark them as active for continuous tracking, and update the trajectory state.

[0045] Further, in step S1, before feature fusion, inverse augmentation is used to reverse the data augmentation related to geometry to achieve geometric alignment between the point cloud of the point cloud data and the pixels of the color image, and LearnableAlign is used to dynamically capture the correlation between the features of the camera color image and the lidar point cloud data during fusion to improve the quality of the three-dimensional object detection result.

[0046] Further, in step S2, the specific method for predicting the state of the target is as follows:

[0047]

[0048] Among them, Ft is the state transition matrix, which describes that the target state transfers from the (t - 1)th moment to the tth moment, ω t-1 is the process noise, X t is the state predicted at the tth moment based on the (t - 1)th moment;

[0049] Update the state covariance matrix:

[0050]

[0051] where Q is the process noise covariance matrix;

[0052] In the three - dimensional extended Kalman filter module, when the detection is successfully associated with the trajectory, use the observation value Z t to update the state;

[0053]

[0054] P t =(I - K t H t )P t|t-1 ;

[0055] where H t is the observation matrix, R is the observation covariance matrix, P t is the updated state covariance, and K t is the Kalman gain.

[0056] Furthermore, in step S3, the matching cost C between the detection result of the target and the predicted trajectory state is ij :

[0057] C ij = cost(d i , T j );

[0058] Use the Hungarian algorithm to find the optimal matching between detections and trajectories:

[0059]

[0060] where x ij is a binary variable indicating whether the detection d i matches the trajectory T j .

[0061] Furthermore, in step S4, calculate the trajectory score:

[0062]

[0063] where φ i is the association factor, 1 for success and 0 for failure; f(Δt i) is a time decay function; Δt i is the difference between the current moment and the historical moment;

[0064] f(Δt i ) The time decay function needs to satisfy:

[0065] f(Δt i ) > 0, the weight is always positive; The influence of historical data decays with time; The detection weight closer to the current moment is larger; f(0) = c, c > 0, the detection weight at the current moment is a constant;

[0066] The time decay function is an exponential decay function:

[0067]

[0068] λ is the decay coefficient;

[0069] If s(t) ≥ θ active , the trajectory is marked as active, retained and continued to be tracked;

[0070] If θ tentative ≤ s(t) < θ active , the trajectory is marked as tentative and the termination is postponed;

[0071] If s(t) < θ tentative , the trajectory terminates;

[0072] Among them, θ active is the active threshold, and θ tentative is the tentative threshold.

[0073] The present invention can achieve the following technical effects:

[0074] The three-dimensional multi-target tracking method and system based on the damping window mechanism provided by the present invention generate high-precision three-dimensional target detection results through DeepFusion multi-modal data fusion, predict the target state using three-dimensional extended Kalman filtering, combine the dynamic trajectory-oriented algorithm for detection and trajectory matching, and finally optimize the trajectory life cycle management through the damping window mechanism.

[0075] The present invention improves the performance of multi-target tracking in the following ways:

[0076] 1. High-precision target detection: By fusing lidar and camera data through DeepFusion, high-quality three-dimensional target detection results are generated, significantly improving the detection accuracy and robustness, especially performing excellently in complex scenarios (such as occlusion, illumination changes).

[0077] 2. Accurate State Prediction: The three-dimensional Extended Kalman Filter (EKF) is used to predict the target state. Combining the constant velocity model and the non-linear motion model, it adapts to the changes in the target motion pattern, improving the accuracy and robustness of the prediction.

[0078] 3. Robust Data Association: Through the dynamic trajectory-oriented algorithm, combining multi-hypothesis tracking and dynamic threshold filtering, it reduces false matches and missed matches, enhancing the accuracy and efficiency of data association.

[0079] 4. Optimized Trajectory Management: The damping window mechanism is adopted. Through the global history weighting and time decay mechanism, it dynamically evaluates the active state of the trajectory, reducing the loss of low-confidence targets, suppressing noise interference, and ensuring that the trajectory remains active when the target is occluded or appears intermittently.

[0080] 5. Intelligent Trajectory Update: According to the historical association state of the trajectory and the current detection results, it dynamically adjusts the active state of the trajectory (active, tentative, terminated), achieving continuous tracking of the target trajectory in complex scenarios and reducing the error rate of the tracking trajectory.

[0081] The three-dimensional multi-target tracking method and system based on the damping window mechanism provided by the present invention significantly improve the tracking efficiency, response speed, and robustness of multi-target tracking, reduce the difficulty of vehicle recognition and tracking, and are applicable to fields such as autonomous driving, intelligent monitoring, and robot navigation, with broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 is a schematic flow chart of the three-dimensional multi-target tracking method based on the damping window mechanism provided by an embodiment of the present invention;

[0083] Figure 2 is a schematic diagram of the DeepFusion three-dimensional target detection algorithm provided by an embodiment of the present invention;

[0084] Figure 3 is a flow chart of the dynamic trajectory-oriented algorithm provided by an embodiment of the present invention;

[0085] Figure 4 is a flow chart of the trajectory optimization of the damping window mechanism provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0086] In the following, embodiments of the present invention will be described with reference to the drawings. In the following description, the same modules are denoted by the same reference numerals. In the case of the same reference numerals, their names and functions are also the same. Therefore, their detailed descriptions will not be repeated.

[0087] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation to the present invention.

[0088] An embodiment of the present invention provides a three-dimensional multi-object tracking system based on a damping window mechanism, including:

[0089] A three-dimensional object detection algorithm module: adopting the DeepFusion three-dimensional object detection algorithm to obtain and fuse the point cloud data of the lidar and the color image data of the camera to achieve three-dimensional object detection;

[0090] A three-dimensional extended Kalman filter module, including a state prediction unit and a state update unit: the state prediction unit uses a constant velocity three-dimensional extended Kalman filter to approximate the inter-frame displacement of an object to predict the state of the target, and the state update unit is used to process the target motion prediction under a non-linear system to improve the accuracy and robustness of target tracking;

[0091] A dynamic trajectory-oriented data association algorithm module: according to the object detection results of the three-dimensional object detection algorithm module, combining the target state predicted by the three-dimensional extended Kalman filter module to obtain the predicted trajectory state, calculating the matching cost between the object detection result and the predicted trajectory state, using the Hungarian algorithm for optimal matching, and according to a preset matching threshold, retaining the matching results with a matching cost lower than the matching threshold for trajectory update and trajectory sorting;

[0092] A trajectory management module based on a damping window mechanism: calculating a trajectory score according to the matching result and the predicted trajectory state, adopting a damping window mechanism, comparing the trajectory score with a preset activity threshold, retaining the trajectories with a trajectory score not lower than the preset value and marking them as active for continuous tracking, and updating the trajectory state.

[0093] The present invention also provides a three-dimensional multi-object tracking method based on a damping window mechanism, and its process is as Figure 1 shown. Applying the above three-dimensional multi-object tracking system, it includes the following steps:

[0094] S1. Adopt the DeepFusion three-dimensional object detection algorithm to obtain point cloud data from the lidar, obtain color image data from the camera, synchronize and calibrate the point cloud data and the color image data, extract features from the point cloud data and the color image and fuse them, and perform object detection based on the fused features. The DeepFusion three-dimensional object detection algorithm is as Figure 2 shown, and the specific process is as follows:

[0095] S11. First, perform data preprocessing. Obtain point cloud data from the lidar and color image data from the camera. Synchronize and calibrate the data to ensure that the data captured by the lidar and the camera is consistent in space and time.

[0096] S12. Extract features from the point cloud data and the color image. Before feature fusion, use InverseAug to invert and geometrically related data augmentation to achieve precise geometric alignment between the lidar point cloud and image pixels.

[0097] At the same time, LearnableAlign dynamically captures the correlation between the image and lidar features during fusion, thereby generating high-quality 3D object detection results.

[0098] S13. Use the fused features to generate potential target candidate regions, and classify and locate the candidate regions. Finally, the detection head based on the fused features provides classification and regression outputs for 3D object detection.

[0099] The DeepFusion 3D object detection algorithm outputs the detection result D = {D1, D2,..., D n}, where each detection box contains the position, velocity, category, and confidence score of the target.

[0100] S2. Use a constant velocity 3D extended Kalman filter to approximate the inter-frame displacement of the object to predict the state of the target.

[0101] In the 3D extended Kalman filter, this system uses a constant velocity model for state prediction to predict the state of the target at the next moment. The state prediction formula is:

[0102]

[0103] where, F t is the state transition matrix, which describes the transfer of the target state from time t - 1 to time t, ω t-1 is the process noise, and X t is the predicted state at time t based on time t - 1.

[0104] Update the state covariance matrix:

[0105]

[0106] where Q is the process noise covariance matrix.

[0107] During the 3D extended Kalman filter process, the state update corrects the predicted state according to the observed data. When the detection is successfully associated with the trajectory, use the observed value Z t to update the state;

[0108]

[0109]

[0110] P t = (I - K t H t )P t|t-1 ; (5)

[0111] where H t is the observation matrix, R is the observation covariance matrix, P t is the updated state covariance, and K t is the Kalman gain.

[0112] The three-dimensional extended Kalman filter (EKF) can effectively handle the nonlinear problems in multi-target tracking by locally linearizing the nonlinear system. The core steps of EKF include prediction and update, and by fusing the motion model of the target and the observation data, an accurate estimation of the target state is achieved.

[0113] Regarding the problems in complex scenarios where the target is occluded by other objects, resulting in incomplete or lost detection, and multiple targets moving crosswise in space, leading to confusion, to solve the above problems, this method adopts a dynamic trajectory-oriented data association algorithm compared with the traditional Hungarian algorithm.

[0114] S3. According to the target detection results obtained in step S1, combined with the target state predicted in step S2 to obtain the predicted trajectory state, calculate the matching cost between the target detection results and the predicted trajectory state, use the Hungarian algorithm for optimal matching, and according to the preset matching threshold, retain the matching results with a matching cost lower than the matching threshold for trajectory update and trajectory arrangement. The process of the dynamic trajectory-oriented algorithm is as Figure 3 shown, and the specific process is as follows:

[0115] The core idea of the dynamic trajectory-oriented algorithm is multiple hypothesis tracking (MHT), that is, in each time step, multiple possible association hypotheses are maintained instead of making a final decision immediately. By delaying the decision, the algorithm can select the most appropriate match after obtaining more observation information. The dynamic trajectory-oriented algorithm improves MHT and generally includes the following steps:

[0116] S31. Obtain the target detection box D t of the current frame from S1, accurately estimate the position, size, and orientation of the target, and then obtain the predicted trajectory state by predicting its position in the next frame combined with the position of the tracking target in the current frame from S2 This state is used as input to the data association module for matching.

[0117] S32. Calculate the matching cost between the detection and the trajectory based on the detection result of the current frame and the predicted trajectory state. The cost matrix is where M det is the number of detections, and N tra is the number of trajectories.

[0118] After obtaining the cost matrix, use the Hungarian algorithm for optimal matching to minimize the total matching cost and obtain a set of matching pairs.

[0119] According to the preset matching threshold, filter out the unsatisfied matches. Only the matches with a matching cost lower than the threshold will be retained, further reducing the false matches. The obtained matching results are used for trajectory update and trajectory arrangement.

[0120] Construct the cost matrix and calculate the matching cost C between the detection and the trajectory ij :

[0121] C ij = cost(d i , T j ); (6)

[0122] Then use the Hungarian algorithm to find the optimal matching between the detection and the trajectory:

[0123]

[0124] where x ij is a binary variable indicating whether the detection d i matches the trajectory T j .

[0125] After the above is completed, perform threshold filtering. For each matching pair (i, j), if its matching cost x ij is greater than the threshold τ, then this matching pair will be filtered out. Finally, output the indices sum of the matched detections and trajectories, and the indices sum of the unmatched detections and trajectories.

[0126] To manage the trajectory life cycle more robustly and solve the problem that it is difficult to balance false positives and false negatives in traditional counting strategies, a damped window mechanism is adopted.

[0127] S4. Calculate the trajectory score based on the matching result and the predicted trajectory state. Adopt the damped window mechanism, compare the trajectory score with the preset activity threshold, retain the trajectories with a trajectory score not lower than the preset value and mark them as active for continuous tracking, and update the trajectory state. The trajectory optimization process of the damped window mechanism is as Figure 4 shown.

[0128] In the trajectory management module, a damping window mechanism is adopted to robustly manage the trajectory life cycle, reduce FN and avoid misjudgment of FP. The damping window algorithm dynamically weights the trajectory historical association status, combines time decay and global information. Its core lies in balancing the influence of recent and historical data, reducing the omission of low-confidence targets, and suppressing noise interference.

[0129] The damping window algorithm follows the following principles:

[0130] Detections closer to the current time have a greater impact on the trajectory score. Considering the association status (success / failure) of all detection points of the trajectory comprehensively, rather than relying only on recent results. By smoothing the trajectory score fluctuations, premature termination of the trajectory caused by short-term missed detections or misdetections is avoided.

[0131] Calculate the trajectory score through the damping window mechanism based on the matching result of the current frame and the historical data of the trajectory:

[0132]

[0133] where φ i is the association factor, 1 for success and 0 for failure; f(Δt i ) is the time decay function, and Δt i is the difference between the current moment and the historical moment;

[0134] where the time decay function f(Δt i ) needs to satisfy:

[0135] f(Δt i ) > 0, and the weight is always positive; The influence of historical data decays with time; The weight of detections closer to the current moment is greater; f(0) = c, c > 0, and the weight of the detection at the current moment is a constant;

[0136] Therefore, the time decay function of this method is an exponential decay function:

[0137]

[0138] where λ is the decay coefficient.

[0139] After obtaining the score, compare it with a preset threshold

[0140] If s(t) ≥ θ active , the trajectory is marked as active, retained, and continued to be tracked;

[0141] If θ tentative ≤ s(t) < θ active , the trajectory is marked as tentative and the termination is postponed;

[0142] If s(t) < θtentative , the trajectory terminates.

[0143] Among them, θ active is the active threshold, and θ tentative is the tentative threshold.

[0144] For unmatched detections, initialize a new trajectory and assign a unique trajectory ID to the new detection. For trajectories with scores below the threshold, the system terminates these trajectories.

[0145] Add the new detection to the trajectory history data and initialize the trajectory score as the confidence score of the current detection.

[0146] Update the trajectory status, add the detection or prediction result of the current frame to the trajectory history data, and recalculate the trajectory score s(t) according to the new association status and the time decay function f(Δt i ). Output the updated trajectory status according to the trajectory score and the threshold.

[0147] The trajectory management module dynamically evaluates the active status of the trajectory through a damping window mechanism to ensure that the trajectory remains active when the target is occluded or appears intermittently. This module plays a crucial role in the multi-object tracking system, improving the robustness and accuracy of tracking.

[0148] The system and method provided by the present invention are verified on multiple data sets and can stably track vehicles. Even in the case of light changes and frequent vehicle occlusions, it can still accurately track. The experimental results show that compared with traditional 3D multi-object tracking methods, the present invention improves the accuracy of target association and significantly improves the tracking accuracy in dense target areas.

[0149] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0150] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

[0151] The specific embodiments of the present invention described above do not limit the protection scope of the present invention. Any other corresponding changes and deformations made according to the technical concept of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A three-dimensional multi-object tracking system based on a damping window mechanism, characterized in that Including: 3D object detection algorithm module: Adopting the DeepFusion 3D object detection algorithm to obtain and fuse the point cloud data of the lidar and the color image data of the camera to achieve 3D object detection; 3D extended Kalman filter module, including a state prediction unit and a state update unit: The state prediction unit uses a constant velocity 3D extended Kalman filter to approximate the inter-frame displacement of the object to predict the state of the target, and the state update unit is used to process the target motion prediction under a non-linear system to improve the accuracy and robustness of target tracking; Dynamic trajectory-oriented data association algorithm module: According to the target detection results of the 3D object detection algorithm module, combining the target state predicted by the 3D extended Kalman filter module to obtain the predicted trajectory state, calculating the matching cost between the detection result of the target and the predicted trajectory state, using the Hungarian algorithm for optimal matching, and according to the preset matching threshold, retaining the matching results with a matching cost lower than the matching threshold for trajectory update and trajectory arrangement; Trajectory management module based on the damping window mechanism: Calculating the trajectory score based on the matching result and the predicted trajectory state, adopting the damping window mechanism, comparing the trajectory score with the preset activity threshold, retaining the trajectories with a trajectory score not lower than the preset value and marking them as active for continuous tracking, and updating the trajectory state.

2. The three-dimensional multi-target tracking system based on the damping window mechanism according to claim 1 is characterized in that: In the 3D extended Kalman filter module, the state of the predicted target is specifically as follows: Among them, F t is the state transition matrix, which describes that the target state has transferred from the (t - 1)th moment to the tth moment, and ω t-1 is the process noise, and X t is the state predicted at the tth moment based on the (t - 1)th moment; Updating the state covariance matrix: where Q is the process noise covariance matrix; In the three-dimensional extended Kalman filter module, when the detection is successfully associated with the trajectory, the observation value Z is used t to update the state; P t = (I - K t H t )P t|t-1 ; Among them, H t is the observation matrix, R is the observation covariance matrix, and P t is the updated state covariance, and K t is the Kalman gain.

3. The three-dimensional multi-object tracking system based on the damping window mechanism according to claim 1, wherein In the dynamic trajectory-oriented data association algorithm module, the matching cost C between the target detection result and the predicted trajectory state is ij : C ij = cost(d i , T j ); Using the Hungarian algorithm to find the optimal match between detections and trajectories: Among them, x ij Is a binary variable, indicating the detection d i Is it consistent with trajectory T j match.

4. The three-dimensional multi-object tracking system based on the damping window mechanism according to claim 1, characterized in that Calculating the trajectory score in the trajectory management module based on the damping window mechanism: Among them, φ i is the correlation factor, 1 for success and 0 for failure; f(Δt i ) is the time decay function; Δt i is the difference between the current moment and the historical moment; f(Δt i )The time decay function needs to satisfy: f(Δt i )>0, the weight is always positive; The influence of historical data decays over time; The closer the detection is to the current moment, the greater the detection weight is; f(0)=c, c>0, the detection weight at the current moment is a constant; The time decay function is an exponential decay function: λ is the decay coefficient; If s(t) ≥ θ active , the trajectory is marked as active, retained, and continued to be tracked; If θ tentative ≤ s(t) < θ active , the trajectory is marked as tentative and the termination is postponed; If s(t) < θ tentative , the trajectory terminates; where θ active is the active threshold, and θ tentative is the tentative threshold.

5. A three-dimensional multi-target tracking method based on a damping window mechanism, applying the three-dimensional multi-target tracking system based on a damping window mechanism according to any one of claims 1 to 4, characterized in that: Including the following steps: S1. Adopting the DeepFusion 3D object detection algorithm, obtaining the point cloud data from the lidar, obtaining the color image data from the camera, synchronizing and calibrating the point cloud data and the color image data, extracting features from the point cloud data and the color image and fusing them, and performing object detection based on the fused features; S2. Using a constant velocity 3D extended Kalman filter to approximate the inter-frame displacement of the object to predict the state of the target; S3. According to the target detection results obtained in step S1, combining the target state predicted in step S2 to obtain the predicted trajectory state, calculating the matching cost between the detection result of the target and the predicted trajectory state, using the Hungarian algorithm for optimal matching, and according to the preset matching threshold, retaining the matching results with a matching cost lower than the matching threshold for trajectory update and trajectory arrangement; S4. Calculating the trajectory score based on the matching result and the predicted trajectory state, adopting the damping window mechanism, comparing the trajectory score with the preset activity threshold, retaining the trajectories with a trajectory score not lower than the preset value and marking them as active for continuous tracking, and updating the trajectory state.

6. The 3D multi-object tracking method based on the damping window mechanism according to claim 5, wherein In step S1, InverseAug is used to invert the geometry-related data enhancement before feature fusion to achieve geometric alignment between the point cloud data point cloud and the color image pixels. LearnableAlign is used to dynamically capture the correlation between the camera color image and the lidar point cloud data features using cross-attention during fusion to improve the quality of the 3D target detection results.

7. The three-dimensional multi-target tracking method based on the damping window mechanism according to claim 5, characterized in that: In step S2, the state of the predicted target is as follows: Among them, F t is the state transfer matrix, describing the target state transferred from time t-1 to time t, ω t-1 is the process noise, X t It predicts the state at time t based on time t-1; Update the state covariance matrix: Where Q is the process noise covariance matrix; In the three-dimensional extended Kalman filter module, when the detection is successfully associated with the trajectory, the observation value Z is used t to update the state; P t = (I - K t H t )P t|t-1 ; Among them, H t is the observation matrix, R is the observation covariance matrix, and P t is the updated state covariance, and K t is the Kalman gain.

8. The three-dimensional multi-target tracking method based on the damping window mechanism according to claim 5, characterized in that: In step S3, the matching cost C between the detection result of the target and the predicted trajectory state ij : C ij =cost(d i ,T j ); Use the Hungarian algorithm to find the best match between detections and trajectories: Among them, x ij Is a binary variable, indicating the detection d i Is it consistent with trajectory T j match.

9. The three-dimensional multi-object tracking method based on the damping window mechanism according to claim 5, characterized in that In step S4, the trajectory score is calculated: Among them, φ i is the correlation factor, success is 1, failure is 0; f(Δt i ) is the time decay function; Δt i is the difference between the current moment and the historical moment; f(Δt i )The time decay function needs to satisfy: f(Δt i )>0, the weight is always positive; The influence of historical data decays over time; The closer the detection is to the current moment, the greater the detection weight is; f(0)=c, c>0, the detection weight at the current moment is a constant; The time decay function is an exponential decay function: λ is the attenuation coefficient; If s(t) ≥ θ active , the trajectory is marked as active, retained, and continued to be tracked; If θ tentative ≤s(t)<θ active ,The trajectory is marked as tentative, and the termination is suspended; If s(t) < θ tentative , the trajectory terminates; where θ active is the active threshold, and θ tentative is the tentative threshold.

Images