Target tracking method, system and related device adapting to scenarios and cascading data
By extending Kalman filtering and employing a three-stage cascaded strategy, the noise covariance matrix is dynamically adjusted, and target tracking is performed using multi-sensor data. This solves the problems of prediction lag and insufficient robustness of traditional methods in complex scenarios, achieving high-precision and stable target tracking.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 深圳市欧冶半导体有限公司
- Filing Date
- 2026-03-20
- Publication Date
- 2026-06-26
Smart Images
Figure CN121880963B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of autonomous driving environmental perception technology, and in particular to target tracking methods, systems and related equipment that adapt to scenarios and cascade data. Background Technology
[0002] With the rapid development of autonomous driving technology, the requirements for the accuracy and real-time performance of vehicle perception of the surrounding dynamic environment are increasing. As a core module of environmental perception, multi-target tracking is tasked with continuously estimating the state and maintaining the identity of dynamic targets such as vehicles and pedestrians in a continuous time series, providing stable and reliable trajectory information for downstream decision-making and planning modules.
[0003] Currently, mainstream multi-target tracking solutions are typically based on filtering algorithms (such as Kalman filtering) and data association strategies. However, in complex driving scenarios, existing technologies face the following major challenges: 1. Traditional methods often employ fixed-parameter Kalman filter models. When the vehicle makes sharp turns, emergency braking, or other violent maneuvers, the fixed parameters cannot respond to sudden changes in motion, leading to lag or even divergence in the predicted trajectory. 2. The data association strategy is simplistic and lacks environmental robustness. When the vehicle travels over bumpy roads causing changes in the sensor pitch angle, even slight deviations in the vertical direction of the target can lead to 3D matching failure and missed detections. 3. Sensor data fusion is shallow, failing to deeply integrate the physical characteristics of different sensors. For example, it fails to fully utilize the high precision of millimeter-wave radar in radial velocity measurement to directly correct the target speed, and it also fails to effectively combine visual information to resist occlusion or utilize the vehicle's high-frequency pose to compensate for its own motion. This results in decreased perception performance in complex scenarios such as backlight, rain, fog, and high dynamics, creating perception blind spots.
[0004] Therefore, there is an urgent need for a 3D multi-target tracking method that can adapt to dynamic scenes, make comprehensive use of the characteristics of multiple sensors, and has strong robustness at the data association level. Summary of the Invention
[0005] This application provides a target tracking method, system, and related equipment that adapt to specific scenarios and cascade data, relating to the field of autonomous driving environmental perception technology. The specific technical solution is as follows:
[0006] Firstly, a target tracking method that adapts to a scenario and cascades data is provided. The method includes: collecting multi-source data, including at least LiDAR point clouds, 3D target detection results, millimeter-wave radar data, vehicle status data, and simultaneous localization and mapping (SMR) pose; predicting the motion state of the detected target based on the multi-source data using extended Kalman filtering (EPK) to generate a predicted trajectory, wherein the process noise covariance matrix in the EPK is dynamically adjusted according to the vehicle's motion state, and the observation noise covariance matrix in the EPK is dynamically adjusted according to the target occlusion degree and detection confidence; associating the detected target with the predicted trajectory based on a three-level cascade strategy, the three-level cascade strategy being: a first-level matching based on 3D intersection-union ratio (IU), a second-level matching based on bird's-eye view and generalized distance IU, and a third-level matching based on 2D IU; updating the predicted trajectory based on the data association results and performing full lifecycle management, outputting tracking results that include at least a stable identity identifier.
[0007] In conjunction with the first aspect, the vehicle's motion state includes real-time speed and angular velocity. The process noise covariance matrix in the extended Kalman filter is dynamically adjusted according to the vehicle's motion state. Specifically, this includes: detecting the vehicle's real-time speed and angular velocity; when the real-time speed or angular velocity exceeds a preset threshold, calculating the gain coefficient based on the real-time speed and angular velocity; and amplifying the preset basic process noise matrix based on the gain coefficient to obtain the process noise covariance matrix.
[0008] In conjunction with the first aspect, the observation noise covariance matrix in the extended Kalman filter is dynamically adjusted according to the target occlusion degree and detection confidence. Specifically, this includes: detecting the target occlusion degree and detection confidence; when the target occlusion degree indicates that the target is partially or completely occluded, or when the detection confidence is lower than a preset threshold, calculating a penalty coefficient based on the target occlusion degree and detection confidence; and amplifying the preset basic observation noise matrix based on the penalty coefficient to obtain the observation noise covariance matrix.
[0009] In conjunction with the first aspect, a three-level cascaded strategy is used to associate data between the detected target and the predicted trajectory. Specifically, this includes: in the first-level matching, calculating the 3D intersection-union ratio (IU / U) of the detected target and the predicted trajectory, and obtaining a first matching result where the matching degree between the detected target and the predicted trajectory is higher than a first threshold; in the second-level matching, projecting the detected target and the predicted trajectory with a matching degree not exceeding the first threshold onto a plane, ignoring their height information, calculating the generalized distance IU / U, and obtaining a second matching result where the matching degree is higher than a second threshold, and the second threshold is lower than the first threshold; in the third-level matching, projecting the detected target and the predicted trajectory with a matching degree not exceeding the second threshold onto a 2D image plane, ignoring their depth information, calculating the 2D IU / U, and obtaining a third matching result where the matching degree is higher than a third threshold, and the third threshold is lower than the second threshold.
[0010] In conjunction with the first aspect, during the process of updating the predicted trajectory and performing full lifecycle management based on the data association results, the method includes: receiving a detection target with a matching degree not exceeding a third threshold, matching the detection target with an initialized new trajectory, and assigning an initial confidence level to the new trajectory; if the new trajectory is successfully associated with the detection target within N0 consecutive frames, then increasing the confidence level of the new trajectory, where N0 is a positive integer greater than or equal to 2; when the confidence level of the new trajectory is higher than a preset stability threshold, it is confirmed as a predicted trajectory.
[0011] In conjunction with the first aspect, a confidence level is set in the predicted trajectory. During the process of updating the predicted trajectory and performing full lifecycle management based on the data association results, the method includes: if the predicted trajectory is not matched with the detected target within N1 consecutive frames, the confidence level of the predicted trajectory is reduced, where N1 is a positive integer greater than or equal to 2; when the confidence level of the predicted trajectory is lower than a preset disappearance threshold, it is determined that the detected target has disappeared and the predicted trajectory is deleted.
[0012] In conjunction with the first aspect, in the process of predicting the motion state of a detected target based on extended Kalman filtering of multi-source data, the method includes: using the radial velocity of the target in the millimeter-wave radar data as the observation value input into the state update equation to correct the motion state estimate of the detected target.
[0013] It should be noted that, in the absence of conflict, the features in the various embodiments of the first aspect can be combined with each other, and any combination of features in different embodiments is also within the protection scope of this application. That is to say, the various embodiments described above can also be arbitrarily combined according to actual needs.
[0014] Secondly, a target tracking system that adapts to a scenario and cascades data is provided, for implementing the method of the first aspect or any of the embodiments described above, including:
[0015] The data acquisition module is used to collect multi-source data, which includes at least lidar point clouds, 3D target detection results, millimeter-wave radar data, vehicle status data, and pose for simultaneous localization and map building.
[0016] The state prediction module is used to predict the motion state of the detected target based on multi-source data using extended Kalman filtering and generate a predicted trajectory. The process noise covariance matrix in the extended Kalman filter is dynamically adjusted according to the vehicle's motion state, and the observation noise covariance matrix in the extended Kalman filter is dynamically adjusted according to the target occlusion degree and detection confidence.
[0017] The data association module is used to associate the detected target with the predicted trajectory based on a three-level cascade strategy. The three-level cascade strategy is as follows: first-level matching based on the three-dimensional intersection-union ratio, second-level matching based on the bird's-eye view and the generalized distance intersection-union ratio, and third-level matching based on the two-dimensional intersection-union ratio.
[0018] The trajectory management module is used to update the predicted trajectory and perform full lifecycle management based on the data association results, and output tracking results that include at least a stable identity identifier.
[0019] Thirdly, a computer device is provided, including one or more memories and one or more processors; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the computer device to implement the method as described in the first aspect or any of the embodiments of the first aspect.
[0020] Fourthly, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the method as described in the first aspect or any of the embodiments in the first aspect.
[0021] Fifthly, a chip is provided for use in a computer, the chip including one or more processors for invoking computer instructions to cause the computer to perform a method as described in the first aspect or any of the embodiments of the first aspect.
[0022] In the embodiments of this application, the method provided by this application has the following beneficial effects:
[0023] 1. Improved tracking accuracy and stability. This application dynamically adjusts the noise covariance matrix of the extended Kalman filter, enabling the state prediction model to adapt to scenarios of violent vehicle movement and target occlusion, effectively reducing prediction lag and trajectory divergence, and significantly reducing the number of target identity jumps.
[0024] 2. Enhanced robustness in complex scenarios. This application employs a three-level cascaded matching strategy to decouple height and depth errors at each level, enabling the system to maintain a high target association success rate and improve system recall even in scenarios where traditional methods are prone to failure, such as bumpy roads and long distances.
[0025] 3. Achieving high-precision state estimation. This application significantly improves the estimation accuracy of the target's absolute velocity by deeply coupling the radial velocity of the millimeter-wave radar into the filtering update process and using pose to decouple the vehicle's motion, thus providing a more reliable input for downstream planning and control.
[0026] 4. Ensuring system real-time performance. This application uses a cascaded filtering mechanism to complete most simple matching in the first level, avoiding unnecessary complex calculations and ensuring that the overall algorithm meets the stringent real-time requirements of autonomous driving systems. Attached Figure Description
[0027] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0028] Figure 1 This is a schematic diagram of a target tracking system that adapts to a specific scenario and cascades data, provided in an embodiment of this application.
[0029] Figure 2 This is a schematic diagram of the overall process of a target tracking method that adapts to a scenario and cascades data, provided in an embodiment of this application.
[0030] Figure 3 This is a schematic flowchart of an adaptive extended Kalman filter method provided in an embodiment of this application;
[0031] Figure 4 This is a schematic flowchart of a three-level cascaded data association strategy provided in an embodiment of this application;
[0032] Figure 5 This is a schematic diagram of a method for trajectory lifecycle management provided in an embodiment of this application;
[0033] Figure 6 This is a schematic diagram of the hardware structure of a computer device provided in an embodiment of this application;
[0034] Figure 7 This is a schematic diagram of a computer-readable storage medium provided in an embodiment of this application. Detailed Implementation
[0035] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0036] It should be understood that "multiple" as mentioned in this application refers to two or more. In the description of this application, unless otherwise stated, " / " indicates "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist, for example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, to facilitate a clear description of the technical solutions of this application, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and that "first," "second," etc., do not necessarily imply differences.
[0037] The terms "one embodiment" or "some embodiments" used in this application mean that one or more embodiments of this application include the specific features, structures, or characteristics described in that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this application do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. Furthermore, the terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.
[0038] The following describes the target tracking method for adapting to scenarios and cascading data provided in this application through three embodiments. Embodiment 1 describes the target tracking system for adapting to scenarios and cascading data, Embodiment 2 describes the method flow for target tracking for adapting to scenarios and cascading data, and Embodiment 3 describes the computer device hardware structure and computer-readable storage medium for executing the target tracking method for adapting to scenarios and cascading data.
[0039] Example 1
[0040] Figure 1 This is a schematic diagram of a target tracking system that adapts to a specific scenario and cascades data, as provided in an embodiment of this application. Figure 1 As shown, the target tracking system 100 that adapts to the scene and cascades data includes a data acquisition module 110, a state prediction module 120, a data association module 130, and a trajectory management module 140.
[0041] In this embodiment, the data acquisition module 110 includes a multi-source data receiving unit 111 and a data preprocessing unit 112.
[0042] In this embodiment, the multi-source data receiving unit 111 can receive the following multi-source data in parallel through a hardware interface: current frame image data from the camera, three-dimensional (3D) point cloud data from the LiDAR, and a list of detected targets output from the upstream receiving network (including category, 3D bounding box information, and detection confidence score). Radial distance and Doppler radial velocity of the detected target acquired from millimeter-wave radar; and vehicle status data (including real-time speed of the vehicle) from the Controller Area Network (CAN) bus. and angular velocity Simultaneous Localization and Mapping (SLAM) pose acquired by Inertial Measurement Unit (IMU) and LiDAR, i.e., the transformation matrix in the vehicle coordinate system. .
[0043] In this embodiment, the data preprocessing unit 112 performs time synchronization processing and spatial coordinate transformation processing on all multi-source data to form a standardized multimodal perception data stream, laying the foundation for subsequent processing. For example, the time synchronization processing includes: using the lidar acquisition time as a reference, employing nearest neighbor interpolation to time-align 3D point cloud data, CAN bus data, and current frame image data, ensuring that the timestamp error of all data is within the allowable range; the spatial coordinate transformation processing includes: using a pre-calibrated parameter matrix to uniformly transform the local coordinate data acquired by the lidar, millimeter-wave radar, and camera to the vehicle coordinate system, and combining this with SLAM pose to transform the coordinates and velocity vectors of all detected targets to a unified navigation coordinate system, thereby eliminating the influence of vehicle motion on the velocity estimation of detected targets.
[0044] In this embodiment of the application, the data preprocessing unit 112 sends the preprocessed multi-source data to the state prediction module 120 and the data association module 130 respectively.
[0045] In this embodiment, the state prediction module 120 includes an adaptive extended Kalman filter unit 121. The adaptive extended Kalman filter (EKF) in the adaptive extended Kalman filter unit 121 takes historical trajectory states and multi-source data as input to predict the motion state of the detected target and generate a predicted trajectory. The EKF provided in this application includes the following two dynamic adjustment mechanisms.
[0046] 1. Dynamic adjustment mechanism for process noise: Receives real-time vehicle speed data from the CAN bus. and angular velocity When the vehicle is detected to be at high speed (i.e., Real-time speed is greater than the speed threshold ) or sharp turn (i.e. Angular velocity greater than angular velocity threshold In the state of ), the gain coefficient is calculated. The noise covariance matrix during the process of updating the current frame .in, , , This is the base process noise matrix. This mechanism can dynamically increase the process noise covariance matrix. This allows the filter weights to be allocated more to the current observations, thus enabling a faster response to sudden changes in state and reducing prediction lag.
[0047] 2. Observation noise dynamic adjustment mechanism: Obtain the occlusion status of the detected target in the current frame. and the detection confidence score in the upstream detection network When the target being detected is obstructed or the detection quality is low (i.e., The detection confidence score is less than the threshold score. When ), the penalty coefficient is calculated. Update the observation noise covariance matrix of the current frame. .in, , , This is based on the observation noise matrix. This mechanism can dynamically increase the observation noise covariance matrix. This reduces the filter's confidence in the current noise observation (i.e., reduces the Kalman gain), forcing the system to rely primarily on the predicted state from the previous moment for inertial extrapolation, thereby reducing interference from detection box deformation caused by occlusion.
[0048] In this embodiment of the application, the state prediction module 120 outputs all predicted trajectories in the current frame and passes them to the data association module 130.
[0049] In this embodiment, the data association module 130 includes a matching update unit 131 and a three-level cascading strategy unit 132.
[0050] In this embodiment, the three-level cascaded strategy unit 132 obtains the detection target of the current frame image data from the data preprocessing unit 112 and the predicted trajectory obtained from the adaptive extended Kalman filter unit 121, and performs the following multi-level matching in sequence by adopting a spatial dimension reduction strategy.
[0051] First-level matching: Utilizing complete 3D geometric information, it prioritizes the detection of targets that are close to the target and have a high degree of shape similarity. Specifically, it calculates the target... With predicted trajectory The 3D Intersection over Union (3D IoU) ratio between them. Construct the cost matrix The cost matrix was analyzed using the Hungarian matching algorithm. Solve the problem to obtain the optimal set of matching pairs; retain The first matching result with a value greater than a first threshold (e.g., 0.7) is used to detect unmatched targets. With predicted trajectory Proceed to the second level of matching calculation.
[0052] Second-level matching: Unmatched detected targets are projected onto the Bird's Eye View (BEV) plane, ignoring height (Z-axis) information, to resolve the issue of vertical (Z-axis) non-overlap caused by factors such as vehicle bumps, changes in road slope, or sensor calibration errors. Specifically, for unmatched detected targets in the first-level matching... With predicted trajectory Projecting onto the BEV plane simplifies the 3D cube into a two-dimensional (2D) rotated rectangle; the generalized distance intersection-union ratio (GD IoU) under the BEV plane is calculated, i.e. This GD IoU, by introducing the minimum circumscribed convex hull, can solve the problem of zero gradient when two boxes do not intersect, and has a good matching effect for targets with slight positional deviations; a cost matrix is constructed. Run the Hungarian matching algorithm to obtain the optimal set; retain For second matching results that are greater than the second threshold (e.g., 0.5), for unmatched detection targets... With predicted trajectory Proceed to the third level of matching calculation.
[0053] Third-level matching: Unmatched detected targets are projected onto the 2D image, ignoring depth information. This is used to eliminate the problem of high noise in depth estimation for distant targets due to the sparse point cloud of the LiDAR. Specifically, it utilizes the camera's intrinsic parameter matrix. and extrinsic parameter matrix Unmatched predicted trajectories The 8 vertices in 3D Through formula Projecting onto the pixel plane of a 2D image generates a 2D bounding box for the predicted trajectory. Calculate the 2D bounding box of the detected target in the front view. and The two-dimensional intersection-union ratio (2D IoU), i.e. Construct the cost matrix Run the Hungarian matching algorithm to obtain the optimal set and retain it. The third match result is greater than the third threshold (e.g., 0.3).
[0054] It should be noted that since the first-level matching has already filtered and removed most of the easily matched targets, the number of unmatched items entering the second-level matching is significantly reduced. Similarly, the second-level matching further filters and resolves some matching difficulties caused by height errors, making the number of targets that ultimately flow into the third-level matching process very limited. Therefore, although the second and third-level matching involve more complex metric calculations or coordinate projection transformations, the additional computational overhead is kept to a very low level due to the limited number of target pairs to be processed.
[0055] In this embodiment, the three-level cascaded strategy unit 132 performs multi-level matching to ultimately output matched pairs (including a first matching result, a second matching result, and a third matching result), unmatched detected targets, and unmatched predicted trajectories. The matched pairs are input to the matching update unit 131, which uses observations acquired by the data acquisition module 110 to update the predicted trajectory, the motion state of the detected targets, and the Kalman filter state, and outputs the optimal predicted trajectory state to the output trajectory management module 140. The unmatched detected targets and unmatched predicted trajectories are input to the trajectory management module 140 for predicted trajectory management.
[0056] In this embodiment, the trajectory management module 140 includes a trajectory management unit 141 and a result output unit 142. The result output unit 142 outputs tracking results for successfully matched predicted trajectories and feeds back the processing results of the current frame image to the next frame image. The tracking results include: the identity document (ID) of the detected target, its three-dimensional position, size, orientation, and velocity information.
[0057] In this embodiment of the application, the trajectory management unit 141 obtains unmatched predicted trajectories, performs trajectory lifecycle management based on confidence level to obtain the final predicted trajectory, and inputs the result output unit 142.
[0058] For example, the trajectory management unit 141 can initialize a new trajectory and assign it an initial confidence level. If the new trajectory can be stably matched within N0 consecutive frames, its corresponding confidence level is increased, and when the confidence level of the new trajectory is higher than a preset stability threshold, it is confirmed as a predicted trajectory. Here, N0 is a positive integer greater than or equal to 2, such as 3. In this way, the trajectory management unit 141 can maintain tracking of targets that are temporarily occluded, ensuring the continuity and reliability of the trajectory.
[0059] For example, the trajectory management unit 141 can reduce the confidence level of predicted trajectories that do not match the detected target in the current frame. For instance, if a predicted trajectory does not match the detected target within N1 consecutive frames, the confidence level of the predicted trajectory is reduced, where N1 is a positive integer greater than or equal to 2, such as 3. When the confidence level of the predicted trajectory is lower than a preset disappearance threshold, it is determined that the detected target has disappeared and the predicted trajectory is deleted. In this way, the trajectory management unit 141 can effectively filter out isolated false detections caused by sensor noise or environmental interference.
[0060] It is understood that the functional division between the modules / units illustrated in the target tracking system 100 that adapts to scenarios and cascades data is merely illustrative and does not constitute a functional limitation on the target tracking system 100 that adapts to scenarios and cascades data. In other embodiments of this application, the target tracking system 100 that adapts to scenarios and cascades data may also employ different modules / units than those in the above embodiments, or a combination of multiple modules / units, to implement the functions of the target tracking system 100 that adapts to scenarios and cascades data.
[0061] Example 2
[0062] Figure 2 This is a schematic diagram of the overall process of a target tracking method that adapts to a scenario and cascades data, provided in an embodiment of this application; applicable to, for example... Figure 1 The target tracking system 100 shown adapts to the scene and cascades data. For example... Figure 2 As shown, the specific steps include:
[0063] S101. Multi-sensor data acquisition and preprocessing.
[0064] In the embodiments of this application, corresponding Figure 1 The data acquisition module 110 is designed to receive data from multiple sensors in parallel, including at least: lidar point clouds, camera images, 3D target detection results (including bounding boxes and confidence scores) generated by a perception network, millimeter-wave radar data (including distance and radial velocity), vehicle status data (such as real-time speed and angular velocity), and SLAM pose.
[0065] In this embodiment, the time synchronization of multi-sensor data is achieved through timestamp interpolation, using the LiDAR scanning time as a reference. A pre-calibrated extrinsic parameter matrix is used to uniformly transform all sensor data into the vehicle coordinate system. Simultaneously, combined with SLAM pose information, the target state is transformed into the navigation coordinate system for precise motion decoupling.
[0066] S102. Motion state prediction based on adaptive extended Kalman filter.
[0067] In the embodiments of this application, corresponding to Figure 1The function of the state prediction module 120 is to read the optimal state estimate of the previous moment as input for each existing trajectory. The adaptive extended Kalman filter provided in this application predicts the motion state (such as position and velocity) of the trajectory in the current frame based on its historical state, and generates the predicted trajectory for the current frame.
[0068] In this embodiment, the key to adaptive EKF lies in implementing two adaptive adjustment mechanisms to adjust the process noise covariance matrix. and observation noise covariance matrix . The system dynamically adjusts its parameters based on real-time readings of the vehicle's longitudinal and yaw rates: when the vehicle is undergoing rapid acceleration, deceleration, or sharp turns, the system calculates a gain coefficient. To amplify the preset basic process noise matrix This makes the filter more trusting of current observations and able to respond quickly to sudden changes. Based on the visual occlusion status of the current matching target (e.g., a value of 1 for full visibility, 2 for partial occlusion, and 3 for full occlusion) and the confidence score given by the upstream detector. Dynamic adjustment: When the target is occluded or the confidence level is low, the system calculates a penalty coefficient. To amplify the preset basic observation noise matrix This forces the filter to reduce the weight of the current observation and rely mainly on historical states for inertial extrapolation, thereby reducing occlusion interference.
[0069] S103. Matching analysis based on three-level cascaded data.
[0070] In the embodiments of this application, corresponding to Figure 1 The data association module 130 functions by associating the detected target obtained in step S101 with the predicted trajectory obtained in step S102 using the following three-level cascading strategy:
[0071] Level 1 matching: 3D full bounding box strongly constrained matching. This is used to calculate the IoU (Intersection over Union) of the complete 3D bounding box between the detected target and the predicted trajectory, i.e., the 3D IoU, constructing a cost matrix and performing matching using the Hungarian algorithm. This level 1 matching is used to lock in high-quality matches that are close and morphologically stable, with a relatively high initial threshold, such as 0.7.
[0072] Second-level matching: BEV plane height decoupled matching. The unmatched detected targets and predicted trajectories from the first level are projected onto the bird's-eye view plane, ignoring height information. The generalized distance intersection-union ratio (GD IoU) is calculated, and a new cost matrix is constructed for matching. This second-level matching is used to eliminate vertical errors caused by vehicle bumps and road slope. The second matching threshold is less than the first threshold, for example, 0.5.
[0073] Third-level matching: 2D image plane depth decoupled matching. Detected targets and predicted trajectories that have not yet been matched in the first two levels are projected onto the forward-looking camera image plane, ignoring depth information. 2D IoU is calculated and a final match is performed. This third-level matching leverages the richness of visual information to address the problem of inaccurate depth estimation caused by the sparsity of distant point clouds. As a fallback strategy, the third threshold for matching is lower than the second threshold, for example, 0.3.
[0074] S104. Track Update and Full Lifecycle Management.
[0075] In the embodiments of this application, corresponding to Figure 1 The trajectory management module 140 performs corresponding processing based on the association results (including matching pairs, non-matching detections, and non-matching trajectories) from step S103. The lifecycle of the predicted trajectory is also processed accordingly based on the association results.
[0076] For a successful match: using the observations (such as position and size) of the matched detected target, update the state of the corresponding predicted trajectory (including motion state and bounding box information) through EKF.
[0077] For unmatched detected targets: treat them as potential newly emerging targets. Create a new trajectory for them and assign an initial confidence level. If the new trajectory can successfully associate with the detected target in the subsequent N0 consecutive frames (e.g., 3 frames), increase its confidence level; when the confidence level exceeds a stable threshold, confirm it as a new stable trajectory and assign it a unique ID.
[0078] For unmatched predicted trajectories: This may be due to the target temporarily disappearing or a false detection. Reduce the confidence of these trajectories. If a trajectory fails to match any detected target within N1 consecutive frames (e.g., 3 frames) and its confidence drops below the disappearance threshold, the target is considered to have disappeared, and the trajectory is deleted to free up resources.
[0079] In this embodiment of the application, the entire lifecycle management of the trajectory is automated through the above-mentioned management mechanism for predicting the trajectory lifecycle.
[0080] S105. Output the tracking results.
[0081] In this embodiment, the tracking results of stable predicted trajectories in the current frame image are output, including at least the stable identity, three-dimensional position, size, orientation, and velocity of each target. These results are fed back to downstream modules (such as prediction and planning) and used for the initialization of the next frame processing, such as returning to step S102.
[0082] It should be understood that, as mentioned above Figure 2The steps in the flowcharts are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise explicitly stated herein, there is no strict order in which these steps are performed; they can be executed in other orders. Furthermore, as mentioned above... Figure 2 The flowchart may include at least some steps or stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.
[0083] Figure 3 This is a schematic flowchart of an adaptive extended Kalman filter method provided in an embodiment of this application, as shown below. Figure 3 As shown, the specific execution steps of the method are as follows:
[0084] S201. Data Acquisition.
[0085] In this embodiment, input data streams from historical trajectory states and real-time scene perception signals are acquired. The historical trajectory state is the optimal state estimate (i.e., posterior estimate) obtained from the trajectory management module after the update of the previous frame, serving as the initial state for this prediction. The real-time scene perception signal is used as a key external input for adaptive adjustment.
[0086] In this embodiment, the real-time scene perception signal includes the vehicle's real-time speed acquired from the vehicle's CAN bus. and angular velocity ; Obtain the occlusion status of the current target And the target detection confidence score obtained from the upstream detection network. .
[0087] S202. Calculation of adaptive adjustment parameters.
[0088] In the embodiments of this application, this application provides an independent adjustment method for two key noise covariance matrices in EKF, including the following two mechanisms.
[0089] Process noise covariance matrix The dynamic adjustment mechanism is used to respond to the vehicle's motion status. Specifically, it includes: real-time detection of the vehicle's real-time speed. and angular velocity When the system detects that the vehicle is in a state of violent motion such as rapid acceleration, rapid deceleration, or sharp turning (i.e., the speed or angular velocity exceeds a preset threshold), the system calculates a gain coefficient based on the current motion intensity. Subsequently, this gain coefficient is applied to a preset base process noise matrix. This is amplified to generate a real-time value that is positively correlated with the vehicle's maneuverability. Value. Among them, , Increased The value indicates that the filter believes the uncertainty in the current frame has increased, so it will give the current observation a higher weight in the update step, so that the predicted trajectory can respond quickly to sudden movements of the vehicle or target and reduce lag.
[0090] Observation noise covariance A dynamic adjustment mechanism is used to respond to the quality of target observations. Specifically, this includes: real-time analysis of the target's occlusion level. and confidence score When a target is partially or completely occluded, or its detection confidence is below a reliable threshold, the system calculates a penalty coefficient based on the severity of the occlusion and the level of detection confidence. . Acting on the preset basic observation noise matrix This is amplified to generate real-time data that is positively correlated with the degree of unreliability of the observation. Value. Among them, , Increased The value indicates that the filter considers the current sensor observation unreliable, thus reducing the confidence in the current observation value in the update step. This forces the state estimation to rely mainly on the state of the previous moment for inertial prediction, effectively reducing the interference caused by detection box jitter or deformation due to occlusion.
[0091] S203. Adaptive Kalman filter calculation.
[0092] In this embodiment of the application, after obtaining the real-time data after the dynamic adjustment in step S202... and After the matrix is processed, a standard two-step extended Kalman filter calculation is performed, but by this point, the filtering process is already aware of the current scene. Specifically, it utilizes the motion model and the state of the previous frame, combined with dynamic... The prior estimate (i.e., predicted trajectory) and its uncertainty for the current frame state are calculated; when a detected target is successfully associated with this predicted trajectory, the filter will associate the associated observations (such as 3D position) with the dynamic... Substitute these values into the update equation to calculate the posterior optimal estimate of the current frame state.
[0093] S204. Output predicted trajectory.
[0094] In this embodiment, step S204 outputs the predicted trajectory set for the current frame. This predicted trajectory includes not only the target's estimated position and velocity at future times, but also dynamic... and The prediction confidence level for a specific scene in the current frame. These predicted trajectories will be directly fed into the next-level cascaded data association module as a benchmark for matching with the current real detection.
[0095] In the embodiments of this application, Figure 3 The illustrated method flow indicates a filter controller with environmental feedback capability, which can receive real-time data from the vehicle and dynamically adjust its internal model confidence level accordingly (i.e., ) and observation confidence (i.e. Thus, this application enables stable and accurate tracking in dynamic and complex scenarios.
[0096] Figure 4 This is a flowchart illustrating a three-level cascaded data association strategy provided in an embodiment of this application. Figure 4 As shown, the specific method and process are as follows:
[0097] S301. Process Initiation and Input.
[0098] In this embodiment, the input to step S301 is the set of detected targets in the current frame and the set of predicted trajectories generated by the EKF provided in this application. Both contain geometric representations of the same target in three-dimensional space, the bird's-eye view plane, and the two-dimensional image plane.
[0099] S302. Calculate and determine whether the first-level match is successful.
[0100] In this embodiment, a cost matrix is constructed by calculating the 3DIoU of the complete three-dimensional bounding boxes between all detected targets and predicted trajectories, and the optimal matching pair is solved by using the Hungarian algorithm. The success of the first-level matching is determined by judging whether the 3DIoU value of the matching pair is higher than a preset first threshold (such as 0.7).
[0101] In this embodiment, if the first-level matching is successful, a first matching result is obtained. The first matching result can be a target that is close-range, has a full point cloud, and a stable shape. The matching result is directly output to the final matching list. The successfully matched first matching result will be removed from the subsequent process, and the subsequent step S305 will be executed.
[0102] In this embodiment of the application, if the first-level matching fails, the detected target and the predicted trajectory that do not reach the first threshold are determined as unmatched items, and the subsequent S303 step is executed to make a more lenient matching attempt.
[0103] S303. Calculate and determine whether the second-level match is successful.
[0104] In this embodiment, the unmatched items left over from the first-level matching are projected onto the BEV plane, ignoring their vertical height (Z-axis) information, and simplified to a two-dimensional rotated rectangle. The GD IoU between them is calculated, a new cost matrix is constructed, and Hungarian matching is performed. The success of the second-level matching is determined by whether the GD IoU value of the matched pair is higher than a preset second threshold (e.g., 0.5). The second threshold is lower than the first threshold.
[0105] In this embodiment, if the second-level matching is successful, a second matching result is obtained. The second matching result indicates the target that was misjudged by the first-level matching due to non-overlapping heights. The successfully matched second matching result will be removed from the subsequent process, and the subsequent step S305 will be executed.
[0106] In this embodiment of the application, if the second-level matching fails, the detected target and the predicted trajectory that do not reach the second threshold are determined as unmatched items, and the subsequent S304 step is executed to make a more lenient matching attempt.
[0107] S304. Calculate and determine whether the third-level match is successful.
[0108] In this embodiment, the unmatched items remaining from the first two levels are projected onto the image plane of the camera, ignoring their depth (distance) information and simplifying them into two-dimensional axial bounding boxes. The 2D IoU between them is calculated for third-level matching. The success of the third-level matching is determined by whether the 2D IoU value of the matched pair is higher than a preset third threshold (e.g., 0.3), where the third threshold is lower than the second threshold.
[0109] In this embodiment of the application, if the third-level matching is successful, the third matching result is obtained, and the successful third matching result will execute the subsequent S305 step.
[0110] In this embodiment of the application, if the third-level matching fails, the matching process ends, and the detected targets and predicted trajectories that have not yet been matched are output and step S306 is executed.
[0111] S305. Update the predicted trajectory.
[0112] In this embodiment of the application, the set of matching pairs (including the first matching result, the second matching result, and the third matching result) is used to update the state of the existing predicted trajectory.
[0113] S306. Output unmatched detected targets.
[0114] In this embodiment, for the set of unmatched detection targets, those that are still not matched after three levels of screening are used as new target candidates for the system to initialize a new trajectory.
[0115] In this embodiment of the application, if a set of unmatched predicted trajectories fails to find a matching predicted trajectory after three levels of attempts, it is marked as a potentially lost target, and its confidence level will be reduced, thus entering the retention or extinction judgment process.
[0116] In the embodiments of this application, Figure 4 The illustrated method flow indicates a multi-level matching approach. This three-level matching strategy ensures that even when sensor data is interfered with by various factors, the system always selects the most reliable dimension for association. This maximizes the success rate of target association (high recall) while maintaining matching accuracy (high precision) under complex road conditions such as bumps, long distances, and occlusions. This improves the robustness of the overall vehicle tracking.
[0117] Figure 5 This is a schematic diagram of a method for trajectory lifecycle management provided in an embodiment of this application, such as... Figure 5 As shown, the specific method and process are as follows:
[0118] S401. Obtain unmatched detection targets and assign new trajectories.
[0119] In this embodiment, when the third-level cascaded association outputs an unmatched detection (i.e., a new observation not claimed by any existing trajectory), the system creates a new trajectory instance for it. This new trajectory can be assigned a temporary ID and given a low initial confidence level (e.g., 0.3). Simultaneously, a successful association counter is started.
[0120] S402. Determine the trajectory status based on confidence level.
[0121] In this embodiment, for an initialized new trajectory, if it successfully associates with the detected target in N0 consecutive frames (e.g., 3 frames), its confidence level will gradually increase with each successful match. When the accumulated confidence level of the new trajectory exceeds a high stability threshold (e.g., 0.7), the new trajectory is confirmed as a stable trajectory. The system can convert the ID of the new trajectory from temporary to stable ID and output it formally to the downstream module, indicating that the system has determined that the trajectory corresponds to a stable detected target.
[0122] In this embodiment, the mechanism ensures that only continuously observed targets are stably output, effectively filtering out isolated false detections caused by instantaneous sensor noise and guaranteeing the purity of the trajectory list.
[0123] In this embodiment of the application, for a confirmed trajectory, as long as it is successfully associated in the current frame, the system can use the observation value to update its status and maintain it in the confirmed status, while resetting the corresponding loss counter.
[0124] In this embodiment, when a confirmed trajectory fails to be associated with any detection in the current frame (i.e., becomes an unmatched trajectory output by the cascaded association module), the system can start or increment the loss counter while gradually reducing its confidence (e.g., decreasing by 0.1 per frame). During the loss period, the system relies solely on the EKF motion model for pure prediction without updating observations.
[0125] Specifically, if the trajectory is successfully re-associated with the detection during the lost state (e.g., within the next 1-2 frames), its state immediately switches back to the confirmed state, the lost counter is reset to zero, and the confidence level begins to recover. If the trajectory remains lost for N1 frames (e.g., 5 frames) without being re-associated, or if its confidence level falls below a very low extinction threshold (e.g., 0.1) during the decay process, it is determined that the trajectory has disappeared, and the system deletes the trajectory and all its historical information, and reclaims its ID resources.
[0126] In this embodiment, by employing the aforementioned mechanism for trajectory loss and recovery, the system can reduce the frequency of ID jumps caused by temporarily obscured targets (such as those briefly blocked by vehicles or trees). Simultaneously, the trajectory extinction condition ensures that detected targets that have moved away or disappeared can be promptly cleared.
[0127] S403. Output trajectory tracking results.
[0128] In the embodiments of this application, the tracking results include, but are not limited to, the ID, three-dimensional position, size, orientation, and velocity information of the detected target.
[0129] In the embodiments of this application, Figure 5 The illustrated method flow indicates adaptive trajectory management logic. It ensures the rigor of trajectory generation through multi-frame confirmation, handles intermittent target occurrences (such as occlusion) through confidence decay and recovery, and guarantees the effectiveness of system resources and the timeliness of the output list through timeout or low-confidence elimination. This mechanism is consistent with the aforementioned... Figures 2 to 3 The methods shown are combined to achieve tracking results with stable IDs.
[0130] In this embodiment, the target tracking method for adapting to scenarios and cascading data provided in this application is compared with the detection results of conventional algorithms. The detection items involved include: multiple object tracking accuracy (MOTA), ID switches (IDSW), trajectory fragmentation (FRAG), and average inference time per frame (Time). The quantization results of the conventional algorithm and the algorithm of this application in the vehicle (Car) and vulnerable road user (VRU) classes are shown in Table 1 below. The average quantization results of the conventional algorithm and the algorithm of this application on the overall test set are shown in Table 2 below.
[0131] Table 1. Quantification results of conventional algorithm and algorithm of this application in vehicle (Car) and vulnerable traffic participant (VRU) classes.
[0132]
[0133] Table 2. Average quantization results of conventional algorithm and algorithm of this application on the overall test set.
[0134]
[0135] In the embodiments of this application, the following advantages of the method provided in the embodiments of this application can be observed from the detection results shown in Tables 1 and 2:
[0136] 1. Improved overall accuracy of multi-target tracking. A higher MOTA indicates better accuracy for multi-target tracking. As shown in Table 1, the algorithm in this application achieves an MOTA of 70.85% for the Car category (with 44,792 true targets (GT), a 17.81 percentage point improvement compared to the 53.04% of the ordinary algorithm. Simultaneously, the algorithm achieves an MOTA of 51.09% for the VRU category (with 8,320 GT), nearly 2.5 times better than the 20.03% of the ordinary algorithm. As shown in Table 2, the algorithm in this application achieves an MOTA of 65.54% in the average quantization results on the overall test set, while the conventional algorithm only achieves 47.33%. In the embodiments of this application, this accuracy improvement directly benefits from the multi-stage cascaded data association strategy proposed in this application. Through a three-level screening process of 3D strong constraints, high decoupling of BEVs, and 2D visual matching, the problem of missed detections by ordinary algorithms in long-distance or occluded scenarios is effectively solved, especially significantly enhancing the recall capability for non-rigid targets such as VRUs.
[0137] 2. This significantly enhances tracking stability and solves the problem of frequent ID jumps. A lower IDSW indicates a more stable algorithm. As shown in Table 1, the IDSW of this application's algorithm in the Car category is 7.50, a reduction of 81.8% compared to the 41.25 of the ordinary algorithm. The IDSW of this application's algorithm in the VRU category is 6.63, compared to 7.13 for the ordinary algorithm. As shown in Table 2, the average IDSW of this application's algorithm on the overall test set is only 14.12, lower than the 48.38 required by the ordinary algorithm. This demonstrates that the state prediction of the EKF in the core of this application's algorithm plays a crucial role. Through rigorous trajectory confirmation logic and occlusion-resistant Kalman filter updates, the system can stably lock the target identity, effectively curbing target identity confusion caused by sensor noise or short-term occlusion.
[0138] 3. Improved trajectory continuity and effective resistance to environmental interference. The trajectory fragmentation (FRAG) index reflects the frequency of trajectory breaks; a lower FRAG indicates a more stable predicted trajectory. As shown in Table 1, the FRAG for the Car category in this application's algorithm is 84.375, while the FRAG for the Car category in the conventional algorithm is 133.88; the FRAG for the VRU category in this application's algorithm is 92.75, while the FRAG for the Car category in the conventional algorithm is 97.125. As shown in Table 2, the overall average FRAG in this application's algorithm is 177.12 times, which is about 37% lower than the FRAG of 285.25 in the conventional algorithm. The lower FRAG value indicates that the algorithm in this application has stronger anti-interference robustness. When the vehicle travels on bumpy roads or the target is partially obscured, the algorithm in this application, based on the image data of previous frames and the BEV plane matching strategy, can maintain trajectory continuity as much as possible, outputting smoother and more continuous perception results.
[0139] 4. Improved computational efficiency to meet the stringent real-time requirements of autonomous driving. As shown in Table 2, even with complex logic, the average inference time per frame of the algorithm in this application is only 25.6ms, which is nearly three times more efficient than the 71.9ms of ordinary algorithms. Although this application introduces multi-stage association, it reduces redundant and ineffective computations through a cascaded filtering mechanism (where most easily computed matching targets are completed in the first stage, eliminating the need for subsequent complex calculations) and efficient engineering design.
[0140] Example 3
[0141] Figure 6 This is a schematic diagram of the hardware structure of a computer device provided in an embodiment of this application. The computer device 600 may include the aforementioned... Figure 1 The target tracking system 100 shown is adapted to various scenarios and uses cascaded data. For example... Figure 6As shown, the computer device 600 includes: a processor 601, a memory 602, a communication module 604, and a computer program 603 stored in the memory 602 and executable on the processor 601. When the processor 601 executes the computer program 603, it implements the aforementioned... Figures 2-5 The execution steps are shown. For example, the computer program 603 described above can be divided into one or more units / modules, which are stored in the memory 602 and executed by the processor 601 to complete this application.
[0142] The aforementioned one or more units / modules may be a series of computer program instruction segments capable of performing a specific function. These instruction segments describe the execution process of the aforementioned computer program 603 within the aforementioned computer device 600. For example, the aforementioned computer program 603 may be used to perform actions such as... Figure 2 The specific functions or mechanisms of the methods shown in steps S101-S105 have been described in the above embodiments and will not be repeated here.
[0143] Those skilled in the art will understand that Figure 6 This is merely an example of computer device 600 and does not constitute a limitation on computer device 600. It may include more or fewer components than shown, or combine certain components, or different components. For example, the computer device 600 described above may also include input / output devices, network access devices, buses, etc.
[0144] The processor 601 mentioned above can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.
[0145] In some embodiments, the processor 601 may include one or more interfaces. These interfaces may include: an internal integrated circuit I2C interface, an integrated circuit built-in audio bus I2S interface, a pulse code modulation (PCM) interface, a universal asynchronous transceiver (URAT) interface, a mobile industry processor MIPI interface, a general purpose input / output (GPIO) interface, an on-board diagnostic (OBD) system interface, and / or a universal serial bus (USB) interface, etc. It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the computer device 600. In other embodiments of this application, the computer device 600 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.
[0146] In some embodiments, the computer device 600 can connect internal devices and modules through one or more interfaces. The aforementioned memory 602 can be an internal storage unit of the computer device 600, such as a hard disk or RAM. The aforementioned memory 602 can also include both internal storage units and external storage devices. The aforementioned memory 602 is used to store the aforementioned computer program and other programs and data required by the computer device 600. The aforementioned memory 602 can also be used to temporarily store data that has been output or will be output.
[0147] The communication module 604 can provide solutions for wireless communication applications on the computer device 600, including Wireless Local Area Network (WLAN), Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), and Infrared (IR). The communication module 604 can be one or more devices integrating at least one communication processing module. The communication module receives electromagnetic waves via an antenna, demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 601. The communication module 604 can also receive signals to be transmitted from the processor 601, frequency modulate and amplify them, and then convert them into electromagnetic waves for radiation via the antenna.
[0148] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above equipment can be divided into different functional units or modules to complete all or part of the functions described above.
[0149] The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of software functional units.
[0150] In the embodiments of this application, the specific names of each functional unit and module are only for easy distinction and are not intended to limit the scope of protection of this application. It should be understood that each step in the above-described method embodiments provided in this application can be completed by the integrated logic circuits in the processor hardware or by instructions in software form. The method steps disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.
[0151] This application also provides a computer program product, which includes: a computer program (also referred to as code or instructions) that, when run, causes a computer to execute the target tracking method of adapting to the scene and cascading data in the above embodiments.
[0152] The various embodiments of this application can be combined arbitrarily to achieve different technical effects.
[0153] In the embodiments provided in this application, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, in the form of a computer program product.
[0154] The computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.
[0155] This application also provides a computer-readable storage medium storing a computer program (also referred to as code or instructions). When the computer program is run, it causes the computer to perform the method executed by the computer device in any of the foregoing embodiments.
[0156] Figure 7 This is a schematic diagram of a computer-readable storage medium provided in an embodiment of this application. For example... Figure 7 As shown, the computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means.
[0157] The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Universal Optical Discs, DVDs), or semiconductor media (e.g., solid-state drives, SSDs), etc.
[0158] Those skilled in the art will understand that implementing all or part of the processes in the foregoing embodiments can be accomplished by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the foregoing method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or RAM, magnetic disks, or optical disks.
[0159] In summary, the above description is merely an embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made based on the disclosure of this application should be included within the scope of protection of this application.
Claims
1. A target tracking method that adapts to a scenario and cascades data, characterized in that, The method includes: Collect multi-source data, which includes at least lidar point clouds, 3D target detection results, millimeter-wave radar data, vehicle status data, and pose for simultaneous localization and map construction. The motion state of the detected target is predicted based on the multi-source data using an extended Kalman filter to generate a predicted trajectory. The process noise covariance matrix in the extended Kalman filter is dynamically adjusted according to the vehicle's motion state, and the observation noise covariance matrix in the extended Kalman filter is dynamically adjusted according to the target occlusion degree and detection confidence. The three-level cascaded strategy is used to associate the detected target and the predicted trajectory with data, including: in the first level matching, calculating the three-dimensional intersection-union ratio (IU / U) of the detected target and the predicted trajectory, and obtaining a first matching result in which the matching degree of the detected target and the predicted trajectory is higher than a first threshold; in the second level matching, projecting the detected target and the predicted trajectory with the matching degree not exceeding the first threshold onto a bird's-eye view plane, ignoring their height information, calculating the generalized distance IU / U, and obtaining a second matching result in which the matching degree is higher than a second threshold, where the second threshold is lower than the first threshold; in the third level matching, projecting the detected target and the predicted trajectory with the matching degree not exceeding the second threshold onto a two-dimensional image plane, ignoring their depth information, calculating the two-dimensional IU / U, and obtaining a third matching result in which the matching degree is higher than a third threshold, where the third threshold is lower than the second threshold; Based on the data association results, the predicted trajectory is updated and managed throughout its entire lifecycle, outputting tracking results that include at least the identity identifier.
2. The method according to claim 1, characterized in that, The vehicle's motion state includes real-time speed and angular velocity. The process noise covariance matrix in the extended Kalman filter is dynamically adjusted according to the vehicle's motion state, specifically including: The real-time speed and angular velocity of the vehicle are detected; When the real-time speed or the angular velocity exceeds a preset threshold, a gain coefficient is calculated based on the real-time speed and the angular velocity; The process noise covariance matrix is obtained by amplifying the preset basic process noise matrix based on the gain coefficient.
3. The method according to claim 1 or 2, characterized in that, The observation noise covariance matrix in the extended Kalman filter is dynamically adjusted according to the target occlusion degree and detection confidence, specifically including: The target occlusion degree and the detection confidence level are detected; When the target occlusion degree indicates that the target is partially or completely occluded, or when the detection confidence is lower than a preset threshold, a penalty coefficient is calculated based on the target occlusion degree and the detection confidence. The observation noise covariance matrix is obtained by amplifying the preset basic observation noise matrix based on the penalty coefficient.
4. The method according to claim 1, characterized in that, In the process of updating the predicted trajectory and performing full lifecycle management based on the data association results, the method includes: Receive the detection target whose matching degree does not exceed the third threshold, match the detection target with an initialized new trajectory, and assign an initial confidence level to the new trajectory; If the new trajectory is successfully associated with the detected target within N0 consecutive frames, the confidence level of the new trajectory is increased, where N0 is a positive integer greater than or equal to 2. When the confidence level of the new trajectory is higher than a preset stability threshold, it is confirmed as the predicted trajectory.
5. The method according to claim 1 or 4, characterized in that, The predicted trajectory is configured with a confidence level. In the process of updating the predicted trajectory and performing full lifecycle management based on the data association results, the method includes: If the predicted trajectory is not matched with the detected target within N1 consecutive frames, the confidence of the predicted trajectory is reduced, where N1 is a positive integer greater than or equal to 2. When the confidence level of the predicted trajectory is lower than a preset disappearance threshold, the detected target is determined to have disappeared and the predicted trajectory is deleted.
6. The method according to claim 1, characterized in that, In the process of predicting and detecting the motion state of the target based on the multi-source data using extended Kalman filtering, the method includes: The target radial velocity from the millimeter-wave radar data is used as an observation value and input into the state update equation of the extended Kalman filter to correct the motion state estimate of the detected target.
7. A target tracking system that adapts to a scenario and cascades data, characterized in that, For implementing the method as described in any one of claims 1 to 6, comprising: The data acquisition module is used to acquire multi-source data, which includes at least lidar point clouds, 3D target detection results, millimeter-wave radar data, vehicle status data, and pose for simultaneous localization and map construction. The state prediction module is used to predict the motion state of the detected target based on the multi-source data using extended Kalman filtering and generate a predicted trajectory. The process noise covariance matrix in the extended Kalman filter is dynamically adjusted according to the vehicle's motion state, and the observation noise covariance matrix in the extended Kalman filter is dynamically adjusted according to the target occlusion degree and detection confidence. A data association module is used to perform data association between the detected target and the predicted trajectory based on a three-level cascaded strategy, including: in the first-level matching, calculating the three-dimensional intersection-union ratio (IU / U) of the detected target and the predicted trajectory, and obtaining a first matching result in which the matching degree of the detected target and the predicted trajectory is higher than a first threshold; in the second-level matching, projecting the detected target and the predicted trajectory with the matching degree not exceeding the first threshold onto the bird's-eye view plane, ignoring their height information, calculating the generalized distance IU / U, and obtaining a second matching result with the matching degree higher than a second threshold, wherein the second threshold is lower than the first threshold; in the third-level matching, projecting the detected target and the predicted trajectory with the matching degree not exceeding the second threshold onto a two-dimensional image plane, ignoring their depth information, calculating the two-dimensional IU / U, and obtaining a third matching result with the matching degree higher than a third threshold, wherein the third threshold is lower than the second threshold; The trajectory management module is used to update the predicted trajectory and perform full lifecycle management based on the data association results, and output tracking results including at least the identity identifier.
8. A computer device, characterized in that, The device includes one or more memories and one or more processors; the memories are coupled to the one or more processors, the memories are used to store computer program code, the computer program code including computer instructions, and the one or more processors invoke the computer instructions to cause the computer device to perform the method as described in any one of claims 1 to 6.
9. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed by the processor, they implement the method of any one of claims 1 to 6.