3D multi-target tracking method based on improved Kalman filter based on the number of scene targets

Abstract

The present invention relates to the technical field of multi-target tracking, and discloses a 3D multi-target tracking method for improving Kalman filtering based on the number of scene targets, obtaining original point cloud data; processing the obtained original point cloud data to obtain a detection frame after processing the i-th frame point cloud data; adjusting the IOU threshold of the data association part by the number of the i-th frame target detection frame to obtain the IOU threshold in the current frame scene; obtaining the target association pairs between the tracking set and the detection set of the i-th frame; judging the obtained association pairs and distinguishing different target types; using the new tracking set as the input of the adaptive Kalman filter, and obtaining the optimal estimate of the target of the i+1-th frame through prediction, repeating the above steps until all point cloud data are traversed, thereby obtaining accurate tracking of multiple targets with target changes in different scenes; the method has high data association, can better balance the accuracy and stability of tracking in different scenes, and improves the overall tracking performance.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-target tracking, and in particular to a 3D multi-target tracking method based on an improved Kalman filter based on the number of scene targets. Background Art

[0002] In autonomous driving and robotics, accurate perception and tracking of dynamic objects in the surrounding environment (such as pedestrians, vehicles, and obstacles) is key to achieving safe and efficient autonomous navigation. Multi-target tracking technology aims to simultaneously track multiple dynamic targets from sensor data, provide the motion trajectory and predicted position of each target. Compared with two-dimensional image data, the three-dimensional point cloud data provided by LIDAR contains rich spatial information, which helps to detect and track targets more accurately.

[0003] Although neural network-based learning methods have performed well in the field of target tracking, they have large computing resource requirements, large data requirements, poor model interpretability, insufficient robustness to environmental changes, and high training and deployment complexity. Therefore, in comparison, traditional target tracking methods have certain advantages in terms of computational complexity, resource requirements, and stability, and still have important application value, especially in resource-constrained applications and applications that require high interpretability.

[0004] At present, traditional target tracking methods usually rely on feature-based algorithms, such as optical flow, particle filtering, and Kalman filtering. These methods use image processing and statistical techniques to track the trajectory of the target. The main advantages of these methods are low computational complexity, low hardware requirements, and the ability to provide stable and accurate tracking results under certain conditions.

[0005] The traditional Kalman filter algorithm is simple, and generally uses a fixed process noise in the Kalman filter process. However, when the scene changes very quickly, the process noise is constantly changing. Therefore, due to its simple calculation method, errors are prone to occur, and the task of 3D multi-target tracking cannot be completed. In the process of judging the accurate association pairs, a fixed IOU threshold is generally used. However, in high-density scenes, the distance between vehicles is close, and there may be more occlusion and confusion. In low-density scenes, the distance between vehicles is far, and there are fewer occlusions and confusions. Therefore, a fixed IOU threshold cannot enable the tracker to flexibly respond to target changes in different scenes. Summary of the invention

[0006] In view of the shortcomings of the prior art, the object of the present invention is to provide a 3D multi-target tracking method based on an improved Kalman filter based on the number of scene targets.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] The 3D multi-target tracking method based on the improved Kalman filter based on the number of scene targets includes the following steps:

[0009] A. Use the radar transmission center as the origin of the coordinate system to establish the radar coordinate system and obtain the original point cloud data;

[0010] B. Processing the obtained original point cloud data to obtain the detection frame after processing the i-th frame of point cloud data, which is used to represent each target information;

[0011] C. Use the number of target detection boxes in the i-th frame to adjust the IOU threshold of the data association part to obtain the IOU threshold in the current frame scene;

[0012] D. Obtain the target association pair between the tracking set and the detection set of the i-th frame;

[0013] E. Determine whether the weight of each obtained association pair is less than the dynamic IOU threshold, and distinguish them into different target types, including tracking success type, detection error type and occlusion type;

[0014] F. Use the association pairs of different target types obtained in step E as the input of the adaptive Kalman filter, and after prediction, obtain the optimal estimate of the target in the i+1th frame. Repeat the above steps until all point cloud data are traversed, thereby obtaining accurate multi-target tracking of target changes in different scenes.

[0015] In the present invention, preferably, in step A, a radar coordinate system is established with the forward direction of the smart vehicle as the X-axis, the left direction of the driver as the Y-axis, and the top of the vehicle as the Z-axis.

[0016] In the present invention, preferably, in step B, the obtained original point cloud data is subjected to the pointrcnn target detection algorithm to obtain a detection frame after processing the i-th frame point cloud data, which is used to represent each target information, and the detection set of the i-th frame and the tracking set of the i-th frame obtained by Kalman filtering prediction of the i-1 frame are passed together to the next step C.

[0017] In the present invention, preferably, in step C, the IOU threshold in the current frame scene is:

[0018] , k is the adjustment parameter, controlling the steepness of the curve, is the midpoint of the density, is the maximum value of the crossover ratio, is the minimum value of the crossover ratio, is the target density value of the current frame, is the final cross ratio threshold.

[0019] In the present invention, preferably, in step D, the method further comprises the following steps:

[0020] D1, use the DIOU algorithm to upgrade to three dimensions to calculate the association matrix between the tracking set and the detection set targets;

[0021] D2. Use the obtained association matrix as the input of the Hungarian algorithm to obtain the accurate association pairs of the detection target set and the tracking target set.

[0022] In the present invention, preferably, in step D1, the DIOU algorithm specifically adopts:

[0023] , is the center distance between the target detection box of the tracking set and the target detection box of the detection set,

[0024] ,

[0025] in , is the three-dimensional coordinate of the center of gravity of the target detection box of the tracking set in the current radar coordinate system, , is the three-dimensional coordinate of the center of gravity of the target detection box of the detection set in the current radar coordinate system, and c is the diagonal length of the smallest cube that can contain two target detection boxes.

[0026] In the present invention, preferably, in the step E, the steps are specifically included:

[0027] E1. Determine whether the weight of each association pair is less than the dynamic IOU threshold. If so, it means that there is an association between the two targets in the association pair, otherwise, there is no association between the two targets;

[0028] E2, the successfully associated pair is a tracking success type, and the Kalman filter's tracking target information is updated with the associated pair as an observation value. At the same time, if the number of consecutive associations of the tracking target in the associated pair is greater than three frames, it is output to the display program;

[0029] E3, the detected target that failed to be associated is regarded as a new target entering the scene. If it can be successfully associated for three consecutive frames in the subsequent tracking process, it is a detection error type and is output to the display program;

[0030] E4. The tracking target that failed to be associated is regarded as the target that is leaving the scene. If there is no successful association for two consecutive frames in the subsequent tracking process, it is an occlusion type and its output to the display program is stopped;

[0031] E5. Use the number of detection boxes in the i-th frame detection set as input again to adjust the Kalman filter process noise Q, thereby realizing an adaptive Kalman filter.

[0032] In the present invention, preferably, in step E5, the noise Q is specifically adjusted according to:

[0033] , is the basic process noise covariance matrix, which is given when the filter is initialized, N is the number of detection boxes in the current frame detection set, The preset maximum value of the scene metric is used to normalize N. Alpha is an adjustment factor used to control the influence of N on Q.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. The method of the present invention adds the weight calculated from the target quantity information in each frame of the scene as a penalty term to the process noise covariance matrix Q. Before predicting the target position of the next frame through the Kalman filter, the process noise Q is updated according to the number of targets in the scene of the previous frame, so that the process noise is more consistent with the current scene, and then more accurate prediction information is obtained, thereby improving the accuracy of subsequent data association, and then improving the tracking accuracy in the case of rapid scene switching.

[0036] 2. The present invention proposes a method for dynamically adjusting the IOU threshold based on the number of scene targets. First, accurate association pairs are obtained through the Hungarian association algorithm, and then the SigMoid algorithm is used to calculate the appropriate IOU threshold for the current scene according to the number of targets in the scene. Finally, the accuracy of the association pair is judged by this threshold. Dynamically adjusting the IOU threshold according to vehicle density can better balance the accuracy and stability of tracking in different scenes, thereby improving the overall tracking performance.

[0037] 3. The present invention proposes to use DIOU to replace GIOU of the previous algorithm and upgrade it to a three-dimensional method. The introduction of DIOU makes up for the defect that GIOU cannot distinguish the relative states of targets when targets overlap, so that even when targets overlap, the weights reflected by different states of targets can be calculated, thereby improving the accuracy of subsequent data association. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 The figure is a schematic diagram of the overall process of the 3D multi-target tracking method based on the improved Kalman filter of the invention based on the number of scene targets.

[0039] Figure 2 The figure is a schematic diagram of the algorithm flow of the 3D multi-target tracking method based on the improved Kalman filter of the invention based on the number of scene targets. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0042] See also Figure 1 and Figure 2 A preferred embodiment of the present invention provides a 3D multi-target tracking method based on an improved Kalman filter based on the number of scene targets, comprising the steps of:

[0043] A. Take the radar transmission center as the origin O of the coordinate system and establish the radar coordinate system O-XYZ; establish the radar coordinate system with the forward direction of the intelligent vehicle as the X-axis, the left direction of the driver as the Y-axis, and the top of the vehicle as the Z-axis, and obtain the original point cloud data. The original point cloud data is the point cloud information of the vehicle's external environment. The traditional tracking algorithm generally uses a fixed IOU threshold, which will cause the algorithm to have poor adaptability to scenes with large changes in target density. This method is to dynamically adjust the IOU threshold and also realize adaptive adjustment of the process noise in the Kalman filter prediction process to realize adaptive Kalman filtering, and finally improve the stability and adaptability of the tracking algorithm.

[0044] B. Process the obtained original point cloud data through the pointrcnn target detection algorithm to obtain the detection frame after the i-th frame of point cloud data is processed, which is used to represent each target information, and the target information is a vehicle, a bicycle or a pedestrian. The specific steps also include:

[0045] B1. The pointrcnn target detection algorithm uses the prediction function of the Kalman filter algorithm to predict the position of the detection frame of the i-th frame of the tracked target. The state vector of each target is , where the motion model uses a uniform velocity model, so the prediction equation is ;

[0046] B2. The prediction box of the i-th frame predicted according to the tracking target of the i-1th frame is used as the tracking set. The tracking set is obtained in the previous cycle frame step E of this method. The detection box of the i-th frame obtained by the pointrcnn target detection algorithm is used as the detection set, that is, the detection box obtained in step B1, and is passed to step C together.

[0047] C. Use the number of target detection boxes in the i-th frame to adjust the IOU threshold of the data association part. The SigMoid function is mainly used to dynamically adjust the IOU threshold to obtain the IOU threshold in the current frame scene:

[0048] , k is the adjustment parameter, controlling the steepness of the curve, is the midpoint of the density, is the maximum value of the crossover ratio, is the minimum value of the crossover ratio, is the target density value of the current frame, is the final cross ratio threshold.

[0049] D. Obtain the target association pair between the tracking set and the detection set of the i-th frame. The target association pair is to associate the predicted target of the i-th frame obtained by the prediction of the i-1-th frame with the detected target of the i-th frame input by the pointrcnn target detection algorithm:

[0050] D1, use the DIOU algorithm to upgrade to three dimensions to calculate the association matrix between the tracking set and the detection set targets, , is the center distance between the target detection box of the tracking set and the target detection box of the detection set,

[0051] , in , is the three-dimensional coordinate of the center of gravity of the target detection box of the tracking set in the current radar coordinate system, , is the three-dimensional coordinate of the center of gravity of the target detection box of the detection set in the current radar coordinate system, and c is the diagonal length of the smallest cube that can contain two target detection boxes;

[0052] D2. Use the obtained association matrix as the input of the Hungarian algorithm to obtain accurate target association pairs of the detection target set and the tracking target set.

[0053] E. Determine whether the weight of each associated pair obtained is less than the dynamic IOU threshold, and distinguish different target types, including tracking success type, detection error type and occlusion type:

[0054] E1. Determine whether the weight of each association pair is less than the dynamic IOU threshold. If so, it means that there is an association between the two targets in the association pair, otherwise, there is no association between the two targets;

[0055] E2, the successfully associated pair is a tracking success type, and the Kalman filter's tracking target information is updated with the associated pair as an observation value. At the same time, if the number of consecutive associations of the tracking target in the associated pair is greater than three frames, it is output to the display program;

[0056] E3, the detected target that failed to be associated is regarded as a new target entering the scene. If it can be successfully associated for three consecutive frames in the subsequent tracking process, it is a detection error type and is output to the display program;

[0057] E4. The tracking target that failed to be associated is regarded as the target that is leaving the scene. If there is no successful association for two consecutive frames in the subsequent tracking process, it is an occlusion type, and it will stop being output to the display program;

[0058] E5. Use the number of detection boxes in the i-th frame detection set as input to adjust the Kalman filter process noise Q. , is the basic process noise covariance matrix, given when the filter is initialized, N is the number of detection boxes in the current frame detection set,, The preset maximum value of the scene metric is used to normalize N. Alpha is an adjustment factor used to control the influence of N on Q, thereby realizing an adaptive Kalman filter.

[0059] F. Take the new tracking set as the input of the adaptive Kalman filter, and after prediction, obtain the optimal estimate of the target in the i+1th frame, repeat the above steps until all point cloud data are traversed, so as to obtain accurate tracking of multiple targets with target changes in different scenes. By introducing a mechanism for dynamically adjusting the IOU threshold, the algorithm can dynamically adjust the IOU threshold according to the number of detection frames, so as to better adapt to the target tracking task in different density scenes. After experiments, as shown in Table 1 below, the improved algorithm shows higher tracking performance and better stability in both high-density and low-density scenes. Compared with the traditional fixed IOU threshold method, this dynamic adjustment mechanism has obvious advantages in dealing with complex scenes. In Table 1, MOTA represents tracking accuracy, MOTP represents tracking precision, AMOTA represents flat tracking accuracy, AMOTP represents average tracking precision, and sAMOTA represents scaling accuracy index.

[0060] Table 1. Comparison of tracking effect parameters between the algorithm of the present invention and the traditional AB3DMOT algorithm

[0061]

[0062] In some other preferred embodiments of the present invention, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the method described in the above embodiment.

[0063] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0064] The above description is a detailed description of the preferred feasible embodiments of the present invention, but the embodiments are not intended to limit the scope of the patent application of the present invention. All equivalent changes or modified changes completed under the technical spirit suggested by the present invention should fall within the patent scope covered by the present invention.

Claims

1. A 3D multi-target tracking method based on an improved Kalman filter based on the number of scene targets, characterized in that: Includes steps: A. Use the radar transmission center as the origin of the coordinate system to establish the radar coordinate system and obtain the original point cloud data, where the original point cloud data is the point cloud information of the vehicle's external environment; B. Process the obtained original point cloud data to obtain the detection frame after processing the i-th frame point cloud data, which is used to represent each target information. The target information is a vehicle, a bicycle or a pedestrian. The obtained i-th frame detection frame is used as the detection set, and the i-th frame prediction frame predicted by the Kalman filter based on the i-1th frame is used as the tracking set. The detection set and the tracking set are passed to the next step C together; C. Use the number of target detection boxes in the i-th frame to adjust the IOU threshold of the data association part to obtain the IOU threshold in the current frame scene: , k is the adjustment parameter, controlling the steepness of the curve, is the midpoint of the density, is the maximum value of the crossover ratio, is the minimum value of the crossover ratio, is the target density value of the current frame, is the final crossover ratio threshold; D. Obtain the target association pair between the tracking set and the detection set of the i-th frame; E. Determine whether the weight of each obtained association pair is less than the dynamic IOU threshold, and distinguish them into different target types, including tracking success type, detection error type and occlusion type; Specifically include step: E1. Determine whether the weight of each association pair is less than the dynamic IOU threshold. If so, it means that there is an association between the two targets in the association pair, otherwise, there is no association between the two targets; E2, the successfully associated pair is a tracking success type, and the Kalman filter's tracking target information is updated with the associated pair as an observation value. At the same time, if the number of consecutive associations of the tracking target in the associated pair is greater than three frames, it is output to the display program; E3, the detected target that failed to be associated is regarded as a new target entering the scene. If it can be successfully associated for three consecutive frames in the subsequent tracking process, it is a detection error type and is output to the display program; E4. The tracking target that failed to be associated is regarded as the target that is leaving the scene. If there is no successful association for two consecutive frames in the subsequent tracking process, it is an occlusion type and its output to the display program is stopped; E5. Use the number of detection boxes in the i-th frame detection set as input again to adjust the Kalman filter process noise Q, thereby realizing an adaptive Kalman filter, wherein the noise Q is specifically adjusted based on: , is the basic process noise covariance matrix, which is given when the filter is initialized, N is the number of detection boxes in the current frame detection set, The preset maximum value of the scene metric is used to normalize N. Alpha is an adjustment factor used to control the influence of N on Q. F. Use the association pairs of different target types obtained in step E as the input of the adaptive Kalman filter, and after prediction, obtain the optimal estimate of the target in the i+1th frame. Repeat the above steps until all point cloud data are traversed, thereby obtaining accurate multi-target tracking of target changes in different scenes.

2. The 3D multi-target tracking method based on the improved Kalman filter based on the number of scene targets according to claim 1 is characterized in that: In step A, a radar coordinate system is established with the forward direction of the smart vehicle as the X-axis, the left direction of the driver as the Y-axis, and the top of the vehicle as the Z-axis.

3. The 3D multi-target tracking method based on the improved Kalman filter based on the number of scene targets according to claim 2 is characterized in that: In step B, the obtained original point cloud data is subjected to the pointrcnn target detection algorithm to obtain the detection frame after processing the i-th frame point cloud data, which is used to represent each target information.

4. The 3D multi-target tracking method based on the improved Kalman filter based on the number of scene targets according to claim 1 is characterized in that: In step D, the method further includes the following steps: D1, use the DIOU algorithm to upgrade to three dimensions to calculate the association matrix between the tracking set and the detection set targets; D2. Use the obtained association matrix as the input of the Hungarian algorithm to obtain the accurate association pairs of the detection target set and the tracking target set.

5. The 3D multi-target tracking method based on the improved Kalman filter based on the number of scene targets according to claim 4 is characterized in that: In step D1, the DIOU algorithm specifically adopts: , is the center distance between the target detection box of the tracking set and the target detection box of the detection set, , in , is the three-dimensional coordinate of the center of gravity of the target detection box of the tracking set in the current radar coordinate system, , is the three-dimensional coordinate of the center of gravity of the target detection box of the detection set in the current radar coordinate system, and c is the diagonal length of the smallest cube that can contain two target detection boxes.

Images