Multi-target tracking method and device for complex scene
Through the combination of YOLOX detector and Kalman filter, the target trajectory is divided and matched, and the identity switching, error matching and target loss problems of multi-object tracking in complex scenarios are solved, improving the accuracy and performance of tracking.
Patent Information
- Application Number
- CN202411046418.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-26
AI Technical Summary
In complex scenarios, the multi-objective tracking method has problems such as identity switching, error matching and target loss, especially in complex and crowded scenarios of target motion patterns.
The target detection is performed using the YOLOX detector, and the high confidence and low confidence detection frames are divided into a confirmed state trajectory and an unconfirmed state trajectory. The target trajectory is predicted through the Kalman filter, and the target matrix is constructed for matching. Hungarian algorithms and interleaving matrix techniques improve matching accuracy.
It significantly improves the accuracy of multi-target tracking, reduces the occurrence of identity switching, error matching and target loss, and improves tracking performance in complex scenarios.
Smart Images

Figure CN119027450A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision technology, and in particular to a multi-target tracking method and device in complex scenes. Background Art
[0002] Multi-target tracking is an important computer vision task that is widely used in video surveillance, autonomous driving, intelligent transportation systems, motion analysis, etc. Its goal is to simultaneously track multiple target objects (such as people, vehicles, animals, etc.) in a sequence of video frames and maintain the identity continuity of each target object.
[0003] Among them, multi-target tracking technology involves technologies such as data association and feature extraction. In terms of data association technology, methods such as Kalman filter or deep neural network are used to predict the position of the target, and corrections are made based on the detection results. Methods such as Hungarian algorithm are used to achieve the optimal match between the target and the trajectory. In terms of feature extraction technology, traditional methods rely on manually designed features. With the development of deep learning technology, feature extraction methods based on neural networks provide better robustness.
[0004] In order to solve the occlusion and interaction problems of multiple targets, the current multi-target tracking method uses more advanced deep learning models to extract more robust features to ensure tracking performance. However, in the face of complex target motion patterns and crowded scenes, there are still many problems of identity switching, mismatching and target loss. Summary of the invention
[0005] In order to overcome the above technical defects, the present invention provides a multi-target tracking method and device for complex scenes, which can solve the above-mentioned problems of identity switching, mismatching and target loss when targets frequently interact and occlude in complex scenes.
[0006] In order to solve the above problems, the present invention is implemented according to the following scheme:
[0007] A multi-target tracking method for complex scenes is provided, which is characterized by comprising:
[0008] The YOLOX detector is used to detect the target in the video frame to be processed, and the detection frame is divided into a high-confidence detection frame and a low-confidence detection frame according to the set confidence threshold;
[0009] Divide the tracking trajectory of the previous video frame into states;
[0010] Based on the state of the tracking trajectory, the target trajectory is divided into a confirmed state trajectory and an unconfirmed state trajectory;
[0011] Predicting the target trajectory in the target tracking frame of the current video frame through a Kalman filter;
[0012] Constructing a target matrix according to the high-confidence detection frame, the low-confidence detection frame, the confirmed state trajectory, the unconfirmed state trajectory, and the target tracking frame;
[0013] Matching the target trajectory according to the target matrix;
[0014] The above steps are repeated for the video frame sequence to be processed to obtain a trajectory set including multiple target trajectories.
[0015] Compared with the prior art, the beneficial effects of a multi-target tracking method for complex scenes provided by the present invention are as follows: the target trajectory is divided into two states, and the confirmed state trajectory and the unconfirmed state trajectory are matched respectively, so as to ensure the accuracy of trajectory matching, the target trajectory and the detection frame are divided according to preset conditions, and matched according to different state conditions, so as to ensure that the target trajectory matches the correct target, improve the accuracy of target tracking, significantly improve the tracking performance in complex scenes, reduce the problems of identity switching, mismatching and target loss in multi-target tracking in the face of complex target motion patterns and crowded scenes, thereby improving the accuracy of multi-target tracking.
[0016] Optionally, the target tracking frame includes:
[0017] The target tracking frame of the confirmed state trajectory is determined as a confirmed target tracking frame, and the target tracking frame of the unconfirmed state trajectory is determined as an unconfirmed target tracking frame.
[0018] Optionally, constructing a target matrix according to the high-confidence detection frame, the low-confidence detection frame, the confirmed state trajectory, the unconfirmed state trajectory, and the target tracking frame includes:
[0019] Constructing an intersection-over-union matrix according to the high-confidence detection frame and the confirmed target tracking frame;
[0020] Constructing an appearance depth feature similarity matrix according to the appearance depth features of the high confidence detection frame and the appearance depth features of the confirmed target tracking frame;
[0021] Constructing a trajectory confidence similarity matrix according to the confidence of the detection frame and the confidence of the target tracking frame;
[0022] A speed direction similarity matrix is constructed according to the speed direction of the detection frame and the speed direction of the target tracking frame.
[0023] Optionally, matching the target trajectory according to the target matrix includes:
[0024] The first matching is to match the high confidence detection frame with the confirmation state trajectory;
[0025] A second matching process is to match the high confidence detection frame and the unconfirmed state trajectory that were not successfully matched in the first matching process;
[0026] A third matching process is to match the confirmation state trajectory that was not successfully matched in the first matching process with the low confidence detection frame;
[0027] The fourth matching is to match the high-confidence detection frame that was not successfully matched in the second matching and the confirmation state track that was not successfully matched in the third matching;
[0028] Update the status of the successfully matched trajectory.
[0029] Optionally, the first matching includes:
[0030] According to the appearance feature similarity matrix, the trajectory confidence similarity matrix, the speed direction similarity matrix, and the intersection-over-union matrix, the high confidence detection frame and the confirmation state trajectory are matched by the Hungarian algorithm;
[0031] Determine whether an intersection-over-union ratio of the high-confidence detection frame and the confirmed target tracking frame is greater than a threshold;
[0032] If the intersection-and-union ratio is greater than the threshold, the match is successful, and the status of the confirmed status track of the successful match is updated;
[0033] If the intersection-over-union ratio is less than the threshold, it means that the match is not successful;
[0034] The high confidence detection frame that has not been successfully matched performs the second matching;
[0035] The confirmation state track that has not been successfully matched is matched for the third time.
[0036] Optionally, the second matching includes:
[0037] Matching the high-confidence detection frame and the unconfirmed state trajectory that were not successfully matched in the first matching according to the appearance feature similarity matrix, the trajectory confidence similarity matrix, the speed direction similarity matrix, and the intersection-over-union matrix;
[0038] Discarding the unconfirmed state track that has not been successfully matched;
[0039] The high confidence detection frame that has not been successfully matched performs the fourth matching;
[0040] The state of the unconfirmed state track that is successfully matched is updated.
[0041] Optionally, the third matching includes:
[0042] Matching the confirmed state trajectory that was not successfully matched in the first matching with the low confidence detection frame according to the appearance feature similarity matrix, the trajectory confidence similarity matrix, and the intersection-over-union matrix;
[0043] The confirmation state track that has not been successfully matched performs the fourth matching;
[0044] The status of the successfully matched confirmation status track is updated.
[0045] Optionally, the fourth matching includes:
[0046] Determine the high-confidence detection frame that is not successfully matched in the second matching as an unmatched detection frame;
[0047] Determining the confirmed state trajectory that is not successfully matched in the third matching as an unmatched trajectory;
[0048] Constructing a quadratic intersection-over-union matrix according to the target tracking frame of the unmatched trajectory in the previous video frame and the unmatched detection frame;
[0049] Matching the unmatched detection frame and the unmatched trajectory according to the quadratic intersection-over-union matrix and the appearance feature similarity matrix;
[0050] Discarding the unmatched track that has not been successfully matched;
[0051] Initialize the unmatched detection frame that has not been successfully matched, and enter the next video frame;
[0052] The status of the unmatched track that is successfully matched is updated.
[0053] Optionally, predicting the target tracking frame of the target trajectory in the current video frame by using a Kalman filter includes:
[0054] Predicting the target tracking frame of the target trajectory in the current video frame by using the Kalman filter;
[0055] Selecting feature points in the previous video frame and the current video frame;
[0056] Comparing the difference between the feature points in the previous video frame and the current video frame to determine a rotation matrix;
[0057] The target tracking frame is corrected according to the rotation matrix.
[0058] A computer device is also provided, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement the above-mentioned multi-target tracking method. BRIEF DESCRIPTION OF THE DRAWINGS
[0059] Figure 1 is a flow chart of the multi-target tracking method of the present invention;
[0060] Figure 2 The present invention is a flowchart of performing trajectory matching on each video frame in the multi-target tracking method. DETAILED DESCRIPTION
[0061] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.
[0062] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and do not have to be used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.
[0063] See also Figure 1 As shown, the present invention provides a multi-target tracking method for complex scenes, which is used to match targets in a video frame sequence, including:
[0064] S1: Perform target detection on the video frame to be processed through the YOLOX detector, and divide the detection frame into a high-confidence detection frame and a low-confidence detection frame according to the set confidence threshold. Specifically, extract the confidence of the target in each detection frame respectively, determine the detection frame with a confidence greater than the confidence threshold as a high-confidence detection frame, and determine the detection frame with a confidence less than the confidence threshold as a low-confidence detection frame.
[0065] Among them, the high-confidence detection frame is the case where the YOLOX detector is more confident in identifying the target. The high-confidence detection frame is usually the main object of tracking. Using these high-confidence detection frames, the target trajectory can be initialized and updated more accurately; and the low-confidence detection frame may contain more false positives, such as background objects or artifacts, compared with the high-confidence detection frame. By distinguishing between high- and low-confidence detection frames, low-confidence detection frames can be selectively ignored during the tracking process, thereby reducing the impact of false positives on the tracking results. When the target is partially occluded or disappears temporarily, the number of high-confidence detection frames may decrease. At this time, the low-confidence detection frame can provide additional information to help restore the target trajectory, thereby reducing the occurrence of target loss.
[0066] S2: dividing the tracking track of the previous video frame into states, specifically, determining the state of the tracking track that is successfully matched as a confirmed state, and determining the state of the tracking track that is not successfully matched as an unconfirmed state.
[0067] S3: Based on the status of the tracking trajectory, the target trajectory is divided into a confirmed state trajectory and an unconfirmed state trajectory.
[0068] S4: predicting the target tracking frame of the target trajectory in the current video frame through the Kalman filter, including: first predicting the target tracking frame of the target trajectory in the current video frame through the Kalman filter, selecting the feature points in the previous video frame and the current video frame, and then using the camera motion compensation module to compare the difference between the feature points in the previous video frame and the current video frame, determine the rotation matrix, and finally correct the target tracking frame according to the rotation matrix to reduce the impact of camera shaking on prediction accuracy.
[0069] According to the confirmed state and the unconfirmed state of the target track, the target tracking frame is divided into states like the target track, the target tracking frame of the confirmed state track is determined as the confirmed target tracking frame, and the target tracking frame of the unconfirmed state track is determined as the unconfirmed target tracking frame.
[0070] S5: construct a target matrix according to the high confidence detection frame, the low confidence detection frame, the confirmed state trajectory, the unconfirmed state trajectory, and the target tracking frame, wherein the target matrix intersection ratio matrix, the appearance depth feature similarity matrix, the trajectory confidence similarity matrix, and the speed direction similarity matrix. The process of constructing the target matrix includes:
[0071] According to the high-confidence detection frame and the confirmed target tracking frame, an intersection-over-union matrix is constructed, that is, according to the high-confidence detection frame and the target tracking frame corresponding to the confirmed state trajectory, an intersection-over-union matrix is constructed.
[0072] The SBS-S50 re-identification model is used to extract the appearance depth features of the target in the high-confidence detection frame and the appearance depth features of the target in the confirmed target tracking frame, and an appearance depth feature similarity matrix is constructed based on the appearance depth features of the target in the high-confidence detection frame and the appearance depth features of the target in the confirmed target tracking frame.
[0073] The confidences of the targets in the high-confidence detection frame, the low-confidence detection frame, the confirmed target tracking frame, and the unconfirmed target tracking frame are extracted respectively. According to the confidences of the targets locked by the high-confidence detection frame, the low-confidence detection frame, the confirmed target tracking frame, and the unconfirmed target tracking frame, a trajectory confidence similarity matrix is constructed, that is, a confidence similarity matrix is constructed according to the confidences corresponding to the four.
[0074] The speed directions of the targets in the high-confidence detection frame, the low-confidence detection frame, the confirmed target tracking frame and the unconfirmed target tracking frame are extracted respectively. According to the speed directions of the targets locked by the high-confidence detection frame, the low-confidence detection frame, the confirmed target tracking frame and the unconfirmed target tracking frame, a speed direction similarity matrix is constructed.
[0075] S6: Matching the target trajectory according to the target matrix, including:
[0076] First matching: According to the appearance feature similarity matrix, trajectory confidence similarity matrix, speed direction similarity matrix, and intersection-over-union matrix, the high-confidence detection frame and the confirmed state trajectory are matched through the Hungarian algorithm; whether the intersection-over-union ratio of the high-confidence detection frame and the confirmed target tracking frame is greater than the threshold is determined; if the intersection-over-union ratio is greater than the threshold, the match is successful, and the state of the successfully matched confirmed state trajectory is updated; if the intersection-over-union ratio is less than the threshold, the match is unsuccessful; and the unsuccessfully matched high-confidence detection frame enters the second matching, and the unsuccessfully matched confirmed state trajectory enters the third matching.
[0077] The specific process of updating the status in the first match includes:
[0078] First, the Kalman filter is used to predict the state of the target in the current video frame based on the state of the target in the previous video frame. The specific formula for the prediction result is as follows:
[0079] x(k∣k-1)=F·x(k-1∣k-1)+B·u(k)
[0080] P(k∣k-1)=F·P(k-1∣k-1)·F T +Q k
[0081] Where x represents the estimated state vector of the system. As new measurement data is acquired, this state will be continuously updated. x(k|k-1) means that the current system state is k, and the current system state is predicted based on the previous state of the system; F represents the dynamic characteristics of the system, that is, predicting the state at the next moment from the current state; u is the control input vector, and B is the control matrix. The control input vector u and the control matrix B are used to describe the influence of known external control inputs on the system state. These items are optional and are usually used when there are external inputs affecting the system; Q is the process noise covariance matrix, which is used to represent the uncertainty of the process model and reflects the changes that cannot be explained by the model during the state transfer process; P is the error covariance matrix, which is used to represent the uncertainty of the state estimate. It reflects the accuracy of the current estimate with each update of the prediction and measurement.
[0082] Then, the Kalman filter is used to calculate the margin and Kalman gain between the observed actual state and the predicted state based on the above predicted state, and the matched trajectory state information and covariance matrix are updated. The specific formula is as follows:
[0083] y(k)=z(k)-H·x(k|k-1)
[0084]
[0085] Kg(k)=P(k|k-1)·H T ·S(k) -1
[0086] x(k∣k)=x(k∣k-1)+Kg(k)·y(k)
[0087] P(k∣k)=(I-Kg(k)·H)·P(k∣k-1)
[0088] Among them, z is the actual measurement value; y is the difference between the actual measurement value and the predicted value; H is the measurement matrix, which is used to map the state vector to the measurement vector, and it describes the relationship between the predicted state and the actual observation value; R is the measurement noise covariance matrix, which is used to represent the uncertainty of the measurement and reflects the noise in the sensor readings; S is the covariance of the observation residual, which is used to represent the covariance matrix of the observation residual when there is observation error in the observation process; Kg is the Kalman gain, which is used to determine the weight distribution of new measurements and existing estimates, and it optimizes the update step to minimize the estimation error; x(k|k) and P(k|k) are the optimal estimates obtained by combining the predicted value and the observed value at state k.
[0089] Finally, the appearance depth features and speed direction information of the successfully matched trajectory are updated.
[0090] Second matching: According to the appearance feature similarity matrix, trajectory confidence similarity matrix, speed direction similarity matrix, and intersection-over-union matrix, the high-confidence detection boxes and unconfirmed state trajectories that were not successfully matched in the first matching are matched; the unconfirmed state trajectories that were not successfully matched are discarded; and the high-confidence detection boxes that were not successfully matched enter the fourth matching; the status of the successfully matched unconfirmed state trajectories is updated.
[0091] The specific process of updating the status in the second match includes:
[0092] First, the state of the target in the current video frame is predicted based on the state of the target in the previous video frame. Then, based on the actual state of the target in the current video frame, the margin and Kalman gain between the actual state and the predicted state are calculated, and the target trajectory state information and covariance matrix are updated. Finally, the appearance depth features and speed direction information of the successfully matched trajectory are updated.
[0093] Third matching: According to the appearance feature similarity matrix, trajectory confidence similarity matrix, and intersection-over-union matrix, the confirmed state trajectory that was not successfully matched in the first matching is matched with the low-confidence detection box; the confirmed state trajectory that was not successfully matched enters the fourth matching; the status of the successfully matched confirmed state trajectory is updated.
[0094] The specific process of updating the status in the third match includes:
[0095] First, the state of the target in the current video frame is predicted based on the state of the target in the previous video frame. Then, based on the actual state of the target in the current video frame, the margin and Kalman gain between the actual state and the predicted state are compared, and the trajectory state information and covariance matrix corresponding to the target are updated. Finally, the speed direction information of the successfully matched trajectory is updated, and the appearance depth features remain unchanged.
[0096] Fourth matching: determine the high-confidence detection frame that was not successfully matched in the second matching as an unmatched detection frame; determine the confirmation state trajectory that was not successfully matched in the third matching as an unmatched trajectory; construct a quadratic intersection-over-union matrix based on the target tracking frame of the unmatched trajectory in the previous video frame and the unmatched detection frame; match the unmatched detection frame and the unmatched trajectory based on the quadratic intersection-over-union matrix and the appearance feature similarity matrix; discard the unmatched trajectory that was not successfully matched; initialize the unmatched detection frame that was not successfully matched and enter the next video frame; update the status of the successfully matched unmatched trajectory, specifically, update the position status, velocity information and trajectory confidence information of the successfully matched trajectory, and keep the appearance depth feature unchanged.
[0097] The present invention divides the trajectories that need to be associated into two different trajectories, a confirmed state and an unconfirmed state, according to whether the match is successful, and divides the association process in the multi-target tracking method into four matches. First, a high-confidence detection frame and a confirmed target tracking frame are matched, then an unmatched high-confidence detection frame and an unconfirmed target tracking frame are matched, then an unmatched confirmed target tracking frame and a low-confidence detection frame are matched, and finally a high-confidence detection frame and a confirmed target tracking frame that have not been matched in the first three times are matched. This can reduce the problems of identity switching, false matching and target loss in multi-target tracking in the face of complex target motion patterns and crowded scenes, improve the tracking performance in complex scenes, and ensure the continuity and accuracy of tracking.
[0098] S7: Repeat the above steps S1-S6 for the video frame sequence to be processed to obtain a trajectory set including multiple target trajectories. The above steps S1-S6 are the process of performing multi-target matching on each video frame in the video frame sequence. Each video frame in the video frame sequence needs to go through steps S1-S6.
[0099] According to the matching results of each video frame with the previous video frame in the above steps S1-S6, the trajectories of multiple targets in the video frame sequence can be obtained. The trajectories of multiple targets are a trajectory set. When the multiple targets in all the video frames of the video frame sequence are matched, the multi-target tracking of the video frame sequence is ended.
[0100] Since no tracking trajectory is stored when the first video frame of the video frame sequence is processed, the first video frame only needs to go through steps S1 and S6, identify the high-confidence detection frame through the YOLOX detector, initialize it as a new tracking trajectory, and set the status of the tracking trajectory to an unconfirmed state; except for the first video frame, the remaining video frames of the video frame sequence need to go through the above steps S1-S7 to achieve multi-target tracking.
[0101] Next, the multi-target tracking method is further explained in combination with the specific implementation process, as follows:
[0102] See also Figure 2 As shown, the YOLOX detector is used to detect the target in the video frame t to obtain a high-confidence detection frame and a low-confidence detection frame, and the deep feature extractor is used to extract the appearance deep features of the high-confidence detection frame and the low-confidence detection frame.
[0103] Through camera motion compensation and Kalman filter, the confirmed state trajectory and unconfirmed state trajectory of the previous video frame Trackletst-1 are predicted. The predicted border of the confirmed state trajectory is the confirmed target tracking frame, and the predicted border of the unconfirmed state trajectory is the unconfirmed target tracking frame.
[0104] Based on the target bounding box intersection-over-union (IoU matrix), velocity direction (velocity direction similarity matrix), trajectory confidence (trajectory confidence similarity matrix) and ReID (appearance feature similarity matrix), the first match is performed between the high-confidence detection box and the predicted bounding box of the confirmed state trajectory (confirmed target tracking box).
[0105] Based on the intersection-and-union ratio of the target bounding box (intersection-and-union ratio matrix), speed direction (speed direction similarity matrix), trajectory confidence (trajectory confidence similarity matrix) and ReID (appearance feature similarity matrix), a second match is performed on the high-confidence detection boxes that were not matched in the first match and the predicted boxes of the unconfirmed state trajectories (unconfirmed target tracking boxes).
[0106] Based on the target bounding box intersection-over-union (IoU matrix), trajectory confidence (trajectory confidence similarity matrix) and ReID (appearance feature similarity matrix), a third match is performed on the predicted bounding box (confirmed target tracking box) of the confirmed state trajectory that was not matched in the first match and the low-confidence detection box.
[0107] Based on the target bounding box intersection-to-union ratio (quadratic intersection-to-union ratio matrix) and ReID (appearance feature similarity matrix), a fourth match is performed on the high-confidence detection box that was not matched in the second match and the predicted bounding box (confirmed target tracking box) of the confirmed state trajectory that was not matched in the third match.
[0108] The trajectories matched in the above four matches are managed, including updating the trajectory speed direction and appearance features through the Kalman filter; creating a new trajectory; deleting the lost trajectory; the trajectory after trajectory management is the trajectory of the current video frame Tracks t, and enters the matching process of the next video frame Trackletst.
[0109] The present invention also provides a computer device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement the above-mentioned platform door BIM model parametric driven design method.
[0110] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
[0111] The memory can be used to store the computer program or module, and the processor realizes various functions of the platform door BIM model parameter-driven design method by running or executing the computer program or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0112] The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims
1. A multi-target tracking method for complex scenes, characterized in that: include: The YOLOX detector is used to detect the target in the video frame to be processed, and the detection frame is divided into a high-confidence detection frame and a low-confidence detection frame according to the set confidence threshold; Divide the tracking trajectory of the previous video frame into states; Based on the state of the tracking trajectory, the target trajectory is divided into a confirmed state trajectory and an unconfirmed state trajectory; Predicting the target trajectory in the target tracking frame of the current video frame through a Kalman filter; Constructing a target matrix according to the high-confidence detection frame, the low-confidence detection frame, the confirmed state trajectory, the unconfirmed state trajectory, and the target tracking frame; Matching the target trajectory according to the target matrix; The above steps are repeated for the video frame sequence to be processed to obtain a trajectory set including multiple target trajectories.
2. The multi-target tracking method for complex scenes according to claim 1, characterized in that: The target tracking frame includes: The target tracking frame of the confirmed state trajectory is determined as a confirmed target tracking frame, and the target tracking frame of the unconfirmed state trajectory is determined as an unconfirmed target tracking frame.
3. The multi-target tracking method for complex scenes according to claim 2, characterized in that: The constructing a target matrix according to the high confidence detection frame, the low confidence detection frame, the confirmed state trajectory, the unconfirmed state trajectory, and the target tracking frame includes: Constructing an intersection-over-union matrix according to the high-confidence detection frame and the confirmed target tracking frame; Constructing an appearance depth feature similarity matrix according to the appearance depth features of the high confidence detection frame and the appearance depth features of the confirmed target tracking frame; Constructing a trajectory confidence similarity matrix according to the confidence of the detection frame and the confidence of the target tracking frame; A speed direction similarity matrix is constructed according to the speed direction of the detection frame and the speed direction of the target tracking frame.
4. The multi-target tracking method for complex scenes according to claim 3, characterized in that: The matching of the target trajectory according to the target matrix includes: The first matching is to match the high confidence detection frame with the confirmation state trajectory; A second matching process is to match the high confidence detection frame and the unconfirmed state trajectory that were not successfully matched in the first matching process; A third matching process is to match the confirmation state trajectory that was not successfully matched in the first matching process with the low confidence detection frame; The fourth matching is to match the high-confidence detection frame that was not successfully matched in the second matching and the confirmation state track that was not successfully matched in the third matching; Update the status of the successfully matched trajectory.
5. The method for tracking multiple targets in complex scenes according to claim 4, characterized in that: The first matching includes: According to the appearance feature similarity matrix, the trajectory confidence similarity matrix, the speed direction similarity matrix, and the intersection-over-union matrix, the high confidence detection frame and the confirmation state trajectory are matched by the Hungarian algorithm; Determine whether an intersection-over-union ratio of the high-confidence detection frame and the confirmed target tracking frame is greater than a threshold; If the intersection-and-union ratio is greater than the threshold, the match is successful, and the status of the confirmed status track of the successful match is updated; If the intersection-over-union ratio is less than the threshold, it means that the match is not successful; The high confidence detection frame that has not been successfully matched performs the second matching; The confirmation state track that has not been successfully matched is matched for the third time.
6. The method for tracking multiple targets in complex scenes according to claim 5, characterized in that: The second matching includes: Matching the high-confidence detection frame and the unconfirmed state trajectory that were not successfully matched in the first matching according to the appearance feature similarity matrix, the trajectory confidence similarity matrix, the speed direction similarity matrix, and the intersection-over-union matrix; Discarding the unconfirmed state track that has not been successfully matched; The high confidence detection frame that has not been successfully matched performs the fourth matching; The state of the unconfirmed state track that is successfully matched is updated.
7. The method for tracking multiple targets in complex scenes according to claim 6, characterized in that: The third matching includes: Matching the confirmed state trajectory that was not successfully matched in the first matching with the low confidence detection frame according to the appearance feature similarity matrix, the trajectory confidence similarity matrix, and the intersection-over-union matrix; The confirmation state track that has not been successfully matched performs the fourth matching; The status of the successfully matched confirmation status track is updated.
8. The method for tracking multiple targets in complex scenes according to claim 7, characterized in that: The fourth matching includes: Determine the high-confidence detection frame that is not successfully matched in the second matching as an unmatched detection frame; Determining the confirmed state trajectory that is not successfully matched in the third matching as an unmatched trajectory; Constructing a quadratic intersection-over-union matrix according to the target tracking frame of the unmatched trajectory in the previous video frame and the unmatched detection frame; Matching the unmatched detection frame and the unmatched trajectory according to the quadratic intersection-over-union matrix and the appearance feature similarity matrix; Discarding the unmatched track that has not been successfully matched; Initialize the unmatched detection frame that has not been successfully matched and enter the next video frame; The status of the unmatched track that is successfully matched is updated.
9. The method for tracking multiple targets in complex scenes according to claim 1, characterized in that: The predicting the target tracking frame of the target trajectory in the current video frame by using a Kalman filter includes: Predicting the target tracking frame of the target trajectory in the current video frame by using the Kalman filter; Selecting feature points in the previous video frame and the current video frame; Comparing the difference between the feature points in the previous video frame and the current video frame to determine a rotation matrix; The target tracking frame is corrected according to the rotation matrix.
10. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement the multi-target tracking method as described in any one of claims 1 to 9.
Citation Information
Cited By
Boat-borne infrared vision sea surface weak target identification and tracking method
CN119229145A
Multi-target tracking method based on trajectory guidance
CN119722741A
Non-motor vehicle illegal parking determination method and device for smart park, and medium
CN121527724A
