Improved target tracking method, system, device and medium
By improving the multi-target tracking algorithm STYOLOTRACK, and combining it with the Kalman filter and the target re-identification model OSNet, the target tracking process is optimized, which solves the problem of poor tracking performance in occlusion and complex backgrounds, and achieves higher tracking accuracy and stability.
Patent Information
- Application Number
- CN202511321391.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-16
- Publication Date
- 2025-12-19
AI Technical Summary
Existing target tracking technologies perform poorly in situations involving occlusion, scale changes, and complex backgrounds, resulting in unsatisfactory tracking results.
An improved multi-target tracking algorithm, STYOLOTRACK, is adopted, which combines Kalman filter, Hungarian algorithm and target re-identification model OSNet. The target tracking is optimized through a two-matching strategy, and the tracking robustness in occluded scenes is improved by using different processing methods for high-resolution and low-resolution bounding boxes.
It significantly improves the accuracy and stability of target tracking, reduces the number of ID switching times, enhances the overall performance of multi-target tracking, and meets real-time requirements.
Smart Images
Figure CN121169969A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of target tracking, in particular to an improved target tracking method, system, device and medium. BACKGROUND
[0002] The target tracking field refers to a technology of identifying a target entering a specific area and tracking a trajectory of the target in a certain monitoring environment. At present, the target tracking field still faces the problem of poor tracking effect caused by occlusion, scale change, nonlinear movement of the target and complex background, so that the effect of the target tracking algorithm is still not ideal.
[0003] Therefore, how to enhance the tracking effect of target tracking is an important problem to be solved. SUMMARY
[0004] The embodiments of the present application provide an improved target tracking method, system, device and medium, which can solve the problem of poor tracking effect of target tracking in the prior art.
[0005] The embodiments of the present application provide an improved target tracking method, which comprises the following steps: obtaining a video frame sequence and a detection box from video image data of a target to be tracked; inputting the video frame sequence and the detection box into a multi-target tracking algorithm BYTETRACK to obtain an intersection over union IoU of the detection box and a predicted trajectory of a next frame predicted by a Kalman filter; extracting an appearance feature of the detection box, and adding a cosine distance of the appearance feature to the IoU to obtain a comprehensive similarity measure; using the Hungarian algorithm, performing first matching on the detection box and the predicted trajectory using the comprehensive similarity measure, and updating the Kalman filter using the detection box matched successfully in the first matching; obtaining a comprehensive similarity measure of an unmatched detection box, and performing second matching on the unmatched detection box and an unmatched predicted trajectory, and updating the Kalman filter using the detection box matched successfully in the second matching; taking the detection box matched successfully as a target trajectory of each frame, until the video frame sequence is processed, and the tracking of the target is completed.
[0006] Further, the obtaining of the video frame sequence and the detection box from the video image data of the target to be tracked specifically comprises: obtaining the video frame sequence and the detection box of the video image data of the target to be tracked by YOLOv5 or YOLOv8.
[0007] Further, the detection box is divided into a high-score box and a low-score box according to a preset confidence; The first matching of the multi-target tracking algorithm BYTETRACK matches the high-scored frame with the predicted track using a comprehensive similarity measure; and the second matching of the multi-target tracking algorithm BYTETRACK matches the low-scored frame with the predicted track that is not matched successfully in the first matching.
[0008] Further, the appearance feature of the detection frame is extracted using a feature recognition model OSNet.
[0009] The embodiment of the present application provides an improved target tracking system, which comprises: A target acquisition module is configured to acquire a video frame sequence and a detection frame from video image data of a target to be tracked. A target tracking module is configured to input the video frame sequence and the detection frame into a multi-target tracking algorithm BYTETRACK, acquire an intersection over union IoU of the detection frame and a predicted track of a next frame predicted by a Kalman filter, extract an appearance feature of the detection frame, add a cosine distance of the appearance feature to the IoU to obtain a comprehensive similarity measure, perform a first matching of the detection frame and the predicted track using the comprehensive similarity measure by using a Hungarian algorithm, update the Kalman filter using the detection frame matched successfully in the first matching, acquire the comprehensive similarity measure of the detection frame not matched, perform a second matching of the detection frame not matched and the predicted track not matched successfully, update the Kalman filter using the detection frame matched successfully in the second matching, and take the detection frame matched successfully as a target track of each frame until the video frame sequence is processed completely, and the tracking of the target is completed.
[0010] The embodiment of the present application provides a computer device, which comprises a memory and a processor; the memory stores a computer program, and the processor realizes the improved target tracking method described above when executing the computer program.
[0011] The embodiment of the present application provides a computer readable storage medium, which stores a computer program; the computer program is executed by a processor to realize the improved target tracking method described above.
[0012] The embodiment of the present application provides an improved target tracking method, system, device and medium, which has the following beneficial effects compared with the prior art: Input the video frame sequence and the detection box into the multi-target tracking algorithm BYTETRACK to obtain the intersection over union IoU of the detection box and the predicted trajectory of the next frame predicted by the Kalman filter; extract the appearance feature of the detection box, and add the cosine distance of the appearance feature to the IoU to obtain a comprehensive similarity measure; use the Hungarian algorithm to perform two matching of the detection box and the Kalman filter predicted trajectory using the comprehensive similarity measure until the video frame sequence processing is completed, and the tracking of the target is completed; the matching process of the multi-target tracking algorithm BYTETRACK is optimized by the comprehensive similarity measure, so that the matching result is more accurate, and the tracking effect of target tracking is enhanced. BRIEF DESCRIPTION OF DRAWINGS
[0013] Figure 1 A DEEPSORT tracking flowchart provided for the embodiment of the present application is provided. Figure 2 A BYTETRACK tracking flowchart provided for the embodiment of the present application is provided. Figure 3 An OS module structure diagram provided for the embodiment of the present application is provided. Figure 4 A flowchart of an improved target tracking method provided for the embodiment of the present application is provided. DETAILED DESCRIPTION
[0014] In order to make the above objectives, features and advantages of the present application more apparent, specific embodiments of the present application will be described in detail below with reference to the accompanying drawings. In the following description, a large number of specific details are set forth in order to provide a sufficient understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the scope of the present application, so the present application is not limited to the specific embodiments disclosed below.
[0015] Referring to Figure 4 The embodiment of the present application provides an improved target tracking method, which comprises the following steps: Step one: obtaining a video frame sequence and a detection box from video image data of a target to be tracked.
[0016] Step two: input the video frame sequence and the detection box into the multi-target tracking algorithm BYTETRACK to obtain the intersection over union IoU of the detection box and the predicted trajectory of the next frame predicted by the Kalman filter; extract the appearance features of the detection box, and add the cosine distance of the appearance features to the IoU to obtain the comprehensive similarity measure; use the Hungarian algorithm to perform the first matching of the detection box and the predicted trajectory using the comprehensive similarity measure, and update the Kalman filter using the detection box that is successfully matched in the first matching; obtain the comprehensive similarity measure of the unmatched detection box, and perform the second matching of the unmatched detection box and the predicted trajectory that is not successfully matched, and update the Kalman filter using the detection box that is successfully matched in the second matching; use the detection box that is successfully matched as the target trajectory of each frame, until the video frame sequence processing is completed, and the tracking of the target is completed.
[0017] 1. Multi-target tracking algorithm DEEPSORT.
[0018] DEEPSORT is a multi-target tracking algorithm based on deep learning, which combines target detection and is an improvement of the SORT algorithm. DEEPSORT realizes more accurate and robust target tracking by combining target detection, Kalman filter, Hungarian algorithm and deep feature embedding. Its tracking process is as shown in Figure 1
[0019] DEEPSORT also employs a Kalman filter to model and predict the position and velocity of targets, and a Hungarian algorithm to determine the best match between each detected target and existing trackers. However, compared to the simple online real-time tracking algorithm SORT, DEEPSORT adds a feature extraction step that converts each detected target into a feature embedding vector. These feature embedding vectors capture semantic and appearance information of the target, with good discriminative ability. Typical feature extraction models include Siamese networks based on convolutional neural networks and ReID (Person Re-identification) models. These models learn a mapping function that maps targets to a feature space through large-scale labeled data training, thereby achieving feature representation of targets. Similarly, because DEEPSORT adds feature extraction, the tracker's update is also improved, using a Kalman filter to predict the target's historical state and motion model, estimating the target's position and velocity at the next time step. Next, the detected target in the current frame is associated with the existing tracker. This step is achieved by calculating the similarity or distance between targets and using the feature embedding vectors obtained by the ReID model. The Hungarian algorithm (or other related target association algorithms) finds the best target matching combination according to the similarity or distance matrix. For the associated target, the Kalman filter uses the observation data (detection results) to correct the predicted state. By fusing the detection results with the predicted state, the Kalman filter can more accurately estimate the target's position and velocity, and update the tracker's state.
[0020] Through such an update process, DEEPSORT can achieve continuous tracking and state updating of multiple targets. The Kalman filter provides smooth and predictive capabilities for target trajectories, while the ReID model provides more accurate similarity measures for target association, thereby improving the accuracy and stability of tracking. This integrated application enables DEEPSORT to perform well in complex target tracking tasks.
[0021] 2. BYTETRACK algorithm.
[0022] The BYTETRACK algorithm is also an improvement of the SORT algorithm, which focuses on using the bounding box information obtained in target tracking for tracking. Similar to other non-ReID methods, the ByteTrack algorithm only uses the bounding box obtained by target tracking for tracking. The algorithm uses Kalman filtering for bounding box prediction and uses the Hungarian algorithm for matching between targets and trajectories. The main innovation of the BYTETRACK algorithm is its use of low-confidence bounding boxes. The core idea of BYTETRACK is that low-confidence bounding boxes may be due to target occlusion, and directly discarding these low-confidence bounding boxes may affect tracking performance. Therefore, the low-confidence bounding boxes are matched again in the tracking algorithm to effectively optimize the target ID switching problem caused by occlusion.
[0023] The implementation process of the BYTETRACK algorithm mainly includes six steps. Compared with DEEPSORT, it adds a step of low-score box secondary matching, but does not calculate the appearance similarity based on the ReID feature. The implementation process is as follows: first, classify the tracking trajectory and the current frame bounding box. It is divided into active and inactive states, and the current frame bounding box is divided into high-score and low-score two categories according to the score threshold. Second, the first tracking is performed. The high-score bounding box of the active state trajectory is matched. First, the tracked and lost trajectories are merged to form a preliminary tracking trajectory. The Kalman filter is used to predict the position and size of the next frame bounding box of the preliminary tracking trajectory. Then, the IoU (intersection over union) value between the next frame bounding box of the preliminary tracking trajectory and the current frame high-score bounding box is calculated to obtain the IoU relationship loss matrix. The Hungarian algorithm is used to match the preliminary tracking trajectory and the current frame high-score bounding box to obtain the matched trajectory and bounding box, and the unmatched trajectory and bounding box. Finally, the current frame bounding box that has been successfully matched is used to update the preliminary tracking trajectory. Third, the second tracking is performed. The low-score bounding box of the active state trajectory is matched. Similarly, the trajectories that are not successfully matched in the first matching are selected as the to-be-tracked trajectories. The IoU between the to-be-tracked trajectory and the current frame low-score bounding box is calculated. Then, the Hungarian algorithm is used to match the to-be-tracked trajectory and the current frame low-score bounding box to obtain the matched trajectory and bounding box. Finally, the current frame bounding box that has been successfully matched is used to update the to-be-tracked trajectory. Fourth, the trajectories in the inactive state are tracked. The high-score bounding box that is not successfully matched in the first step and the inactive trajectory are found. The IoU between the inactive trajectory and the current frame bounding box is calculated. Then, the Hungarian algorithm is used to match the inactive trajectory and the bounding box. The current frame bounding box that has been successfully matched is used to update the inactive trajectory, and the trajectory that is not successfully tracked is marked as a deleted trajectory. Fifth, new trajectories are created. If there are still high-score bounding boxes that are not successfully matched, they are considered to be new targets, and new trajectories and unique IDs are assigned to them. Sixth, the results are returned. All tracked trajectories are returned, each with a unique ID, as the result of each frame tracking. The tracking process is as shown in Figure 2
[0024] 3. STYOLOTRACK algorithm improved based on target re-identification model.
[0025] Object Re-Identification Model is an important model in computer vision, used to identify and match the same target in different scenes or different time images or videos. It is mainly applied in the field of target tracking. In the task of target tracking, the Object Re-Identification Model can identify and associate the appearance changes of the same target under different frames or different cameras. It connects the appearance of the target in different scenes or times by learning the visual features of the target, so as to realize the continuous tracking and identification of the target. The Object Re-Identification Model not only can associate the same target in the tracking process, but also can perform target re-identification in different images or videos. By learning the feature representation of the target, it can match the target with the previously observed target when given an image or video frame, so as to identify the identity of the target. The Object Re-Identification Model first uses deep learning methods to extract the visual features of the target, usually using convolutional neural network (CNN) architecture to learn the compact feature representation of the target. Then, by calculating the similarity or distance between features, the target is matched and associated. The Object Re-Identification Model can also match across time and across cameras. In target tracking, the Object Re-Identification Model can determine whether the features of the target at different times or different cameras belong to the same target by comparing the feature representations of the target. By establishing a feature library of the target and using matching algorithms such as nearest neighbor matching, graph-based matching, etc., the model can realize the continuous tracking and identity association of the target in different scenes and times. Over time, the appearance of the target may change, such as occlusion, pose change, etc. The Object Re-Identification Model can gradually update the feature representation of the target through online learning or incremental learning, to adapt to the changes in the appearance of the target and improve the accuracy of re-identification.
[0026] OSNet (Omni-Scale Feature Learning for Person Re-Identification) is a lightweight network model proposed for the Re-Identification (ReID) problem. It aims to capture features at different spatial scales (isotropic and anisotropic) and encapsulate arbitrary feature combinations at multiple scales. Through the residual connection block composed of multiple convolutional feature streams, full-scale feature learning is achieved, and a unified aggregation gate AG (aggregation gate) is introduced to generate channel weights for dynamic multi-scale feature fusion, thereby generating full-scale features. The structure of the OS module is as shown in Figure 3 .
[0027] The core of the OSNet is a full-scale feature learning module composed of multiple convolutional feature streams with different receptive fields, which ensures multi-scale feature acquisition. Each convolutional layer adopts a depth separable convolution, which separates the original convolution kernel into a depth convolution kernel and a point convolution kernel, thereby reducing the amount of calculation and the number of parameters. In addition, the stacked convolution structure makes the OSNet very lightweight. Different scale features are combined together and input into the unified aggregation gate AG. By utilizing the shared and learnable characteristics of AG, different scale features are given different weights, thereby realizing dynamic combination of different scale feature streams. Finally, the aggregated features are combined with the original input features to obtain full-scale feature representation.
[0028] The present application is inspired by DEEPSORT, and adds a re-identification model to BYTETRACK to improve tracking ability and reduce the number of ID switches. However, only a simple CNN network is used in the original DEEPSORT to extract the appearance features of the target, which is not sufficient in terms of performance. In subsequent research, some researchers have applied resnet50 to DEEPSORT to extract appearance features, which has improved the tracking effect compared to the original DEEPSORT, but it is still not ideal. Therefore, the present application combines the more advanced OSNet model with BYTETRACK in the appearance feature extraction module to achieve better tracking effect. The parameter amount of the OSNet model is only 2.2 million, which is much less than the 24 million parameter amount of resnet50, and its accuracy is higher, with a multi-branch structure, and different receptive fields under each branch can adapt to different scenes.
[0029] The specific improvements of the present application are as follows: Osnet is used as a target re-identification model to identify appearance features and is placed in the cascade matching part of bytetrack for cascade matching, and then the subsequent iou motion feature matching is performed.
[0030] BYTETRACK is a detection-based multi-target tracking algorithm, and the core idea is to improve the tracking robustness in occlusion scenes through two matching strategies (high-score frame priority matching and low-score frame secondary matching). Further introduce the OSNet target re-identification model to fuse appearance features to reduce ID switching, forming the improved algorithm STYOLOTRACK-v5 or STYOLOTRACK-v8. The following are the specific execution steps:
[0031] 1. Input and preprocessing: Input: video frame sequence and corresponding detection frame (generated by YOLOv5 / YOLOv8 or improved detection algorithm CBAM_YOLOv5 / BiFPN_YOLOv8).
[0032] Preprocessing: Divide the detection boxes into high-score boxes (confidence ≥ threshold, such as 0.5) and low-score boxes (confidence < threshold) according to confidence.
[0033] Initialize tracker state, including Kalman filter prediction trajectory, history trajectory activation state (activated / deactivated).
[0034] 2. First matching: high-score boxes match with activated trajectories: Objective: Prioritize processing high-confidence detection boxes and match activated target trajectories.
[0035] Step: Trajectory prediction: Use Kalman filter to predict the next frame position for all activated trajectories.
[0036] Similarity calculation: Calculate the IoU (intersection over union) between the predicted trajectory and the high-score box to generate a cost matrix.
[0037] Improvement point: If using STYOLOTRACK-v5 or STYOLOTRACK-v8, extract the appearance features of the detection box (through the OSNet model) and calculate the cosine distance of the features, combined with the IoU to form a comprehensive similarity measure.
[0038] Hungarian algorithm matching: Solve the optimal matching through the Hungarian algorithm to assign high-score boxes to activated trajectories.
[0039] Trajectory update: Matched trajectories: Update the Kalman filter state with the detection box.
[0040] Unmatched trajectories: Marked as "deactivated" and enter the secondary matching pool.
[0041] Unmatched high-score boxes: Temporarily stored, waiting for secondary matching.
[0042] 3. Second matching: low-score boxes match with deactivated trajectories: Objective: Use low-score boxes to recover lost trajectories due to occlusion and reduce ID switching.
[0043] Step: Select deactivated trajectories: Collect deactivated trajectories that were not successfully matched in the first matching.
[0044] Similarity calculation: Calculate the IoU between the deactivated trajectory prediction box and the low-score box (only use spatial position information, as the BYTETRACK original algorithm does not introduce appearance features, while STYOLOTRACK-v5 or STYOLOTRACK-v8 will supplement appearance feature matching at this step).
[0045] Hungarian algorithm matching: Assign low-score boxes to deactivated trajectories to recover occluded targets.
[0046] Track activation: matched tracks are reactivated, unmatched tracks continue to be deactivated (if they are not matched for a long time, they are deleted).
[0047] 4. New track and unactivated track processing: New track: for high-score boxes that have not been assigned after two matches, create a new track and assign a unique ID.
[0048] Unactivated track processing: for unactivated tracks that do not participate in matching (such as targets that have not appeared for a long time), calculate their IoU with the remaining high-score boxes and try to reactivate.
[0049] If still not matched, mark as "deleted" and remove from the track list.
[0050] 5. Output and post-processing: Output results: each frame of tracking results contains target ID, bounding box coordinates, track confidence, etc.
[0051] Post-processing: use non-maximum suppression (NMS) to remove redundant detection boxes.
[0052] Improvements: STYOLOTRACK-v5 or STYOLOTRACK-v8 use the appearance features extracted by OSNet to further optimize cross-frame track association and reduce ID jumps caused by occlusion or similar targets.
[0053] The improvement point of the tracking part is to add an osnet appearance feature matching. The original bytetrack does not have appearance feature matching.
[0054] 4. Experiment and analysis.
[0055] 4.1 Experimental evaluation index.
[0056] There are many evaluation indexes for target tracking, and the present invention selects five commonly used evaluation indexes to evaluate the improved tracking performance in the present invention.
[0057] ID-switch: ID-switch evaluates the correctness of target ID transfer and update in multi-target tracking technology. In a tracking track, the smaller the ID-switch index, the fewer the number of changes of the same target label, and the better the tracking performance.
[0058] Multiple object tracking accuracy (MOTA): MOTA is an evaluation metric that measures the accuracy of multi-object tracking. It takes into account factors such as missed detections, false positives, and ID confusion, and only considers the matching of all targets, regardless of detection accuracy, to objectively evaluate the performance of a target tracking algorithm. The value of MOTA ranges from [-∞, 1], with a larger value indicating better algorithm performance, and a value of 1 indicating perfect matching. Generally, when MOTA is greater than 0.5, the target tracking algorithm performs well, and when MOTA is less than 0.3, the target tracking algorithm performs poorly. The calculation formula of MOTA is:
[0059] .
[0060] where, is the number of false positive occurrences, is the number of false negative occurrences, is the number of ID-switch occurrences, is the number of labeled targets.
[0061] Multiple object tracking precision (MOTP): MOTP is an evaluation metric that measures the precision of multi-object tracking. It is mainly used to evaluate the accuracy of target position estimation in the target tracking algorithm, that is, the distance error between the tracked target position and the true position. The value of MOTP ranges from [0, ∞), with a smaller value indicating larger tracking error and a larger value indicating smaller tracking error, and a value of 0 indicating no correct matching. The calculation formula of MOTP is:
[0062] .
[0063] where, is the number of successfully matched targets in the current frame, is the intersection over union of the true box and the detection box, is the current detection target.
[0064] IDF1 (ID F1-score): IDF1, relative to other evaluation metrics, places more emphasis on evaluating the accuracy and stability of the trajectory, and its calculation formula is: .
[0065] where, is the number of id true positive occurrences, is the number of id false positive occurrences, is the number of id false negative occurrences.
[0066] Image processing frames per second (FPS): The more frames processed per second, the faster and more efficient the algorithm is.
[0067] 4.2 MOT17 dataset.
[0068] The MOT17 dataset was released in 2017 to promote research in the field of multi-object tracking, particularly pedestrian tracking. By providing video sequences in real-world scenarios, the dataset aims to evaluate the performance of tracking algorithms, promote algorithm development, and provide multiple effective evaluation benchmarks to compare algorithm performance.
[0069] The MOT17 dataset focuses primarily on pedestrians, but the categories cover seven classes: pedestrians, occlusions between pedestrians and people, crowds, vehicles, occlusions between vehicles, bicycles, and an unknown category. The dataset contains video sequences from seven different scenes, each providing training and testing segments.
[0070] The MOT17 dataset presents multiple challenges to target tracking algorithms, such as occlusions between targets, scale variations, and fast motion. This motivates researchers to develop new tracking algorithms that address these challenges and improve the robustness and accuracy of algorithms in real-world scenarios.
[0071] The proposed tracking algorithm will be tested on the MOT17 dataset to demonstrate its sufficient accuracy, robustness, and generalization performance. By conducting experiments on this challenging dataset, the algorithm will be verified for its performance in various complex scenarios.
[0072] 4.3 Comparison of experimental parameters.
[0073] To further explore the performance of the proposed tracking algorithm, a series of comparative experiments were conducted, in which the model size and related parameters of the tracking-before-detection algorithm are as follows, to avoid the influence of irrelevant variables on the experimental results: The platform is based on Ubuntu 18.04.6 LTS operating system, the processor uses an x64 architecture Intel(R) Xeon(R) Gold 6348 model CPU, and the graphics card uses a professional graphics card NVIDIA A40 GPU supporting CUDA driver. The core frequency of the above processor and graphics card is the default frequency to prevent performance improvement due to overclocking and avoid irrelevant variables in the experiment. In terms of running memory and video memory, Samsung 3200 MHZ DDR4 ECC 32G*8 memory is used, totaling 256G, and 48G GDDR6 video memory is used. Similarly, no overclocking is set to ensure normal operation of the experiment. In terms of environment version, the PyCharm integrated compilation environment is used, version 2021.2, and the CUDA version is 12.0. The deep learning framework used is PyTorch 1.14.0. The subsequent experimental software and hardware environment is the same as this, and will not be described in detail. The experimental software and hardware environment is shown in Table 1.
[0074] Table 1 Experimental software and hardware environment.
[0075] The maximum target tracking reserved frame number of BYTETRACK is still set to 30 frames. The DEEPSORT tracking algorithm is added as a performance benchmark for comparison. The target tracking data set for training and testing in this experiment is MOT17, and the comparison experiment results are shown in Table 2.
[0076] Table 2 Comparison experiment results of different tracking models on MOT17 data set.
[0077] 4.4 Experimental results and analysis.
[0078] STYOLOTRACK-v5 represents YOLOv5+BYTETRACK+target re-identification model OSNet; STYOLOTRACK-v8 represents YOLOv8+BYTETRACK+target re-identification model OSNet.
[0079] The experimental results show that the STYOLOTRACK-v5 and the STYOLOTRACK-v8 proposed in the application have greater improvement compared with DEEPSORT and BYTETRACK, in terms of IDF1, the IDF1 of the STYOLOTRACK-v5 is 68.9%, the IDF1 of the STYOLOTRACK-v8 is 69.7%, which is improved by 5.1 percentage points and 5.9 percentage points respectively compared with YOLOv5+DEEPSORT, and is improved by 1 percentage point and 1.8 percentage points respectively compared with YOLOv8+BYTETRACK, and the STYOLOTRACK-v5 and the STYOLOTRACK-v8 have obvious improvement effect on MOTA and IDs. The improvement of MOTA verifies the improvement of the tracking comprehensive performance, the reduction of IDs proves that the problem of tracking wrong targets after short-time target occlusion and disappearance is improved, especially the IDs, the improved STYOLOTRACK-v5 and the STYOLOTRACK-v8 greatly reduce the ID switching phenomenon, which is reduced by 35.0% and 39.2% respectively compared with YOLOv5+DEEPSORT, and is reduced by 22.8% and 27.8% respectively compared with YOLOv8+BYTETRACK, so that the tracking effect is more robust. The tracking rate of the STYOLOTRACK-v5 and the STYOLOTRACK-v8 still meets the real-time requirement, and is basically equivalent to YOLOv5+DEEPSORT+OSNet, the STYOLOTRACK-v5 is slightly faster than the STYOLOTRACK-v8, so in the selection of the subsequent tracking model, the actual demand can be selected, for example, if the tracking performance is higher, the STYOLOTRACK-v8 can be selected, and if the tracking speed is higher, the STYOLOTRACK-v5 can be selected.
[0080] 5. Summary.
[0081] The STYOLOTRACK-v5 and the STYOLOTRACK-v8 are proposed based on the BYTETRACK algorithm. The STYOLOTRACK combines the OSNet target re-identification model, adds the extraction of the target appearance feature, and combines the original target motion feature extraction of the BYTETRACK to make the target feature more rich and stereoscopic.
[0082] The STYOLOTRACK-v5 is CBAM_YOLOv5+improved BYTETRACK, and the STYOLOTRACK-v8 is BiFPN_YOLOv8+improved BYTETRACK.
[0083] YoloV5 adds CBAM attention mechanism in the detection part, and combines the improved BYTETRACK tracking algorithm in the tracking part, and the improved BYTETRACK tracking algorithm is formed by combining the OSNet target re-identification model in the original BYTETRACK tracking algorithm. YoloV8 adds BiFPN attention mechanism in the detection part, and combines the improved BYTETRACK tracking algorithm in the tracking part, and the improved BYTETRACK tracking algorithm is formed by combining the OSNet target re-identification model in the original BYTETRACK tracking algorithm.
[0084] During the experiment, the related experimental parameters are determined to ensure the rationality and accuracy of the experiment, and the comparative experiment is carried out. The IDF1 index of STYOLOTRACK-v5 and STYOLOTRACK-v8 reaches 68.9% and 69.7% on the MOT dataset, the MOTA index reaches 65.5% and 67.2%, and the IDs index is reduced to 743 times and 695 times.
[0085] The application is based on the BYTETRACK target tracking algorithm, and the STYOLOTRACK-v5 and STYOLOTRACK-v8 target tracking algorithms are proposed by adding an advanced target re-identification module to cope with the many challenges in the target tracking field. In order to verify the effectiveness and generalization ability of the target tracking algorithm proposed in the application, the MOT17 public data set is used for training and comparison test with the original algorithm to verify the actual effect. The application also adds the experimental results of the DEEPSORT target tracking algorithm in the experiment to assist in explaining the superiority of the tracking algorithm of the application.
[0086] The embodiment of the application provides an improved target tracking system, which comprises: A target acquisition module is configured to acquire a video frame sequence and a detection box from video image data of a target to be tracked.
[0087] A target tracking module is configured to input the video frame sequence and the detection box into a multi-target tracking algorithm BYTETRACK, acquire an intersection over union IoU of the detection box and a predicted trajectory of a next frame predicted by a Kalman filter, extract an appearance feature of the detection box, and add a cosine distance of the appearance feature to the IoU to obtain a comprehensive similarity measure; use the comprehensive similarity measure to perform a first matching on the detection box and the predicted trajectory by using a Hungarian algorithm, and update the Kalman filter by using a detection box matched successfully in the first matching; acquire a comprehensive similarity measure of an unmatched detection box, perform a second matching on the unmatched detection box and an unmatched predicted trajectory, and update the Kalman filter by using a detection box matched successfully in the second matching; and use a detection box matched successfully as a target trajectory of each frame until the video frame sequence is processed completely, and complete tracking of the target.
[0088] The embodiment of the present application provides a computer device, comprising a memory and a processor; the memory stores a computer program, and the processor implements steps of an improved target tracking method when executing the computer program.
[0089] The embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program implements steps of an improved target tracking method when executed by a processor.
[0090] One specific implementation is as follows: The embodiment discloses an improved target tracking method, and specific steps are as follows: S1. Obtain a video frame sequence and a detection box from video image data of a target to be tracked.
[0091] S2. Input the video frame sequence and the detection box into a multi-target tracking algorithm BYTETRACK to obtain an intersection over union IoU of the detection box and a predicted track of a next frame predicted by a Kalman filter; extract an appearance feature of the detection box, and add a cosine distance of the appearance feature to the IoU to obtain a comprehensive similarity measure; use the comprehensive similarity measure to perform first matching on the detection box and the predicted track by using a Hungarian algorithm, and update the Kalman filter by using a detection box matched successfully in the first matching; obtain a comprehensive similarity measure of an unmatched detection box, and perform second matching on the unmatched detection box and an unmatched predicted track matched unsuccessfully in the first matching, and update the Kalman filter by using a detection box matched successfully in the second matching; take the detection box matched successfully as a target track of each frame, until the video frame sequence is processed completely, and the tracking on the target is completed. The detection box is divided into a high-score box and a low-score box according to a preset confidence; the first matching of the multi-target tracking algorithm BYTETRACK is performed on the high-score box and the predicted track by using the comprehensive similarity measure; and the second matching of the multi-target tracking algorithm BYTETRACK is performed on the low-score box and the predicted track unmatched successfully in the first matching.
[0092] The above-described embodiments only express several implementation manners of the present application, which are described in detail and specifically, but should not be understood as limitation on the patent scope of the present application. It should be noted that, for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the patent protection scope of the present application should be subject to the appended claims.
Claims
1. An improved target tracking method, characterized in that, Includes the following steps: Obtain video frame sequences and detection boxes from the video image data of the target to be tracked; The video frame sequence and detection boxes are input into the multi-target tracking algorithm BYTETRACK to obtain the intersection-over-union ratio (IoU) between the detection boxes and the predicted trajectory of the next frame predicted by the Kalman filter. The appearance features of the detection boxes are extracted, and the cosine distance of the appearance features is added to the IoU to obtain a comprehensive similarity metric. The Hungarian algorithm is used to perform the first matching between the detection boxes and the predicted trajectory using the comprehensive similarity metric. The Kalman filter is updated using the detection boxes that are successfully matched in the first match. Obtain the comprehensive similarity metric of the unmatched detection boxes, perform a second match between the unmatched detection boxes and the unsuccessfully matched predicted trajectories, and update the Kalman filter using the second successfully matched detection boxes; use the successfully matched detection boxes as the target trajectory for each frame until the video frame sequence is processed, thus completing the target tracking.
2. The improved target tracking method as described in claim 1, characterized in that, The step of obtaining video frame sequences and detection boxes from the video image data of the target to be tracked specifically includes: The video frame sequence and detection box of the target to be tracked are obtained using YOLOv5 or YOLOv8.
3. The improved target tracking method as described in claim 1, characterized in that, The detection boxes are divided into high-scoring boxes and low-scoring boxes according to a preset confidence level; The first matching step of the BYTETRACK multi-object tracking algorithm matches the high-scoring bounding box with the predicted trajectory using a comprehensive similarity metric; the second matching step of the BYTETRACK multi-object tracking algorithm matches the low-scoring bounding box with the predicted trajectory that failed to match in the first matching step.
4. An improved target tracking method as described in claim 1, characterized in that, The appearance features of the detection box are extracted using the OSNet feature recognition model.
5. An improved target tracking system, characterized in that, include: The target acquisition module is used to acquire video frame sequences and detection boxes from the video image data of the target to be tracked; The target tracking module is used to input video frame sequences and detection boxes into the multi-target tracking algorithm BYTETRACK, obtain the intersection-over-union (IoU) between the detection boxes and the predicted trajectory of the next frame predicted by the Kalman filter; extract the appearance features of the detection boxes, and add the cosine distance of the appearance features to the IoU to obtain a comprehensive similarity metric; use the Hungarian algorithm to perform the first matching between the detection boxes and the predicted trajectory using the comprehensive similarity metric, and update the Kalman filter using the detection boxes that successfully match in the first match; Obtain the comprehensive similarity metric of the unmatched detection boxes, perform a second match between the unmatched detection boxes and the unsuccessfully matched predicted trajectories, and update the Kalman filter using the second successfully matched detection boxes; use the successfully matched detection boxes as the target trajectory for each frame until the video frame sequence is processed, thus completing the target tracking.
6. A computer device, comprising: Memory and processor; The memory stores a computer program, characterized in that the processor executes the computer program to implement an improved target tracking method according to any one of claims 1 to 4.
7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements an improved target tracking method as described in any one of claims 1 to 4.