DeepSort-based multi-target pedestrian tracking algorithm

Abstract

The invention belongs to the field of target tracking, and particularly relates to a DeepSort-based multi-target pedestrian tracking method. According to the invention, the real-time online video is obtained and is preprocessed; according to the invention, a tracking algorithm of DeepSort is optimized on the basis of an improved YOLOV5 detector. Firstly, a YOLOV5 detector is improved, a triple attention mechanism is added to the YOLOV5 detector, the detection precision is improved, follow-up tracking is facilitated, and under the detection condition, detection result information is input into a DeepSort algorithm. Secondly, in DeeepSort, trajectory curve optimization based on context association is provided, and the problem that Kalman filtering cannot be effectively predicted due to the fact that noise of state estimation of Kalman filtering is larger and larger under time accumulation under the shielding condition is effectively solved. And finally, in the tracking matching process of DeepSort, replacing the original IOU matching with CIOU matching to optimize the matching process of target tracking. In a multi-target pedestrian tracking task, accurate tracking can be more effectively carried out.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a multi-target pedestrian tracking algorithm based on DeepSort. Background Art

[0002] In the 21st century, with the advancement of urbanization, urban traffic difficulties have expanded to include both motor vehicles and pedestrians. Most collisions between motor vehicles and vehicles are caused by pedestrians failing to obey traffic rules. Real-time monitoring of pedestrian behavior and effective pedestrian identification and tracking in complex scenarios are key challenges. Due to the diverse characteristics of pedestrians and the occlusions that can occur during movement, identifying pedestrians and effectively re-identifying them after occlusion is a key challenge in the field of multi-target tracking.

[0003] Currently, target recognition algorithms for complex traffic scenarios fall into two main categories: deep learning-based target detection algorithms and classic detection algorithms. Traditional detection methods typically first extract candidate image frames using a sliding window, then perform feature extraction on the local information within each window, and then classify the extracted features. Consequently, the typical drawbacks of traditional detection algorithms can be summarized as poor target recognition accuracy, slow computational performance, and inadequate classification of derived image categories. Due to the continuous advancement of deep learning, the use of deep learning for multi-target recognition and tracking has gained widespread acceptance in the current research community. Therefore, an innovative context-based multi-pedestrian tracking model is proposed. This improved front-end detection incorporates an attention mechanism, and then utilizes CIOU matching to optimize target matching, resulting in improved tracking speed and accuracy for multiple pedestrians. Summary of the Invention

[0004] (1) Technical problems solved

[0005] Current mainstream online trackers suffer from occlusion in dense scenes, resulting in low tracking accuracy, weak re-identification correlation, and false detection and missed detection of targets. The present invention provides a multi-target pedestrian tracking algorithm based on DeepSort, which solves the problems raised in the above background technology.

[0006] (2) Technical solution

[0007] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions:

[0008] A multi-target pedestrian tracking algorithm based on DeepSort, including:

[0009] Obtain a pedestrian dataset. Use the Market-1501 dataset to train the feature extraction network. The Market-1501 dataset was collected on the Tsinghua University campus and constructed and made public in 2015. Use the CrowdHuman dataset to train the YOLO V5 detector. This dataset contains dense and complex crowds, as well as various occlusions.

[0010] Furthermore, image preprocessing was performed, randomly dividing the nearly 15,000 images in the dataset into training and validation sets in an 8:2 ratio. The training set contained nearly 12,000 pedestrian sample images, and the validation set contained nearly 300 pedestrian sample images. Finally, the obtained pedestrian images were scaled to a suitable size and fed into the improved model. Both the YOLO V5 model and the Deepsort feature extraction ReID model were trained.

[0011] Furthermore, the Triplet Attention mechanism was proposed. This mechanism first reduces the dimensions of the three dimensions: H, W, and C. It then performs average and maximum pooling operations, and then concatenates the information using a concatenation function. The concatenated information is 2*H*W. To restore the compressed dimensions, a convolution operation is performed to reduce the dimensions to 1*H*W, and the input is adjusted using a weight function. The same operation is performed on the other two dimensions. The so-called rotation operation is the corresponding transposition operation, so a dimensional conversion and addition operation are performed at the end.

[0012] The Triplet Attention mechanism module, built into the YOLOv5 backbone network, establishes connections between target channels or spatial locations, and also captures directional and positional information. Accuracy has always been a goal of target detection and tracking. To ensure accurate and smooth tracking, this paper incorporates the Triplet Attention mechanism into the model. This improves the accuracy of the front-end detector without significantly affecting tracking speed, providing strong support for subsequent algorithms and tracking.

[0013] Furthermore, in practice, the target can be re-associated by DeepSort after occlusion, but the target information will soon be lost again. This is because the parameter K of the Kalman filter deviates from the correct value due to the amplification of time error superposition. To solve this problem of lost reconnection, this paper proposes a Kalman filter based on context connection. Each target information will be stored during the tracking process. When the target is re-associated after occlusion, we will perform a context association between the last frame before occlusion and the first frame after occlusion reconnection based on context connection, and connect them into a smooth trajectory to modify and optimize the Kalman filter K value, so as to make the posterior estimation of the target more accurate after occlusion. We regard the information of the last frame before occlusion and the first frame of association as R, the last frame as t1, and the first frame after association as t2, where t1 < t < t2. The trajectory is the formula:

[0014]

[0015] where λ1 is the context connection trajectory weight and λ2 is the trajectory weight. The updated trajectory R is input into the Kalman filter to update the Kalman parameter K value and reduce the noise superposition caused by time accumulation. The updated Kalman posterior estimation formula is as follows:

[0016]

[0017] In the multi-target tracking task, the object occlusion problem has always been the core problem of target tracking. During occlusion, the variability of target changes and the accumulation of occlusion time will make this phenomenon more serious. The widespread existence of noise will reduce the re-identification ability of the tracking task. Only by analyzing the cause and effect can we grasp the true trend of target movement. This paper proposes that the Kalman filter based on context connection can effectively reduce and solve this problem.

[0018] Furthermore, in the original DeepSort tracking algorithm, IOU is used for target matching, and IOU has some limitations. For example, when there is no overlap between two bounding boxes, IOU is zero, resulting in a zero gradient and unable to perform secondary data optimization. Here, CIOU matching is used to replace the original model matching. Whether there is overlap between the target box and the prediction box, CIOU can still guide the movement of the target. Since CIOU quickly returns the target to the original position without moving the prediction position, the matching association is greatly accelerated.

[0019] (III) Beneficial effects

[0020] Compared with the prior art, the present invention provides a multi-target pedestrian tracking algorithm based on DeepSort, which has the following beneficial effects:

[0021] This invention optimizes the backbone network design through YOLOV5. The attention mechanism is integrated into the backbone network to help the model locate pedestrians more accurately, identify targets of interest, and improve tracking from the perspective of detection accuracy. A context-based Kalman filter is proposed to solve the problem of being unable to re-identify and track targets due to the accumulation of noise over time under occlusion. In order to quantify the match between the detection frame and the prediction frame and improve the accuracy of target matching, this paper adopts the CIOU correlation matching metric, which greatly speeds up the association of matching. Finally, through the optimized model, more accurate multi-target pedestrian tracking can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a schematic diagram of the present invention:

[0023] Figure 2 This is a schematic diagram of the multi-target pedestrian tracking framework flow of the present invention;

[0024] Figure 3 This is a schematic diagram of the YOLOV5 structure of the present invention;

[0025] Figure 4 Schematic diagram of the Triplet Attention mechanism of the present invention;

[0026] Figure 5 Schematic diagram of the improved YOLOV5 backbone structure of the present invention.

[0027] Figure 6 This is a schematic diagram of the cascade matching structure of the present invention.

[0028] Figure 7 Schematic diagram of the visualization structure of the model results before and after the improvement of the present invention. DETAILED DESCRIPTION

[0029] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0030] Example

[0031] The rapid development of technologies such as deep learning and computer vision has provided strong technical support and insights for pedestrian tracking. These technologies can track multiple pedestrians using real-time video and independently process pedestrian information to achieve target re-identification and localization. Compared with other traditional methods, deep learning-based tracking systems offer higher tracking accuracy and faster system response speeds.

[0032] like Figure 2 As shown, an embodiment of the present invention proposes a multi-target pedestrian tracking algorithm based on DeepSort, comprising the following steps:

[0033] Step 1: Obtain a pedestrian image dataset and pre-process the images by performing size transformation, classification, and labeling.

[0034] Step 2: Divide the processed images into training and validation sets to train the YOLOV5 model and DeepSort's REID corresponding model.

[0035] Step 3: Input the video sequence into the multi-target pedestrian tracking algorithm model to identify and track pedestrians.

[0036] Among them, such as Figure 3 As shown in some embodiments, it is the internal process framework of the multi-target pedestrian tracking model algorithm, and the specific process is as follows:

[0037] This model is divided into three parts. YOLOV5 is the primary framework for detection, outputting information about all detected pedestrians. A prediction branch, comprised of a Kalman filter, predicts pedestrian positions. Pedestrian positions are then matched using DeepSort cascade matching and Hungarian matching.

[0038] The T-time frame image is input to the YOLOV5 detector, which outputs multi-dimensional position information for multiple pedestrian targets. The detection box is primarily matched against the predicted box information at time T, generated by the Kalman filter at the previous time T-1. Whether to perform cascade matching is determined by the detected object's state and uncertainty. Cascade matching is performed for targets in the deterministic state, using the Mahalanobis distance between the detection and prediction and the minimum cosine of the feature vector extracted by DeepSort's ReID feature network. If a target in the deterministic state is not successfully matched, CIUO matching is performed on the target in the uncertainty state using the Hungarian loss algorithm.

[0039] The matching results are classified into three categories: 1. If the predicted trajectory matches the detected trajectory, the target is considered successfully tracked. 2. If the detected trajectory does not match the predicted trajectory, the target is considered a new target in the current state, and a new ID is created for it and the corresponding features are stored. 3. If the predicted trajectory does not match the detected trajectory, the target is considered no longer in the tracking range. Therefore, the ID is deleted based on the target's confirmed or unconfirmed state and the corresponding rematch threshold.

[0040] like Figure 4 As shown, in some embodiments, the TripletAttention in the improved YOLOV5 model backbone network is specifically implemented as follows:

[0041] The attention mechanism first reduces dimensions in three dimensions: H, W, and C. It then performs average and maximum pooling operations, and then concatenates the information using a concatenation function. The concatenated information is 2*H*W. To restore the compressed dimensions, a convolution operation is performed to reduce the dimensions to 1*H*W, and the input is adjusted using a weight function. The same operation is performed on the other two dimensions. The so-called rotation operation is equivalent to the corresponding transposition operation, so a dimensional conversion and addition operation are performed at the end.

[0042] like Figure 5 As shown, in some embodiments, the YOLOV5 backbone network structure after adding the attention mechanism is optimized, and the specific process is as follows:

[0043] This module can establish connections between target channels or spatial locations and capture directional and positional information. Accuracy has always been a goal of target detection and tracking. To ensure accurate detection and smoothness during the tracking process, this paper incorporates the TA attention mechanism into the model. This improves the accuracy of the front-end detector without significantly affecting tracking speed, providing strong support for subsequent algorithms and tracking.

[0044] To address the current problems of misdetection, missed detection, and poor tracking performance in multi-target pedestrian tracking, we proposed a multi-target pedestrian tracking model based on YOLOV5 and DeepSort. This method mainly uses front-end target detection as target information input, uses Kalman filtering to provide corresponding predictions, and the matching system classifies and matches the detected and predicted targets by ID. In the detector, we added an attention mechanism to improve detection accuracy in complex target scenes; in the prediction branch, we proposed a context-based Kalman filter algorithm that can effectively solve and improve the re-identification ability after target occlusion; in the matching system, we proposed using CIOU to replace the IOU matching of the original algorithm to enhance the matching process of target detection and prediction. Experimental results show that our model algorithm is significantly better than other tracking models.

[0045] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A multi-target pedestrian tracking algorithm based on DeepSort, characterized by: (1) Obtaining pedestrian feature images (2) Preprocessing of images and models (3) YOLOV5 was proposed to optimize the backbone network design, and the attention mechanism was integrated into the backbone network. (Triplet Attention) (4) In order to quantify the matching between the detection frame and the prediction frame and improve the accuracy of target matching, the present invention adopts the CIOU correlation matching metric.

2. The DeepSort-based multi-target pedestrian tracking algorithm according to claim 1, characterized in that: Method Step (1) The present invention uses the Market-1501 dataset to train the feature extraction network. The Market-1501 dataset was collected on the Tsinghua University campus and constructed and made public in 2015. The YOLOV5 detector is trained using the CrowdHuman dataset, which contains dense and complex crowds and various occlusions. This is to verify the performance of the tracker and its ability to re-identify multiple pedestrians in complex scenes.

3. The DeepSort-based multi-target pedestrian tracking algorithm according to claim 1, characterized in that: Method step (2) randomly divides the nearly 15,000 images in the dataset into a training set and a validation set in an 8:2 ratio. The training set contains 12,000 pedestrian sample images, and the validation set contains 300 pedestrian sample images. Finally, the obtained pedestrian images are expanded to a suitable size and input into the improved model.

4. The DeepSort-based multi-target pedestrian tracking algorithm according to claim 1, characterized in that: Method step (3) adds the Triplet Attention mechanism to the YOLOV5 detection module. This attention mechanism first reduces the dimensions of the three dimensions, performing dimensionality reduction operations on the H, W, and C dimensions respectively, through average pooling and maximum pooling operations, and then performing a concatenation function for splicing. The same operation is performed on the other two dimensions. The so-called rotation operation is the corresponding transposition operation, so a dimensional conversion is performed at the end for output.

5. The DeepSort-based multi-target pedestrian tracking algorithm according to claim 1, characterized in that: In step (4) of the method, CIOU matching is adopted at the target tracking matching end. Regardless of whether there is overlap between the target box and the predicted box, CIOU can still guide the movement of the target. Since CIOU quickly returns the target to the original position without moving the predicted position, it greatly speeds up the matching association.

Images