Method and device for detecting and tracking dense pedestrians

Abstract

The invention discloses a detection and tracking method and device for dense pedestrians, relates to the technical field of computer vision, and can solve the problem of low tracking accuracy during real-time detection and tracking in a scene with extremely dense pedestrians in the prior art. According to the specific technical scheme, firstly, a dense pedestrian detection data set and a tracking data set are obtained; an alpha-EIoU loss, a GAM attention mechanism and a Swin Transform module are used for improving a YOLOv5s algorithm, and a detection model is obtained; training and optimizing the detection model to obtain a target detection model; a ShuffleNetV2 module is introduced to improve a DeepSORT algorithm, and then training is performed to obtain a target tracking model; and detecting and tracking the pedestrian target video according to the target detection model and the target tracking model to obtain an experimental result of detection and tracking. The method is used for detecting and tracking dense pedestrians.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method and device for detecting and tracking dense pedestrians. Background Art

[0002] In recent years, multi-target tracking has become a research hotspot in many fields such as autonomous driving, intelligent monitoring, and human-computer interaction. The continuous development of multi-target tracking technology has provided strong support for the application and innovation in these fields. For example, in applications such as autonomous driving and intelligent monitoring, the accurate tracking of multiple pedestrian targets is directly related to the safety and reliability of the system. Accurately tracking targets can enable the system to identify potential threats in a timely manner and take appropriate countermeasures, thereby avoiding accidents and safety hazards.

[0003] At present, the research on multi-target tracking algorithms is mainly divided into two categories: detection-based tracking DBT algorithm and joint detection and tracking JDT algorithm. The former detects all targets in each frame first, and then tracks them based on the detection results. The latter generates target trajectories directly from the video sequence, without relying on a separate target detection step, but combining detection and tracking increases the complexity and computing requirements of the model, requiring higher hardware resources. Compared with the JDT algorithm, the DBT algorithm is more flexible and more adaptable to different scenarios. It is the current mainstream method for visual multi-target tracking based on deep learning.

[0004] However, for targets of different sizes, traditional target detection and tracking algorithms are difficult to achieve good results. There are many problems when used for real-time detection and tracking in scenes with extremely dense pedestrians. For example, when there is insufficient light, pedestrians are blocked by vehicles and buildings, targets block each other and deform, or the target is extremely small, it is easy to cause false detection, missed detection or matching errors, thereby greatly reducing the accuracy of tracking. Summary of the invention

[0005] The embodiment of the present invention provides a method and device for detecting and tracking dense pedestrians, which can solve the problem in the prior art that when used for real-time detection and tracking in extremely dense pedestrian scenes, when there is insufficient light, pedestrians are blocked by vehicles and buildings, targets block each other and deform, targets are extremely small, etc., it is easy to cause false detection, missed detection or matching errors, thereby greatly reducing the tracking accuracy. The technical solution is as follows:

[0006] According to a first aspect of an embodiment of the present invention, a method for detecting and tracking dense pedestrians is provided, the method comprising:

[0007] Acquire a dense pedestrian detection dataset and a tracking dataset, preprocess the detection dataset, and divide the detection dataset and the tracking dataset into a training set, a test set, and a validation set, respectively;

[0008] The α-EIoU loss is used to replace the CIOU loss of the YOLOv5s algorithm as the loss function of the bounding box regression, and the GAM attention mechanism is integrated into the Neck module of the YOLOv5s algorithm. At the same time, the Swin Transformer module is introduced to replace the backbone network of the YOLOv5s algorithm to obtain the detection model;

[0009] The detection model is trained according to the detection training set, and the detection model is optimized according to the detection test set and the detection verification set to obtain a target detection model;

[0010] The ShuffleNetV2 module is introduced into the pedestrian re-identification feature extraction network of the DeepSORT algorithm to obtain a tracking model, and the tracking model is trained according to a tracking training set, a tracking test set, and a tracking verification set to obtain a target tracking model;

[0011] Pedestrian target videos are detected and tracked according to the target detection model and the target tracking model to obtain detection and tracking experimental results.

[0012] The dense pedestrian detection and tracking method provided by the embodiment of the present invention first obtains a dense pedestrian detection data set and a tracking data set, preprocesses the detection data set, and divides the detection data set and the tracking data set into a training set, a test set, and a verification set, respectively; then, α-EIoU loss is used to replace CIOU loss of the YOLOv5s algorithm as the loss function of bounding box regression, and the GAM attention mechanism is integrated into the Neck module of the YOLOv5s algorithm, and a SwinTransformer module is introduced to replace the backbone network of the YOLOv5s algorithm to obtain a detection model; then, the detection model is trained according to the detection training set, and the detection model is optimized according to the detection test set and the detection verification set to obtain a target detection model; the ShuffleNetV2 module is introduced into the pedestrian re-identification feature extraction network of the DeepSORT algorithm to obtain a tracking model, and the tracking model is trained according to the tracking training set, the tracking test set, and the tracking verification set to obtain a target tracking model; finally, the pedestrian target video is detected and tracked according to the target detection model and the target tracking model to obtain experimental results of detection and tracking. The present invention can increase the detection and tracking accuracy of the model and improve the feature extraction capability. It can significantly improve missed detection and false detection caused by mutual occlusion, deformation, small targets and other problems among dense pedestrians. It can accurately track the expected trajectory and achieve accurate target detection and tracking.

[0013] As a further solution of the present invention: the preprocessing of the detection data set comprises the following steps:

[0014] Parse the dense pedestrian detection dataset and convert it into a YOLO format annotation file, where the dense pedestrian detection dataset comes from the CrowdHuman public dataset;

[0015] The dense pedestrian detection dataset after the conversion is divided into visible body, full body and head according to label types.

[0016] As a further solution of the present invention: the use of α-EIoU loss to replace the CIOUloss of the YOLOv5s algorithm as the loss function of the bounding box regression specifically refers to: using the loss function under the joint action of EIoU loss and αIoU loss to replace the CIOU loss of the YOLOv5s algorithm as the loss function of the bounding box regression.

[0017] As a further solution of the present invention: the EIOU loss function is expressed as:

[0018]

[0019] Among them, w c and h c are the width and height of the minimum bounding rectangle of the two boxes, ρ 2 (w,w gt ) and ρ 2 (b,b gt ) are the Euclidean distances of the width and height of the two frames respectively;

[0020] The αIOU loss function is expressed as:

[0021]

[0022] Among them, α is a parameter used to control the sensitivity and convergence speed of the loss function, υ is the relative ratio of the two detection frames, β is the weight coefficient, ρ is the Euclidean distance between the two detection frames, b,b gt The coordinates of the center points of the two detection frames, c is the diagonal distance of the closure area of ​​the two detection frames.

[0023] As a further solution of the present invention: the integration of the GAM attention mechanism into the Neck module of the YOLOv5s algorithm specifically includes:

[0024] A global attention mechanism GAM is added before the C3 structure of the neck network of the YOLOv5s algorithm. The GAM adopts a sequential channel-space attention mechanism and redesigns the CBAM submodule.

[0025] As a further solution of the present invention: the use of the tracking training set to train the tracking model specifically refers to:

[0026] The tracking model is trained on the Market-1501 dataset using the tracking training set.

[0027] As a further solution of the present invention: after detecting and tracking the pedestrian target video according to the target detection model and the target tracking model to obtain the detection and tracking experimental results, the method further includes:

[0028] The target detection model and the target tracking model are combined and input into the MOT17 dataset for tracking evaluation. The MOT17 dataset includes at least seven indoor and outdoor public place scenes with pedestrians.

[0029] According to a second aspect of an embodiment of the present invention, there is provided a dense pedestrian detection and tracking device, comprising an acquisition module, an improvement module, a first training module, a second training module and a test module;

[0030] The acquisition module is used to acquire a dense pedestrian detection data set and a tracking data set, preprocess the detection data set, and divide the detection data set and the tracking data set into a training set, a test set, and a validation set respectively;

[0031] The improved module is used to replace the CIOUloss of the YOLOv5s algorithm with α-EIoU loss as the loss function of bounding box regression, and integrate the GAM attention mechanism into the Neck module of the YOLOv5s algorithm, and introduce the SwinTransformer module to replace the backbone network of the YOLOv5s algorithm to obtain a detection model;

[0032] The first training module is used to train the detection model according to the detection training set, and optimize the detection model according to the detection test set and the detection verification set to obtain a target detection model;

[0033] The second training module is used to introduce the ShuffleNetV2 module into the pedestrian re-identification feature extraction network of the DeepSORT algorithm to obtain a tracking model, and train the tracking model according to the tracking training set, the tracking test set and the tracking verification set to obtain a target tracking model;

[0034] The test module is used to detect and track pedestrian target videos according to the target detection model and the target tracking model to obtain detection and tracking experimental results.

[0035] The dense pedestrian detection and tracking device provided by the embodiment of the present invention comprises an acquisition module, an improvement module, a first training module, a second training module and a test module; the acquisition module acquires a dense pedestrian detection data set and a tracking data set, preprocesses the detection data set, and divides the detection data set and the tracking data set into a training set, a test set and a verification set respectively; the improvement module uses α-EIoU loss to replace CIOU loss of the YOLOv5s algorithm as the loss function of the bounding box regression, and integrates the GAM attention mechanism into the Neck module of the YOLOv5s algorithm, and introduces Swin The Transformer module replaces the backbone network of the YOLOv5s algorithm to obtain a detection model; the first training module trains the detection model according to the detection training set, and optimizes the detection model according to the detection test set and the detection verification set to obtain a target detection model; the second training module introduces the ShuffleNetV2 module in the pedestrian re-identification feature extraction network of the DeepSORT algorithm to obtain a tracking model, and trains the tracking model according to the tracking training set, the tracking test set and the tracking verification set to obtain a target tracking model; the test module detects and tracks the pedestrian target video according to the target detection model and the target tracking model to obtain the experimental results of detection and tracking. The present invention can increase the detection and tracking accuracy of the model, improve the feature extraction capability, and can improve the missed detection and false detection caused by the mutual occlusion, deformation, small targets and other problems between dense pedestrians, and can accurately track the expected trajectory to achieve accurate detection and tracking of the target.

[0036] According to a third aspect of an embodiment of the present invention, a device for detecting and tracking dense pedestrians is provided, the device comprising a processor and a memory, the memory storing at least one computer instruction, the instruction being loaded and executed by the processor to implement the steps performed in any of the above-mentioned methods for detecting and tracking dense pedestrians.

[0037] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein at least one computer instruction is stored in the storage medium, and the instruction is loaded and executed by a processor to implement the steps performed in any of the above-mentioned methods for detecting and tracking dense pedestrians.

[0038] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] (1) The improved algorithm was verified using the CrowdHuman dense pedestrian detection dataset and the MOT17 tracking dataset. The experimental results show that the average detection accuracy of the improved YOLOv5s model is 4.2% higher than that of the original model. Combined with the improved DeepSORT algorithm tracking, the multi-target tracking accuracy is 88.26%, which is 6.8% higher than the original YOLOv5 and DeepSORT algorithm tracking accuracy, and the identity switching is reduced by 21 times on average. Real-time detection is also possible on devices with poor computing power, with good accuracy.

[0041] (2) The problems of false detection, missed detection or matching errors caused by insufficient lighting, pedestrians blocked by vehicles or buildings, objects blocking each other and deforming, and extremely small objects are greatly improved, reducing the missed detection and false detection rates.

[0042] (3) After DeepSORT with the ShuffleNetV2 lightweight network was introduced for pedestrian re-identification training, its model size was reduced by 18 times compared with the original DeepSORT model. The model size was reduced from 45M to 2.5M. It not only meets the real-time and accurate tracking of dense pedestrians, but also has a very small model size and greatly reduced computational complexity, which can support deployment on edge devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0044] Figure 1 is a flow chart of a method for detecting and tracking dense pedestrians provided by an embodiment of the present invention;

[0045] Figure 2 It is a schematic diagram of the structure of an improved YOLOv5s algorithm network model in a dense pedestrian detection and tracking method provided by an embodiment of the present invention;

[0046] Figure 3 It is a schematic diagram of the structure of a SwinTransformer model in a method for detecting and tracking dense pedestrians provided by an embodiment of the present invention;

[0047] Figure 4 Schematic diagram of the structure of a GAM module in a method for detecting and tracking dense pedestrians provided by an embodiment of the present invention;

[0048] Figure 5 is a DeepSORT tracking implementation flow chart of a dense pedestrian detection and tracking method provided by an embodiment of the present invention;

[0049] Figure 6It is a structural diagram of a dense pedestrian detection and tracking device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of systems and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0051] The embodiment of the present invention provides a method for detecting and tracking dense pedestrians. Figure 1 As shown, the dense pedestrian detection and tracking method includes the following steps:

[0052] Step 101: Acquire a dense pedestrian detection dataset and a tracking dataset, preprocess the detection dataset, and divide the detection dataset and the tracking dataset into a training set, a test set, and a validation set, respectively;

[0053] In one embodiment, the detection data set is preprocessed, including the following steps:

[0054] Parse the dense pedestrian detection dataset and convert it into a YOLO format annotation file. The dense pedestrian detection dataset comes from the CrowdHuman public dataset.

[0055] The converted dense pedestrian detection dataset is divided into visible body, full body and head according to label types.

[0056] In this embodiment, the detection data set comes from the CrowdHuman public data set in various complex scenarios. The annotation files have their own unique formats and need to be parsed and converted into yolo format annotation files that can be recognized by YOLOv5s. The label types are reclassified in the annotation files, specifically: visible body, full body, head three types of labels, and finally the data set is divided into training set, test set and validation set according to a preset ratio, such as 3:1:1.

[0057] Step 102: Use α-EIoU loss to replace CIOU loss of YOLOv5s algorithm as the loss function of bounding box regression, integrate GAM attention mechanism into Neck module of YOLOv5s algorithm, and introduce Swin Transformer module to replace the backbone network of YOLOv5s algorithm to obtain the detection model.

[0058] Specifically, the YOLOv5 algorithm consists of four parts: input, backbone network, neck network and output. After data enhancement, adaptive padding and anchor box calculation at the input, the image enters the backbone network to extract multi-level features, and then the BottleneckCSP module is used to reduce the amount of calculation and improve the inference speed, and the SPP module is used to extract features of different scales to improve the accuracy. The neck network combines FPN to transmit semantic information from top to bottom and PAN to transmit positioning information from bottom to top to achieve global fusion and local refinement. The output predicts the target box and category confidence on feature maps of different sizes to complete the detection task. In view of the characteristics of the CrowdHuman pedestrian dataset, such as dense targets, large scale changes, and serious occlusion and overlap, the YOLOv5s algorithm is improved. First, the GAM attention mechanism is integrated into the neck module to extract rich road pedestrian image features; secondly, the Swin Transformer with a hierarchical structure and sliding window attention mechanism is introduced into its backbone network to enhance the algorithm's recognition ability of pedestrians; finally, α-EIoU is used as the loss function of the pedestrian detection network to accelerate the convergence of the model. The overall network model structure of the improved YOLOv5s algorithm is as follows Figure 2 shown.

[0059] In one embodiment, using α-EIoU loss to replace CIOU loss of the YOLOv5s algorithm as the loss function of the bounding box regression specifically refers to: using a loss function under the joint action of EIoU loss and αIoU loss to replace CIOU loss of the YOLOv5s algorithm as the loss function of the bounding box regression.

[0060] In one embodiment, the EIOU loss function is expressed as:

[0061]

[0062] Among them, w c and h c are the width and height of the minimum bounding rectangle of the two boxes, ρ 2 (w,w gt ) and ρ 2 (b,b gt ) are the Euclidean distances of the width and height of the two frames respectively;

[0063] As can be seen from the above formula, EIoU Loss includes IoU loss, distance loss and edge length loss.

[0064] The αIOU loss function is expressed as:

[0065]

[0066] Among them, α is a parameter used to control the sensitivity and convergence speed of the loss function, υ is the relative ratio of the two detection frames, β is the weight coefficient, ρ is the Euclidean distance between the two detection frames, b,b gt The coordinates of the center points of the two detection frames, c is the diagonal distance of the closure area of ​​the two detection frames.

[0067] In one embodiment, incorporating the GAM attention mechanism into the Neck module of the YOLOv5s algorithm specifically includes:

[0068] A global attention mechanism GAM is added before the C3 structure of the neck network of the YOLOv5s algorithm. GAM adopts a sequential channel-spatial attention mechanism and redesigns the CBAM submodule.

[0069] Specifically, adding a global attention mechanism GAM before the C3 structure of the neck network of YOLOv5s can strengthen the interaction and integration of global information and make the model pay more attention to important global information.

[0070] In this embodiment, the preprocessed CrowdHuman pedestrian detection dataset is used as the input of the improved YOLOv5s algorithm network, and the data passes through the backbone part and the Neck part successively. In the backbone module, SwinTransformer-tiny is used as the backbone, and the input RGB image is split into non-overlapping equal-sized modules through the splicing and splitting module, and each module is flattened into a 48-dimensional vector. Secondly, the linear embedding module projects the tensor of dimension (H / 4×W / 4)×48 at this time to an arbitrary dimension C to obtain a fully connected layer of dimension (H / 4×W / 4)×C. Subsequently, these image block markers are sent to the Swin Transformer module with improved self-attention. The first Swin Transformer block keeps the number of input and output vectors constant at (H / 4×W / 4), and is designated as Stage1 together with the linear embedding layer (such as Figure 3 Finally, the final prediction result is obtained through feature extraction and downsampling of Stages 1, 2, 3 and 4. Each stage doubles the tensor dimension and reduces the image size to half of the original. This hierarchical structure similar to the convolutional neural network improves the model's ability to obtain global information. At the same time, the shift window division method in Swin Transformer realizes the information interaction between pixels in different windows, indirectly increases the receptive field of feature extraction, improves the efficiency of information utilization, and thus can more accurately identify targets, especially for small targets.

[0071] The Neck part is composed of a spatial pyramid pooling (SPP) module and a feature pyramid network (FPN) module, which fuse features of different scales and different semantic levels respectively, so that the features of the fused backbone contain multi-scale and multi-semantic information. The extracted data features then enter the Neck part, and we add a global attention mechanism GAM before the C3 structure of the neck network of YOLOv5s to strengthen the interaction and integration of global information, dynamically adjust the weights of each position in the feature map, and make the model pay more attention to important global information. GAM adopts a sequential channel-spatial attention mechanism and redesigns the CBAM submodule. The GAM module structure is as follows: Figure 4 As shown in the figure. The channel attention module retains information through three-dimensional arrangement and uses two layers of MLP (multi-layer perceptron) to amplify cross-dimensional dependencies. The spatial attention module uses two convolutional layers to fuse spatial information and borrows the same reduction ratio r as BAM from the channel attention module. In order to retain more feature maps, the pooling operation is removed, and the number of channels and the amount of calculation are reduced through 7x7 convolution. The data is finally output through the Sigmoid activation function.

[0072] In the final prediction stage, the loss function of the combined effect of EIoU loss and αIoU loss is used as the target box loss function of the improved model to obtain the final model output. EIoU Loss is an improved positioning loss function for CIoU Loss. EIoU calculates the difference in width and height based on CIoU, replaces the aspect ratio, minimizes the width and height loss, and accelerates the convergence speed. At the same time, Focal loss is introduced to optimize the sample imbalance problem and effectively prevent occluded targets from being mistakenly filtered out by NMS. αIoU is more adaptable to small data sets and noise. When α=1, it regresses to the original losses. αIoU and EIoU are selected to work together to replace the CIoU of the original model, so that the new algorithm is more adaptable to complex environments and further improves the detection accuracy. Experimental verification shows that when EIoU Loss is set to true and α=2, the detection effect is the best and there is no additional increase in calculation amount.

[0073] Step 103: train the detection model according to the detection training set, and optimize the detection model according to the detection test set and the detection verification set to obtain the target detection model.

[0074] In this embodiment, the improved YOLOv5s algorithm is trained, and the optimizer and learning rate are adjusted according to the situation to make the model converge better.

[0075] Specifically, the improved network is trained using the detection training set, the model parameters are continuously adjusted during the training process, the trained model is verified and tested on the detection test set and the detection validation set, and the α value with the best detection effect and the model with the best detection effect are selected.

[0076] In this embodiment, the weights and thresholds of YOLOv5s before and after improvement are first initialized.

[0077] Then, the model is trained in batch training mode for a total of 300 epochs. Each epoch trains all images in the train file. Each piece of data includes the image itself and the label. According to step 1, the pre-processed data is input into the model, the data is transmitted module by module, and the difference between the model output and the actual detection is calculated. The IOU of the model and the mAP value calculated by the accuracy and recall rate and other performance evaluation indicators of the model are calculated.

[0078] The retrained parameters are back-propagated through the model, adjusting and updating weights and biases layer by layer.

[0079] Finally, determine whether the pedestrian detection CrowdHuman dataset has been trained. If not, return to step 2. If yes, end the training, then calculate the accuracy of the model, save the trained YOLOv5s model, and load it for testing. If the model performance is not ideal and the error is too large, consider adjusting the step size and replacing the optimizer.

[0080] Step 104: introducing the ShuffleNetV2 module into the pedestrian re-identification feature extraction network of the DeepSORT algorithm to obtain a tracking model, and training the tracking model according to the tracking training set, the tracking test set, and the tracking verification set to obtain a target tracking model;

[0081] Specifically, ShuffleNetV2 is a lightweight neural network architecture that uses deep separable convolution to reduce the amount of computation. It decomposes the standard convolution into two steps: deep convolution and point-by-point convolution, which are used to process spatial information and cross-channel feature fusion respectively. In addition, it uses a multi-scale strategy in its feature extraction stage to capture image information at different scales, enhancing the appearance feature extraction capability of DeepSORT.

[0082] In one embodiment, using the tracking training set to train the tracking model specifically refers to:

[0083] The tracking model is trained on the Market-1501 dataset using the tracking training set.

[0084] Specifically, the DeepSORT algorithm is improved and the ShuffleNetV2 lightweight module is introduced to replace the original CNN appearance feature extraction model. ShuffleNetV2 uses a multi-scale strategy in the feature extraction stage to capture image information of different scales, thereby enhancing the appearance feature extraction capability of DeepSort.

[0085] Load the trained and improved DeepSORT person re-identification model.

[0086] The preprocessed Market-1501 person re-identification dataset is input into the model to obtain training data.

[0087] Step 105: Detect and track the pedestrian target video according to the target detection model and the target tracking model to obtain experimental results of detection and tracking.

[0088] Specifically, the video to be tracked is input into the improved YOLOv5s target detection network to extract the appearance information features, screen the candidate frames, and then use NMS to remove overlapping candidate frames for further screening. Then the data is associated through the improved DeepSORT tracking algorithm, and the Kalman filter is used to predict and update the position of the target in the next frame. Then the optimal allocation of the prediction frame and the detection frame is achieved according to the Hungarian algorithm, and finally the tracking result is output. The DeepSORT tracking implementation process is as follows: Figure 5 shown.

[0089] In one embodiment, after detecting and tracking the pedestrian target video according to the target detection model and the target tracking model to obtain the detection and tracking experimental results, the method further includes:

[0090] The target detection model and the target tracking model are combined and input into the MOT17 dataset for tracking evaluation. The MOT17 dataset includes at least seven indoor and outdoor public place scenes with pedestrians.

[0091] The evaluation indicators of the detection experiment are precision, recall, and mAP50. The larger the mAP50, the higher the average detection accuracy of the network model. The evaluation indicators of the tracking experiment are multi-target tracking accuracy MOTA, multi-target tracking precision MOTP, and the number of identity switching IDSW.

[0092] In this embodiment, ablation experiments are performed on the improved YOLOv5s algorithm and DeepSORT algorithm, and the results are analyzed. Table 1 is the ablation experiment of the YOLOv5s algorithm, and Table 2 is the comparison experiment of person target re-identification (where P refers to precision, R refers to recall, and mAP refers to average detection accuracy).

[0093] Table 1 YOLOv5s algorithm ablation experiment

[0094]

[0095] From the data in Table 1, we can see that after the introduction of GAM, the average detection precision and recall rate of the YOLOv5s model have been improved, with mAP increased by 1.1%, R increased by 1.2%, and P increased by 1.8%; after replacing the backbone network of YOLOv5s with SwinTransformer-tiny, the precision and recall rate of the model have been greatly improved, with mAP increased by 3.1%, R increased by 3.2%, and P increased by 1.9%; when its loss function is changed to α-EIoU (where α=2), mAP increased by 1.9%, R increased by 2.3%, and P increased by 1.6%; combined with the three, the performance of the improved YOLOv5s model has been significantly improved, with mAP increased by 4.2%, R increased by 3.5%, and P increased by 2.6%. The improved model significantly improves the detection accuracy, recall rate and average detection precision, and also enhances the robustness of the algorithm in complex situations.

[0096] Table 2 Comparative experiments on person target re-identification

[0097]

[0098] It can be seen from Table 2 that when the improved DeepSORT model is used for person target re-identification training, its model size is reduced by 18 times compared with the original DeepSORT model, and the model size is reduced from the original 45M to 2.5M, and it still has a high accuracy on the training set and the validation set.

[0099] To verify the performance of the improved YOLOv5s combined with the improved DeepSORT in dense pedestrian tracking, the test was conducted on the MOT17 dataset, and a combined ablation experiment was conducted on detection and tracking, and the results were analyzed. Table 3 shows the comparison experiment of multi-target tracking results (where MOTA is the multi-target tracking accuracy, MOTP is the multi-target tracking precision, IDS is the number of identity switches, and FPS is the number of frames per second of video processed).

[0100] Table 3 Comparative experiment results of multi-target tracking

[0101]

[0102] As can be seen from the table, the MOTA value of the proposed algorithm is 88.26% and the MOTP value of the multi-target tracking precision is 87.47%. Compared with the original algorithm, MOTA is improved by 6.8%, MOTP is improved by 4.23%, the identity switching frequency is significantly reduced from 47 times to 19 times, the tracking stability and continuity are enhanced, and the number of frames processed per second is increased.

[0103] The dense pedestrian detection and tracking method provided by the embodiment of the present invention first obtains a dense pedestrian detection data set and a tracking data set, preprocesses the detection data set, and divides the detection data set and the tracking data set into a training set, a test set, and a verification set, respectively; then, α-EIoU loss is used to replace CIOU loss of the YOLOv5s algorithm as the loss function of bounding box regression, and the GAM attention mechanism is integrated into the Neck module of the YOLOv5s algorithm, and a SwinTransformer module is introduced to replace the backbone network of the YOLOv5s algorithm to obtain a detection model; then, the detection model is trained according to the detection training set, and the detection model is optimized according to the detection test set and the detection verification set to obtain a target detection model; the ShuffleNetV2 module is introduced into the pedestrian re-identification feature extraction network of the DeepSORT algorithm to obtain a tracking model, and the tracking model is trained according to the tracking training set, the tracking test set, and the tracking verification set to obtain a target tracking model; finally, the pedestrian target video is detected and tracked according to the target detection model and the target tracking model to obtain experimental results of detection and tracking. The present invention can increase the detection and tracking accuracy of the model and improve the feature extraction capability. It can significantly improve missed detection and false detection caused by mutual occlusion, deformation, small targets and other problems among dense pedestrians. It can accurately track the expected trajectory and achieve accurate target detection and tracking.

[0104] Based on the above Figure 1 The method for detecting and tracking dense pedestrians described in the corresponding embodiments is described below as a system embodiment of the present invention, which can be used to execute the method embodiment of the present invention.

[0105] The embodiment of the present invention provides a dense pedestrian detection and tracking device, such as Figure 6 As shown, the device includes an acquisition module 201, an improvement module 202, a first training module 203, a second training module 204 and a testing module 205;

[0106] An acquisition module 201 is used to acquire a dense pedestrian detection dataset and a tracking dataset, preprocess the detection dataset, and divide the detection dataset and the tracking dataset into a training set, a test set, and a validation set respectively;

[0107] Improved module 202, used to replace CIOU loss of YOLOv5s algorithm with α-EIoU loss as loss function of bounding box regression, integrate GAM attention mechanism into Neck module of YOLOv5s algorithm, and introduce SwinTransformer module to replace backbone network of YOLOv5s algorithm to obtain detection model;

[0108] The first training module 203 is used to train the detection model according to the detection training set, and optimize the detection model according to the detection test set and the detection verification set to obtain the target detection model;

[0109] The second training module 204 is used to introduce the ShuffleNetV2 module into the pedestrian re-identification feature extraction network of the DeepSORT algorithm to obtain a tracking model, and train the tracking model according to the tracking training set, the tracking test set and the tracking verification set to obtain a target tracking model;

[0110] The testing module 205 is used to detect and track pedestrian target videos according to the target detection model and the target tracking model to obtain experimental results of detection and tracking.

[0111] The dense pedestrian detection and tracking device provided by the embodiment of the present invention includes an acquisition module 201, an improvement module 202, a first training module 203, a second training module 204 and a test module 205; the acquisition module 201 acquires a dense pedestrian detection data set and a tracking data set, preprocesses the detection data set, and divides the detection data set and the tracking data set into a training set, a test set and a verification set respectively; the improvement module 202 uses α-EIoU loss to replace the CIOUloss of the YOLOv5s algorithm as the loss function of the bounding box regression, and integrates the GAM attention mechanism into the Neck module of the YOLOv5s algorithm, and introduces the Swin The Transformer module replaces the backbone network of the YOLOv5s algorithm to obtain a detection model; the first training module 203 trains the detection model according to the detection training set, and optimizes the detection model according to the detection test set and the detection verification set to obtain a target detection model; the second training module 204 introduces the ShuffleNetV2 module in the pedestrian re-identification feature extraction network of the DeepSORT algorithm to obtain a tracking model, and trains the tracking model according to the tracking training set, the tracking test set and the tracking verification set to obtain a target tracking model; the test module 205 detects and tracks the pedestrian target video according to the target detection model and the target tracking model to obtain the experimental results of detection and tracking. The present invention can increase the detection and tracking accuracy of the model, improve the feature extraction capability, and can improve the missed detection and false detection caused by the mutual occlusion, deformation, small targets and other problems between dense pedestrians, and can accurately track the expected trajectory to achieve accurate detection and tracking of the target.

[0112] Based on the above Figure 1 Corresponding to the dense pedestrian detection and tracking method described in the embodiment, another embodiment of the present invention also provides a dense pedestrian detection and tracking device, the dense pedestrian detection and tracking device includes a processor and a memory, the memory stores at least one computer instruction, the instruction is loaded and executed by the processor to implement the above Figure 1 The dense pedestrian detection and tracking method described in the corresponding embodiment.

[0113] Based on the above Figure 1 In accordance with the detection and tracking method for dense pedestrians described in the embodiments, the embodiments of the present invention further provide a computer-readable storage medium, for example, a non-temporary computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. The storage medium stores at least one computer instruction for executing the above Figure 1 The dense pedestrian detection and tracking method described in the corresponding embodiment will not be repeated here.

[0114] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the disclosure disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed by the present invention. The description and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0115] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for detecting and tracking dense pedestrians, characterized in that: The following steps are involved: Acquire a dense pedestrian detection dataset and a tracking dataset, preprocess the detection dataset, and divide the detection dataset and the tracking dataset into a training set, a test set, and a validation set, respectively; The α-EIoU loss is used to replace the CIOU loss of the YOLOv5s algorithm as the loss function of the bounding box regression, and the GAM attention mechanism is integrated into the Neck module of the YOLOv5s algorithm. At the same time, the Swin Transformer module is introduced to replace the backbone network of the YOLOv5s algorithm to obtain the detection model; The detection model is trained according to the detection training set, and the detection model is optimized according to the detection test set and the detection verification set to obtain a target detection model; The ShuffleNetV2 module is introduced into the pedestrian re-identification feature extraction network of the DeepSORT algorithm to obtain a tracking model, and the tracking model is trained according to a tracking training set, a tracking test set, and a tracking verification set to obtain a target tracking model; Pedestrian target videos are detected and tracked according to the target detection model and the target tracking model to obtain detection and tracking experimental results.

2. The method for detecting and tracking dense pedestrians according to claim 1, characterized in that: The preprocessing of the detection data set comprises the following steps: Parse the dense pedestrian detection dataset and convert it into a YOLO format annotation file, where the dense pedestrian detection dataset comes from the CrowdHuman public dataset; The dense pedestrian detection dataset after the conversion is divided into visible body, full body and head according to label types.

3. The method for detecting and tracking dense pedestrians according to claim 1, characterized in that: The use of α-EIoUloss to replace the CIOU loss of the YOLOv5s algorithm as the loss function of the bounding box regression specifically refers to: using the loss function under the joint action of EIoU loss and αIoU loss to replace the CIOU loss of the YOLOv5s algorithm as the loss function of the bounding box regression.

4. The method for detecting and tracking dense pedestrians according to claim 3, characterized in that: The EIOU loss function is expressed as: Among them, w c and h c are the width and height of the minimum bounding rectangle of the two boxes, ρ 2 (w,w gt ) and ρ 2 (b,b gt ) are the Euclidean distances of the width and height of the two frames respectively; The αIOU loss function is expressed as: Among them, α is a parameter used to control the sensitivity and convergence speed of the loss function, υ is the relative ratio of the two detection frames, β is the weight coefficient, ρ is the Euclidean distance between the two detection frames, b,b gt The coordinates of the center points of the two detection frames, c is the diagonal distance of the closure area of ​​the two detection frames.

5. The method for detecting and tracking dense pedestrians according to claim 1, characterized in that: The incorporation of the GAM attention mechanism into the Neck module of the YOLOv5s algorithm specifically includes: A global attention mechanism GAM is added before the C3 structure of the neck network of the YOLOv5s algorithm. The GAM adopts a sequential channel-space attention mechanism and redesigns the CBAM submodule.

6. The method for detecting and tracking dense pedestrians according to claim 1, characterized in that: The using the tracking training set to train the tracking model specifically refers to: The tracking model is trained on the Market-1501 dataset using the tracking training set.

7. The method for detecting and tracking dense pedestrians according to claim 1, characterized in that: After detecting and tracking the pedestrian target video according to the target detection model and the target tracking model to obtain the detection and tracking experimental results, the method further includes: The target detection model and the target tracking model are combined and input into the MOT17 dataset for tracking evaluation. The MOT17 dataset includes at least seven indoor and outdoor public place scenes with pedestrians.

8. A device for detecting and tracking dense pedestrians, characterized in that: It includes an acquisition module, an improvement module, a first training module, a second training module and a test module; The acquisition module is used to acquire a dense pedestrian detection data set and a tracking data set, preprocess the detection data set, and divide the detection data set and the tracking data set into a training set, a test set, and a validation set respectively; The improved module is used to replace the CIOU loss of the YOLOv5s algorithm with α-EIoU loss as the loss function of the bounding box regression, and integrate the GAM attention mechanism into the Neck module of the YOLOv5s algorithm, and introduce the SwinTransformer module to replace the backbone network of the YOLOv5s algorithm to obtain a detection model; The first training module is used to train the detection model according to the detection training set, and optimize the detection model according to the detection test set and the detection verification set to obtain a target detection model; The second training module is used to introduce the ShuffleNetV2 module into the pedestrian re-identification feature extraction network of the DeepSORT algorithm to obtain a tracking model, and train the tracking model according to the tracking training set, the tracking test set and the tracking verification set to obtain a target tracking model; The test module is used to detect and track pedestrian target videos according to the target detection model and the target tracking model to obtain detection and tracking experimental results.

Images