A traffic small target detection method based on YOLOV5 fusion multi-target feature enhancement network and attention mechanism

Abstract

The application relates to the technical field of urban traffic vehicle management, and discloses a traffic small target detection method based on a YOLOV5 fusion multi-target feature enhancement network and an attention mechanism, steps 1, main feature data is obtained by preprocessing a picture; step 2, the preprocessed feature map is sent into a backbone network for feature extraction, and a CBAM attention mechanism is used to recombine feature channels, so that key features are given larger weights to highlight semantic information and detailed features of small targets; step 3, the feature map recombined through the backbone network is input into a feature enhancement network, a multi-scale feature recombination module is optimized, a detection head more suitable for small targets is added on the basis, small target features of 16 pixels can be detected, and candidate frames of possible targets are calibrated; the application can improve the detection precision and accuracy of small targets in unmanned aerial vehicle aerial images, so that the unmanned aerial vehicle can be better applied to the monitoring of urban traffic.

Description

Technical Field

[0001] This invention relates to the field of urban traffic vehicle management technology, specifically to a method for detecting small traffic targets based on YOLOv5 fusion of multi-target feature enhancement network and attention mechanism. Background Technology

[0002] With the continuous improvement of drone performance and the expansion of application scenarios, drones can be used as a new sensing module for urban traffic to help monitor traffic systems, detect small traffic targets such as vehicles, pedestrians, and bicycles on the road, promote the formation of intelligent transportation systems, and prevent accidents. However, with the increase in car ownership, urban road traffic often experiences target overlap in dense scenes. Due to the significant mutual occlusion between targets, missed detections and duplicate detections are very likely to occur. To improve the detection accuracy and precision of existing algorithms for small targets, this paper proposes a traffic small target detection method based on YOLOv5 fusion multi-target feature enhancement network and attention mechanism. Summary of the Invention

[0003] The purpose of this invention is to provide a traffic small target detection method based on YOLOv5, which integrates a multi-target feature enhancement network and an attention mechanism. This addresses the issue raised in the background section where, with the increasing number of cars, overlapping targets frequently occur in dense urban traffic scenes. Due to significant mutual occlusion between targets, missed detections and duplicate detections are easily caused. By improving upon existing algorithms, the accuracy and precision of small target detection are enhanced.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a traffic small target detection method based on YOLOv5 fusion of multi-target feature enhancement network and attention mechanism, comprising the following steps:

[0005] Step 1: Preprocess the image to obtain the main feature data;

[0006] Step 2: Feed the preprocessed feature map into the backbone network for feature extraction and use the CBAM attention mechanism to reorganize the feature channels, giving key features a larger weight to highlight the semantic information and detailed features of small targets.

[0007] Step 3: The feature map after being reconstructed by the backbone network is input into the feature enhancement network. By optimizing the multi-scale feature reconstruction module, a detection head more suitable for small targets is added on the original basis, so that it can detect small target features of 16 pixels. Candidate boxes are marked for possible targets. At the same time, depthwise separable convolution is used to replace ordinary convolution in the feature enhancement network, thereby reducing the number of parameters and accelerating the model inference speed, improving the receptive field and feature expression ability.

[0008] Step 4: The feature map processed by the feature enhancement network is fed into the object detection network. The position of the candidate box is corrected according to the result of the position loss function. At the same time, the non-maximum suppression algorithm is used to suppress and delete redundant candidate boxes, thereby outputting more accurate detection results.

[0009] As a further preferred embodiment of this technical solution: In step 2, the input image is preprocessed with data augmentation, the processed image is fed into a deep neural network for training, and the trained model is output. In the backbone network, the Mosaic data augmentation algorithm is used on the input image to randomly flip, translate, and crop the image to stitch the four images into one image, thereby improving the diversity and richness of the data.

[0010] As a further preferred embodiment of this technical solution: [The image feature map] The input is processed in the backbone network for feature extraction. A CBAM attention module is used to reorganize different channels of the feature map, giving key features a larger weight to highlight the semantic information and detailed features of small targets, and reducing the negative impact of irrelevant features, such as background, on target feature extraction.

[0011] As a further preferred embodiment of this technical solution: global average pooling is used to process the feature map. Dimensionality reduction is performed, where C represents the number of channels, H represents the height, and W represents the width, to obtain the global feature description of each channel. The specific operation process is as follows:

[0012]

[0013] Where C represents the number of channels. This represents feature point information on different channels;

[0014] The fully connected network layer maps the acquired global features and learns the importance weights of each channel. The channel weights are multiplied by the original feature map to adjust the representation of channel features and enhance the representation ability of important channels. The specific operation process is as follows:

[0015]

[0016] in, For the sigmoid function, , To represent the feature maps generated spatially using average pooling and max pooling, and This indicates that two multilayer perceptron networks are used to fuse the results of max pooling and average pooling, respectively.

[0017] After max pooling and average pooling operations along the channel dimension, two different spatial feature representations are obtained. A convolutional operation is then used to fuse these two spatial features. By learning the weights of the convolutional kernels, the correlation between different locations is captured. The learned spatial weights are multiplied with the original feature map to obtain a feature map adjusted for spatial attention, emphasizing the spatial information of important regions. This increases the weight of small target features in aerial images, improving the global perception of small targets. The specific operation process is as follows:

[0018]

[0019] in, For the sigmoid function, The kernel size is Convolution operation, This represents the result of using an MLP network to combine average pooling and max pooling.

[0020] As a further preferred embodiment of this technical solution: the feature map output in step 3 is fed into the feature enhancement network for multi-scale feature recombination, and a new small target detection head with 4x downsampling is added to enable it to detect small target features of 16 pixels.

[0021] As a further preferred embodiment of this technical solution: in step 3, depthwise separable convolution is used to replace ordinary convolution in the feature enhancement network. The method of separating depth information and spatial information and processing them layer by layer reduces the number of parameters, while accelerating the model inference speed, improving the receptive field and feature expression ability, controlling overfitting and saving memory consumption.

[0022] As a further preferred embodiment of this technical solution: First, deep convolution is used to extract shared features from the input features. The extracted features are then used for subsequent calculations of spatial attention weights and channel attention weights. The specific operation process is as follows:

[0023]

[0024] in It is the output of a single depth layer. It is the input data. It is a filtering matrix. Represents the position coordinates in the spatial dimension, H and W are the height and width of the filter, c is the index of the input channel, and k is the index of the output channel;

[0025] Then, pointwise convolution is performed using 1x1 kernels at depth to add or multiply the feature maps of different channels element-wise, achieving feature integration and interaction. The specific operation process is as follows:

[0026]

[0027] Where y is the output data, v is the weight matrix, and c,k represent the number of input channels and the number of output channels.

[0028] As a further preferred embodiment of this technical solution: the content in step 4 proceeds to the final target recognition stage. Based on the result of the position loss function, the position of the candidate boxes is corrected, thereby outputting more accurate detection results. First, the ratio of the area of ​​the intersection between the predicted box and the ground truth box to the area of ​​the merged portion of the two boxes is calculated, also known as the intersection-union ratio (Iou).

[0029]

[0030] in This represents the area of ​​the intersection of the two frames. This represents the area of ​​the portion where the two frames meet.

[0031] For each predicted bounding box, first calculate its Intersection over Union (IOU) with all other predicted bounding boxes, and sort the results from highest to lowest confidence. Starting with the predicted bounding box with the highest confidence, calculate the IOU with each of the remaining predicted bounding boxes one by one. If the IOU of a predicted bounding box is higher than the set IOU threshold, delete it and retain high-quality detection results.

[0032] Normalize the aspect ratio difference between the retained predicted and ground truth bounding boxes to obtain... :

[0033] in and This represents the width and height of the actual bounding box. and This indicates the height and width of the prediction box;

[0034] Based on the results of the two formulas above, calculate the balance factor that weighs the losses caused by the difference in length-to-width ratio and the losses caused by the Iou portion. :

[0035]

[0036] The weights of the loss function are adjusted by a loss balancing factor, ultimately yielding the algorithm's loss function. :

[0037]

[0038] in This represents the distance between the center of the predicted bounding box and the center of the ground truth bounding box. This represents the diagonal length of the smallest bounding rectangle between the predicted bounding box and the ground truth bounding box.

[0039] The positional relationship between the predicted bounding box and the ground truth bounding box is measured using a loss function. Backpropagation is performed based on the calculated results to optimize and update the algorithm parameters, reducing the error of the prediction results until the calculation results reach the preset effect. The final predicted bounding box is output based on the confidence level and the value of the loss function, marking the detected small target results.

[0040] Compared with existing technologies, the beneficial effects of this invention are as follows: First, this invention preprocesses the image using a backbone network, and simultaneously utilizes the CBAM attention mechanism module to reorganize the feature map, assigning greater weight to key features to highlight the semantic information and detailed features of small targets. Subsequently, the feature map from the backbone network is input into a feature enhancement network, where multiple target detection head modules fully fuse shallow and deep features, identifying candidate boxes for potential target regions. At the same time, depthwise separable convolutions replace ordinary convolutions in the feature enhancement network, thereby reducing the number of parameters, accelerating model inference speed, and improving the receptive field and feature representation ability. Finally, the fused feature map is fed into a target recognition network, and the candidate box positions are corrected based on the results of the positional loss function, resulting in more accurate detection results. This invention can improve the detection accuracy and precision of small targets in UAV aerial images, enabling UAVs to be better applied to urban traffic monitoring. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a schematic diagram of the process of the present invention;

[0043] Figure 2 This is a schematic diagram of the overall framework of the present invention;

[0044] Figure 3 This is a schematic diagram of the optimized and improved CBAM module structure in the backbone network of the present invention;

[0045] Figure 4 This is a schematic diagram of the multi-target detection framework in the feature enhancement network of the present invention;

[0046] Figure 5 This is a schematic diagram of depthwise separable convolution operations in the feature enhancement network of the present invention. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] It should be noted that the structures, proportions, sizes, etc., shown in the accompanying drawings of this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed in the specification, and are not intended to limit the conditions under which this application can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size should still fall within the scope of the technical content disclosed in this application, provided that they do not affect the effects and purposes that this application can produce.

[0049] Example

[0050] In existing technologies, with the increase in car ownership, overlapping targets frequently occur in dense urban road traffic scenes. Due to significant mutual occlusion between targets, missed detections and duplicate detections are highly likely. By improving the existing algorithm, its detection accuracy and precision for small targets can be enhanced.

[0051] Please see Figure 1-5 This invention provides a technical solution: a traffic small target detection method based on YOLOv5 fusion of multi-target feature enhancement network and attention mechanism, comprising the following steps:

[0052] Step 1: Preprocess the image to obtain the main feature data;

[0053] Step 2: Feed the preprocessed feature map into the backbone network for feature extraction and use the CBAM attention mechanism to reorganize the feature channels, giving key features a larger weight to highlight the semantic information and detailed features of small targets.

[0054] Step 3: The feature map after being reconstructed by the backbone network is input into the feature enhancement network. By optimizing the multi-scale feature reconstruction module, a detection head more suitable for small targets is added on the original basis, so that it can detect small target features of 16 pixels. Candidate boxes are marked for possible targets. At the same time, depthwise separable convolution is used to replace ordinary convolution in the feature enhancement network, thereby reducing the number of parameters and accelerating the model inference speed, improving the receptive field and feature expression ability.

[0055] Step 4: The feature map processed by the feature enhancement network is fed into the object detection network. The position of the candidate box is corrected according to the result of the position loss function. At the same time, the non-maximum suppression algorithm is used to suppress and delete redundant candidate boxes, thereby outputting more accurate detection results.

[0056] In step 2, the input images are preprocessed with data augmentation. The processed images are then fed into a deep neural network for training, and the trained model is output. In the backbone network, the Mosaic data augmentation algorithm is used on the input images to stitch four images into one image by randomly flipping, translating, and cropping them, thereby improving the diversity and richness of the data.

[0057] Feature maps of images The input is processed in the backbone network for feature extraction. A CBAM attention module is used to reorganize different channels of the feature map, giving key features a larger weight to highlight the semantic information and detailed features of small targets, and reducing the negative impact of irrelevant features, such as background, on target feature extraction.

[0058] Using global average pooling to process feature maps Dimensionality reduction is performed, where C represents the number of channels, H represents the height, and W represents the width, to obtain the global feature description of each channel. The specific operation process is as follows:

[0059]

[0060] Where C represents the number of channels. This represents feature point information on different channels;

[0061] The fully connected network layer maps the acquired global features and learns the importance weights of each channel. The channel weights are multiplied by the original feature map to adjust the representation of channel features and enhance the representation ability of important channels. The specific operation process is as follows:

[0062]

[0063] in, For the sigmoid function, , To represent the feature maps generated spatially using average pooling and max pooling, and This indicates that two multilayer perceptron networks are used to fuse the results of max pooling and average pooling, respectively.

[0064] After max pooling and average pooling operations along the channel dimension, two different spatial feature representations are obtained. A convolutional operation is then used to fuse these two spatial features. By learning the weights of the convolutional kernels, the correlation between different locations is captured. The learned spatial weights are multiplied with the original feature map to obtain a feature map adjusted for spatial attention, emphasizing the spatial information of important regions. This increases the weight of small target features in aerial images, improving the global perception of small targets. The specific operation process is as follows:

[0065]

[0066] in, For the sigmoid function, The kernel size is Convolution operation, This represents the result of using an MLP network to combine average pooling and max pooling.

[0067] The feature map output in step 3 is fed into the feature enhancement network for multi-scale feature recombination. Based on the original model, a new small target detection head with 4x downsampling is added, enabling it to detect small target features of 16 pixels.

[0068] In step 3, depthwise separable convolutions are used to replace ordinary convolutions in the feature enhancement network. By separating depth information and spatial information and processing them layer by layer, the number of parameters is reduced while accelerating the model's inference speed, improving the receptive field and feature representation ability, controlling overfitting, and saving memory consumption.

[0069] First, depthwise convolution is used to extract shared features from the input features. The extracted features are then used to calculate the spatial attention weights and channel attention weights. The specific operation process is as follows:

[0070]

[0071] in It is the output of a single depth layer. It is the input data. It is a filtering matrix. Represents the position coordinates in the spatial dimension, H and W are the height and width of the filter, c is the index of the input channel, and k is the index of the output channel;

[0072] Then, pointwise convolution is performed using 1x1 kernels at depth to add or multiply the feature maps of different channels element-wise, achieving feature integration and interaction. The specific operation process is as follows:

[0073]

[0074] Where y is the output data, v is the weight matrix, and c and k are the number of input and output channels, respectively.

[0075] Step 4 leads to the final target recognition stage. Based on the position loss function, the candidate box positions are corrected to output more accurate detection results. First, the ratio of the area of ​​the intersection between the predicted and ground truth boxes to the area of ​​the merged boxes is calculated, also known as the intersection-union ratio (Iou).

[0076]

[0077] in This represents the area of ​​the intersection of the two frames. This represents the area of ​​the portion where the two frames meet.

[0078] For each predicted bounding box, first calculate its Intersection over Union (IoU) with all other predicted bounding boxes, and sort the results in descending order of confidence. Starting with the predicted bounding box with the highest confidence, calculate the IoU with each of the remaining predicted bounding boxes one by one. If the IoU of a predicted bounding box is higher than a set IoU threshold, it is deleted, retaining only high-quality detection results.

[0079] Normalize the aspect ratio difference between the retained predicted and ground truth bounding boxes to obtain... :

[0080] in and This represents the width and height of the actual bounding box. and This indicates the height and width of the prediction box;

[0081] Based on the results of the two formulas above, calculate the balance factor that weighs the losses caused by the difference in length-to-width ratio and the losses caused by the Iou portion. :

[0082]

[0083] The weights of the loss function are adjusted by a loss balancing factor, ultimately yielding the algorithm's loss function. :

[0084]

[0085] in This represents the distance between the center of the predicted bounding box and the center of the ground truth bounding box. This represents the diagonal length of the smallest bounding rectangle between the predicted bounding box and the ground truth bounding box.

[0086] The positional relationship between the predicted bounding box and the ground truth bounding box is measured using a loss function. Backpropagation is performed based on the calculated results to optimize and update the algorithm parameters, reducing the error of the prediction results until the calculation results reach the preset effect. The final predicted bounding box is output based on the confidence level and the value of the loss function, marking the detected small target results.

[0087] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A traffic small target detection method based on YOLOv5 fusion of multi-target feature enhancement network and attention mechanism, characterized in that: Includes the following steps: Step 1: Preprocess the image to obtain the main feature data; Step 2: Feed the preprocessed feature map into the backbone network for feature extraction and use the CBAM attention mechanism to reorganize the feature channels, giving key features a larger weight to highlight the semantic information and detailed features of small targets. Step 3: The feature map after being reconstructed by the backbone network is input into the feature enhancement network. By optimizing the multi-scale feature reconstruction module, a detection head more suitable for small targets is added on the original basis, so that it can detect small target features of 16 pixels. Candidate boxes are marked for possible targets. At the same time, depthwise separable convolution is used to replace ordinary convolution in the feature enhancement network, thereby reducing the number of parameters and accelerating the model inference speed, improving the receptive field and feature expression ability. Step 4: The feature map processed by the feature enhancement network is fed into the target detection network. The position of the candidate box is corrected according to the result of the position loss function. At the same time, the non-maximum suppression algorithm is used to suppress and delete redundant candidate boxes, thereby outputting more accurate detection results. In step 2, the input image is preprocessed with data augmentation. The processed image is then fed into a deep neural network for training, and the trained model is output. In the backbone network, the Mosaic data augmentation algorithm is used on the input image to stitch four images into one image by randomly flipping, translating, and cropping the image, thereby improving the diversity and richness of the data. Feature maps of images The input is processed in the backbone network for feature extraction. A CBAM attention module is used to reorganize different channels of the feature map, giving key features a larger weight to highlight the semantic information and detailed features of small targets, and reducing irrelevant features. Using global average pooling to process feature maps Dimensionality reduction is performed, where C represents the number of channels, H represents the height, and W represents the width, to obtain the global feature description of each channel. The specific operation process is as follows: Where C represents the number of channels. Represents the coordinates of feature points on different channels; The fully connected network layer maps the acquired global features and learns the importance weights of each channel. The channel weights are multiplied by the original feature map to adjust the representation of channel features and enhance the representation ability of important channels. The specific operation process is as follows: in, For the sigmoid function, , To represent the feature maps generated spatially using average pooling and max pooling, and This indicates that two multilayer perceptron networks are used to fuse the results of max pooling and average pooling, respectively. After max pooling and average pooling operations along the channel dimension, two different spatial feature representations are obtained. A convolutional operation is then used to fuse these two spatial features. By learning the weights of the convolutional kernels, the correlation between different locations is captured. The learned spatial weights are multiplied with the original feature map to obtain a feature map adjusted for spatial attention, emphasizing the spatial information of important regions. This increases the weight of small target features in aerial images, improving the global perception of small targets. The specific operation process is as follows: in, For the sigmoid function, The kernel size is Convolution operation, This represents the result of using an MLP network to combine average pooling and max pooling. The feature map output in step 3 is fed into the feature enhancement network for multi-scale feature recombination. Based on the original model, a new small target detection head with 4x downsampling is added so that it can detect small target features of 16 pixels. In step 3, depthwise separable convolutions are used to replace ordinary convolutions in the feature enhancement network. By separating depth information and spatial information and processing them layer by layer, the number of parameters is reduced while accelerating the model inference speed, improving the receptive field and feature expression ability, controlling overfitting, and saving memory consumption. First, depthwise convolution is used to extract shared features from the input features. The extracted features are then used to calculate the spatial attention weights and channel attention weights. The specific operation process is as follows: in It is the output of a single depth layer. It is the input data. It is a filtering matrix. Represents the position coordinates in the spatial dimension, H and W are the height and width of the filter, c is the index of the input channel, and k is the index of the output channel; Then, pointwise convolution is performed using 1x1 kernels at depth to add or multiply the feature maps of different channels element-wise, achieving feature integration and interaction. The specific operation process is as follows: Where y is the output data, v is the weight matrix, and c and k are the number of input and output channels, respectively; Step 4 leads to the final target recognition stage. Based on the position loss function, the candidate box positions are corrected to output more accurate detection results. This is achieved by first normalizing the difference in aspect ratio between the predicted and ground truth boxes. : Then calculate the balance factor that weighs the losses caused by the aspect ratio and the losses caused by the IoU portion. : Finally, the loss function of the algorithm is obtained. : 。

Images