Small traffic sign target detection method in complex weather based on improved YOLOv5

Abstract

The application provides a small traffic sign target detection method under complex weather based on an improved YOLOv5, which is used for solving the problem that the detection speed and recognition accuracy of a traffic sign recognition model are unbalanced and small targets and occluded targets are difficult to detect; the steps are as follows: firstly, a data set of traffic sign images is loaded, and the images in the data set are subjected to data enhancement; secondly, an improved YOLOv5 network model is constructed, and model input parameters are set; then, the improved YOLOv5 network model is trained by using the traffic sign image data subjected to data enhancement, so that a detection model is obtained; finally, the detection model is used for predicting an image to be recognized, and the position of the traffic sign and the category to which the traffic sign belongs are output. The application improves the feature extraction capability and detection efficiency by adding a coordinate attention mechanism and a small target detection layer, and further improves the accuracy of small target detection by using an improved loss function.

Description

Technical Field

[0001] The present invention relates to the field of traffic sign detection technology, and in particular to a method for detecting small traffic signs in complex weather conditions based on an improved YOLOv5, which can be applied in fields such as unmanned driving and target detection. Background Art

[0002] In recent years, with the continuous development and improvement of artificial intelligence technology, object detection algorithms have been widely used in fields such as autonomous driving and video indexing. It is crucial for autonomous vehicles to accurately detect and identify vehicles, traffic signs, and traffic lights on the road. Traffic signs on urban roads contain rich navigation information and can guide and warn drivers about their driving behavior. Accurately detecting and identifying traffic signs is crucial for improving road safety and is a key component of intelligent driving.

[0003] Traffic signs around the world follow clear design standards for eye-catching colors and regular shapes. However, in practical applications, the requirements for traffic sign detection accuracy, reliability, and real-time performance remain high, making it a challenging task. There are three main difficulties to overcome. The first is that traffic sign detection is a type of tiny object detection. Such tiny objects usually have only a few pixels, carry limited information, and bring a lot of noise. Therefore, tiny objects pose challenges for feature extraction. The second difficulty comes from complex weather conditions, which can easily lead to false detections and missed detections. Complex weather conditions can interfere with variable lighting, extreme weather (such as rain, snow, or fog), etc. Finally, the categories of traffic signs are always unbalanced because in reality, some signs are commonly used, while others are rarely used.

[0004] Before the widespread application of deep learning, traffic sign recognition tasks typically relied on color, shape, and machine learning-based methods for detection and classification. Most methods used color segmentation based on the AdaBoost binary classifier and cyclic Hough transform to detect traffic signs, which boasts high accuracy and robustness. The emergence of deep learning has provided new insights into traffic sign detection. Deep learning has achieved significant success in the field of traffic sign detection. Unlike traditional detection algorithms, deep learning-based traffic sign detection algorithms build convolutional neural networks (CNNs) to learn traffic sign target features. Traffic sign data samples and label files are then fed into the network through network training. Feature extraction is performed on the image by varying the combination of network layers and the scale of the convolution kernel. The continuous evolution of network parameters enables the algorithm to learn the transformation relationships between different traffic signs, avoiding the large errors associated with manual feature extraction. Traffic sign detection algorithms based on deep learning are primarily categorized into single-stage and two-stage detection. Two-stage detection methods first determine the region of interest by retrieving the approximate location of the object, then use a feature extraction network to determine the coordinates and specific category of the target. Typical examples of this type of algorithm include R-CNN, Fast R-CNN, and Faster R-CNN. Traffic sign detection algorithms based on single-stage detection can implement the entire process from traffic sign image input to classification output within a single network, offering advantages in detection speed. Typical examples of this type of algorithm include the YOLO series and the SSD series.

[0005] Compared to traditional traffic detection algorithms, deep learning-based traffic detection algorithms improve detection accuracy and speed. However, due to the varying sizes of traffic signs, there is a significant gap between detecting small objects and detecting normal objects. Most algorithms still perform poorly for smaller traffic signs. Furthermore, complex weather conditions can lead to missed detections and high false positive rates in most algorithms in inclement weather. Summary of the Invention

[0006] Current traffic sign recognition models suffer from imbalanced detection speed and accuracy under different weather conditions, as well as difficulty detecting occluded and small targets. This paper proposes a small traffic sign target detection method in complex weather conditions based on an improved YOLOv5. This method adds a coordinate attention mechanism and a small target detection layer to YOLOv5 to improve the algorithm's feature extraction capability and detection efficiency. Furthermore, the loss function of the original YOLOv5 target detection network is improved to further enhance the algorithm's accuracy in detecting small targets.

[0007] The technical solution of the present invention is achieved as follows:

[0008] A method for detecting small traffic signs in complex weather conditions based on improved YOLOv5 is proposed. The steps are as follows:

[0009] S1: Load the traffic sign image dataset and perform data augmentation on the images in the dataset;

[0010] S2: Build an improved YOLOv5 network model and set the model input parameters;

[0011] S3: Use the data-enhanced traffic sign image data to train the improved YOLOv5 network model to obtain a detection model;

[0012] S4: Use the detection model to predict the image to be recognized and output the location of the traffic sign and the category to which the traffic sign belongs.

[0013] The data enhancement method includes random horizontal flipping, random vertical flipping, random translation and random cropping.

[0014] The network structure of the improved YOLOv5 model is as follows: the input end of Conv-I is used to receive the input image, the output end of Conv-I is connected to the input end of CBS-I, the output end of CBS-I is connected to the input end of CSP1_1, the output end of CSP1_1 is respectively connected to the input end of CBS-II and the input end of Concat-III, the output end of CBS-II is connected to the input end of CSP1_3-I, the output end of CSP1_3-I is respectively connected to the input end of CBS-III and the input end of Concat-IV, the output end of CBS-III is connected to CSP1_3- The output of CSP1_3-II is connected to the input of CBS-IV and Concat-V respectively. The output of CBS-IV is connected to the input of SPP. The output of SPP is connected to the input of CSP2_1-I. The output of CSP2_1-I is connected to the input of CA. The output of CA is connected to the input of CSB-V. The output of CSB-V is connected to the input of Upsample-I and Concat-VI respectively. The output of Upsample-I is connected to the input of Concat-I. The output of cat-I is connected to the input of CSP2_1-II, the output of CSP2_1-II is connected to the input of CBS-VI, the output of CBS-VI is connected to the input of Upsample-II and Concat-IV respectively, the output of Upsample-II is connected to the input of Concat-II, the output of Concat-II is connected to the input of CSP2_1-III, the output of CSP2_1-III is connected to the input of CBS-VII, the output of CBS-VII is connected to Upsample-III The output of Concat-III is connected to the input of CSP2_1-IV, the output of Concat-III is connected to the input of CSP2_1-IV, the output of CSP2_1-IV is connected to the input of Conv-II, and the output of Conv-II outputs Head-I; the output of Concat-IV is connected to the input of CSP2_1-V, the output of CSP2_1-V is connected to the input of Conv-III and the input of CBS-VIII respectively, and the output of Conv-III outputs Head-II;The output of CBS-VIII is connected to the input of Concat-V, which is connected to the input of CSP2_1-VI. The output of CSP2_1-VI is connected to the input of Conv-IV and the input of CBS-IX respectively. The output of Conv-IV outputs Head-III. The output of CBS-IX is connected to the input of Concat-VI, which is connected to the input of CSP2_1-VII. The output of CSP2_1-VII is connected to the input of Conv-V, which outputs Head-IV.

[0015] The Head-I is a structure output by the small target detection layer.

[0016] The loss function when training the improved YOLOv5 network model is:

[0017]

[0018] Among them, Alpha_IoU is the loss function value, IoU represents the similarity between the bounding box and the actual bounding box, α is the additional Power regularization term, b represents the predicted box, c represents the number of categories in the data set, and b gt represents the ground-truth box.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] 1) The present invention adds a coordinate attention (CA) mechanism to the backbone network, embedding location information into channel attention, enabling the network to obtain a wider range of regional information and avoiding the increase of parameters and computational complexity as much as possible.

[0021] 2) Add a small target detection layer to reduce missed detection of small targets and improve the accuracy of small target detection.

[0022] 3) The original positioning loss function CIoU is improved to achieve more accurate loss calculation between the predicted box and the true box in the traffic sign recognition task.

[0023] 4) The present invention can effectively detect smaller traffic signs under different weather conditions, with a low missed detection rate and high detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0025] Figure 1 It is a flow chart of the present invention.

[0026] Figure 2 This is the overall framework structure diagram of the improved YOLOv5 of the present invention.

[0027] Figure 3 This is the structural diagram of the coordinate attention module in the present invention.

[0028] Figure 4 This is a structural diagram of the small target detection layer in the present invention. DETAILED DESCRIPTION

[0029] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without creative work are within the scope of protection of the present invention.

[0030] like Figure 1 As shown, the embodiment of the present invention provides a method for detecting small traffic signs in complex weather conditions based on an improved YOLOv5, and the specific steps are as follows:

[0031] S1: Load the traffic sign image dataset and perform data augmentation on the images in the dataset; resize the images in the traffic sign image dataset to a uniform size of 640×640 as input to the improved YOLOv5 model.

[0032] The CCTSDB 2021 dataset was selected as the traffic sign detection dataset, which contains a total of 3268 training set images and 2000 test set images. CCTSDB 2021 divides the samples in the test set in more detail according to three dimensions: category meaning, size, and weather conditions. According to the definition of common traffic signs in road traffic signs and markings, the signs appearing in the benchmark dataset are divided into three categories according to their meaning: prohibition signs, mandatory signs, and warning signs. Unlike traditional traffic detection datasets, CCTSDB 2021 divides traffic signs into five categories according to size: extremely small (XS), small (S), medium (M), large (L), and extra large (XL). CCTSDB 2021 focuses more on the detection of small objects. According to the weather and lighting conditions of the sample images, the dataset is divided into six categories: fog, snow, rain, night, clear, and cloud.

[0033] Data augmentation is performed on the images in the dataset to prevent overfitting. The data augmentation method includes random horizontal flipping, random vertical flipping, random translation, and random cropping of the input image with a probability of 0.5.

[0034] S2: An improved YOLOv5 network model was constructed using the Pytorch deep learning framework, and model input parameters were set; these parameters include convolution kernel size, stride, and padding. The original Focus module was removed from the YOLOv5 model and replaced with a large convolutional layer to further reduce module size and achieve better performance. Furthermore, a coordinate attention (CA) mechanism was introduced at the end of the YOLOv5 model. Due to the unique location information of traffic signs, this coordinate attention mechanism effectively improves network performance. The enhanced feature extraction network uses spatial pyramid pooling (SPP). The convolutional layer is concatenated into a one-dimensional vector through three max-pooling operations of different sizes to accommodate input images of varying sizes. The FPN from PANet was used for feature fusion to extract higher-quality features, transforming the previously valid initial feature layer into a higher-quality feature layer for prediction. Among the convolutional operations, 3×3 depthwise separable convolutions were used instead of 3×3 standard convolutions to further reduce model weight.

[0035] like Figure 2As shown in Figure 1, the network structure of the improved YOLOv5 model is as follows: the input end of Conv-I is used to receive the input image, the output end of Conv-I is connected to the input end of CBS-I, the output end of CBS-I is connected to the input end of CSP1_1, the output end of CSP1_1 is connected to the input end of CBS-II and the input end of Concat-III, the output end of CBS-II is connected to the input end of CSP1_3-I, the output end of CSP1_3-I is connected to the input end of CBS-III and the input end of Concat-IV, and the output end of CBS-III is connected to CSP1_3- The output of CSP1_3-II is connected to the input of CBS-IV and Concat-V respectively. The output of CBS-IV is connected to the input of SPP. The output of SPP is connected to the input of CSP2_1-I. The output of CSP2_1-I is connected to the input of CA. The output of CA is connected to the input of CSB-V. The output of CSB-V is connected to the input of Upsample-I and Concat-VI respectively. The output of Upsample-I is connected to the input of Concat-I. The output of cat-I is connected to the input of CSP2_1-II, the output of CSP2_1-II is connected to the input of CBS-VI, the output of CBS-VI is connected to the input of Upsample-II and Concat-IV respectively, the output of Upsample-II is connected to the input of Concat-II, the output of Concat-II is connected to the input of CSP2_1-III, the output of CSP2_1-III is connected to the input of CBS-VII, the output of CBS-VII is connected to Upsample-III The output of Concat-III is connected to the input of CSP2_1-IV, the output of Concat-III is connected to the input of CSP2_1-IV, the output of CSP2_1-IV is connected to the input of Conv-II, and the output of Conv-II outputs Head-I; the output of Concat-IV is connected to the input of CSP2_1-V, the output of CSP2_1-V is connected to the input of Conv-III and the input of CBS-VIII respectively, and the output of Conv-III outputs Head-II;The output of CBS-VIII is connected to the input of Concat-V, which is connected to the input of CSP2_1-VI. The output of CSP2_1-VI is connected to the input of Conv-IV and the input of CBS-IX respectively. The output of Conv-IV outputs Head-III. The output of CBS-IX is connected to the input of Concat-VI, which is connected to the input of CSP2_1-VII. The output of CSP2_1-VII is connected to the input of Conv-V, which outputs Head-IV.

[0036] The network structure of CSP1_X includes CBS-X, CBS-XI, Resunit component, Concat-VII, and CBS-XII. The input data of CSP1_X is input to CBS-X and CBS-XI respectively. The output of CBS-X is connected to the input of Resunit component. The output of Resunit component and the output of CBS-XI are connected to the input of Concat-VII. The output of Concat-VII is connected to the input of CBS-XII. The output of CBS-XII is used to output the output of CSP1_X.

[0037] The Resunit component includes X Resunits, the input ends of the X Resunits are all connected to the output ends of CBS-X, and the output ends of the X Resunits are all connected to the input ends of Concat-VII.

[0038] The network structure of CSP2_1 includes CBS-XIII, CBS-XIV, CBS-XV, Concat-VIII, and CBS-XVI. The input data of CSP2_1 is input into CBS-XIII and CBS-XV respectively. The output end of CBS-XIII is connected to the input end of CBS-XIV. The output ends of CBS-XIV and CBS-XV are both connected to the input end of Concat-VIII. The output end of Concat-VIII is connected to the input end of CBS-XVI. The output end of CBS-XVI is used to output the output result of CSP2_1.

[0039] The network structure of the SPP includes CBS-XVII, MaxPool-I, MaxPool-II, MaxPool-III, Concat-IX, and CBS-XVIII; the input data of the SPP is input into CBS-XVII, the output end of CBS-XVII is respectively connected to the input ends of MaxPool-I, MaxPool-II, MaxPool-III, and Concat-IX, the output ends of MaxPool-I, MaxPool-II, and MaxPool-III are all connected to the input end of Concat-IX, the output end of Concat-IX is connected to the input end of CBS-XVIII, and the output end of CBS-XVIII is used to output the output result of the SPP.

[0040] The network structure of CBS is Conv-BN-SiLU.

[0041] The input image size of this embodiment is 640×640. First, shallow feature extraction is performed on the input image. Then, the coordinate attention mechanism is used to lock the candidate area of ​​focus. As a new and efficient attention mechanism, the main idea of ​​coordinate attention is to embed position information into channel attention, which can make the model more accurate in locating and identifying important information. The structure of the coordinate attention module is as follows: Figure 3 As shown. First, the position information is embedded in the channel attention, so that the lightweight network can obtain information from a larger area, reducing the number of parameters in the attention module while avoiding excessive computational overhead. In the process of embedding coordinate information, the input in the X direction is encoded along the horizontal and vertical coordinate directions using pooling kernels of size (H, 1) and (1, W). H and W are the height and width of the input feature map corresponding to the current attention module, respectively. The value will change with the downsampling of the network. The output of the cth channel with a height of h is shown in Formula 1:

[0042]

[0043] Similarly, the output of the cth channel with width w is shown in Formula 2:

[0044]

[0045] Two one-dimensional global pooling operations are used to aggregate the input features in the horizontal and vertical directions into two independent direction-aware feature maps, and respectively encode them into two attention maps. Each attention map contains the long-range dependency of the input feature map along one spatial direction and saves the precise position information along the other spatial direction, allowing the network to more accurately obtain the region of interest. Then, in order to better utilize the global receptive field generated by the coordinate information embedding and have precise position, the output tensors of the average pooling in the X and Y directions are spliced ​​and F1 transformed through a shared 1×1 convolution operation, as shown in Formula 3. The generated It is the intermediate feature map of spatial information in the horizontal and vertical directions. r represents the step size of downsampling, which is used to control the size of the module like the SE module.

[0046] f=δ(F1([z h ,z w ])) (3)

[0047] After normalization and nonlinear processing, f is split into two independent tensors along the spatial dimension and Then use two 1×1 convolution F h and F w The feature map f h and f W They are transformed to the same number of channels as the X input, as shown in Equations 4 and 5.

[0048] g h =δ(F h (f h )) (4)

[0049] g w =δ(F w (f w )) (5)

[0050] Here, σ represents the sigmoid activation function, which is used to reduce model complexity and computational overhead. Finally, the obtained results are expanded and the final attention weight matrix is ​​obtained using matrix multiplication. The final output of the coordinate attention module is shown in Equation 6.

[0051]

[0052] In order to extract more shallow targets to improve the detection effect of small traffic signs, a detection layer for small targets is added to the network, that is, the small target detection layer. The specific model structure is as follows Figure 4As shown in the figure, after the last CBS module, the feature map is continued to be upsampled and processed so that the feature map continues to expand, and the obtained feature map of size 160X160 is concat fused with the second layer feature map in the backbone network to obtain a larger feature map for small target detection. That is, Head-I is a structure output by the small target detection layer. The present invention uses four yolo heads to predict the bounding boxes and categories of traffic signs of four different scales. Each scale has three anchor points as a priori. The tensor of a yolo head is M×M×[3×(N+4+1)], which is used for N class predictions, 4 bounding box offsets and 1 object prediction. In this embodiment, N is equal to 3, so the tensor of a yolo head is equal to 24. These improvements allow the network to obtain more fine-grained information from the original feature map, more abstract semantic information from the depth feature map, and more dense anchor sampling, effectively improving the detection accuracy of small targets.

[0053] S3: The improved YOLOv5 network model is trained using data-augmented traffic sign image data to obtain a detection model. During training, the original localization loss CIoU is improved using Alpha-IoU. By adjusting α, the loss and gradient of objects with high IoU are increased, thereby improving the bounding box regression accuracy. This achieves more accurate loss calculation between the predicted and ground-truth boxes in the traffic sign recognition task.

[0054] The improved loss function is:

[0055]

[0056] Among them, Alpha_IoU is the loss function value, IoU represents the similarity between the bounding box and the actual bounding box, α is the additional Power regularization term, b represents the predicted box, c represents the number of categories in the data set, and b gt In this embodiment, α=3.

[0057] S4: Use the detection model to predict the image to be recognized and output the location of the traffic sign and the category to which the traffic sign belongs.

[0058] This paper adds an attention mechanism and a small target detection layer to YOLOv5, solving the problems of the existing traffic sign recognition model, such as the imbalance between detection speed and recognition accuracy, as well as the difficulty in detecting occluded and small targets. It also improves the algorithm's feature extraction capability and detection efficiency. When this model was applied to the CCTSDB 2021 dataset, the recognition accuracy for small targets reached 88.1% and the recall rate reached 79.8%, respectively, improvements of 12.5% ​​and 23.9% compared to the traditional YOLOv5s model. It can effectively detect smaller traffic signs in all weather conditions, with a low missed detection rate and high detection accuracy.

[0059] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A small traffic sign target detection method in complex weather conditions based on improved YOLOv5, characterized in that: The steps are as follows: S1: Load the traffic sign image dataset and perform data augmentation on the images in the dataset; S2: Build an improved YOLOv5 network model and set the model input parameters; S3: Use the data-enhanced traffic sign image data to train the improved YOLOv5 network model to obtain a detection model; S4: Use the detection model to predict the image to be recognized and output the location of the traffic sign and the category to which the traffic sign belongs; The network structure of the improved YOLOv5 network model is as follows: the input end of Conv-I is used to receive the input image, the output end of Conv-I is connected to the input end of CBS-I, the output end of CBS-I is connected to the input end of CSP1_1, the output end of CSP1_1 is respectively connected to the input end of CBS-II and the input end of Concat-III, the output end of CBS-II is connected to the input end of CSP1_3-I, the output end of CSP1_3-I is respectively connected to the input end of CBS-III and the input end of Concat-IV, and the output end of CBS-III is connected to CSP1_3 -II, the output of CSP1_3-II is connected to the input of CBS-IV and the input of Concat-V respectively, the output of CBS-IV is connected to the input of SPP, the output of SPP is connected to the input of CSP2_1-I, the output of CSP2_1-I is connected to the input of CA, the output of CA is connected to the input of CSB-V, the output of CSB-V is connected to the input of Upsample-I and the input of Concat-VI respectively, the output of Upsample-I is connected to the input of Concat-I, Co The output of ncat-I is connected to the input of CSP2_1-II, the output of CSP2_1-II is connected to the input of CBS-VI, the output of CBS-VI is connected to the input of Upsample-II and Concat-IV respectively, the output of Upsample-II is connected to the input of Concat-II, the output of Concat-II is connected to the input of CSP2_1-III, the output of CSP2_1-III is connected to the input of CBS-VII, the output of CBS-VII is connected to Upsample-II The output of Concat-III is connected to the input of CSP2_1-IV, the output of CSP2_1-IV is connected to the input of Conv-II, and the output of Conv-II outputs Head-I; the output of Concat-IV is connected to the input of CSP2_1-V, the output of CSP2_1-V is connected to the input of Conv-III and the input of CBS-VIII respectively, and the output of Conv-III outputs Head-II;The output of CBS-VIII is connected to the input of Concat-V, which is connected to the input of CSP2_1-VI. The output of CSP2_1-VI is connected to the input of Conv-IV and the input of CBS-IX respectively. The output of Conv-IV outputs Head-III. The output of CBS-IX is connected to the input of Concat-VI, which is connected to the input of CSP2_1-VII. The output of CSP2_1-VII is connected to the input of Conv-V, which outputs Head-IV.

2. The small traffic sign target detection method under complex weather conditions based on improved YOLOv5 according to claim 1 is characterized in that: The data enhancement includes random horizontal flipping, random vertical flipping, random translation and random cropping.

3. The small traffic sign target detection method under complex weather conditions based on improved YOLOv5 according to claim 1 is characterized in that The Head-I is a structure output by the small target detection layer.

4. The method for detecting small traffic signs in complex weather based on improved YOLOv5 according to claim 1, characterized in that: The loss function when training the improved YOLOv5 network model is: ； in, is the loss function value, IoU represents the similarity between the bounding box and the actual bounding box, is the additional Power regularization term, b represents the prediction box, c Represents the number of categories in the dataset, represents the ground-truth box.

Images