A real-time detection method for small target traffic signs based on improved YOLOv5s

Abstract

The present invention discloses a small target traffic sign real-time detection method based on improved YOLOv5s, which specifically includes the following steps: (1) establishing a traffic sign image data set; (2) adding annotation information to the images in the data set; (3) constructing a small target traffic sign real-time detection model based on improved YOLOv5s; (4) adopting a new loss function; (5) using a training set and a validation set to train the model; (6) using a test set to test the model to meet the accuracy requirements, that is, to obtain the final small target traffic sign real-time detection model. Compared with the prior art, the small target traffic sign real-time detection method based on improved YOLOv5s disclosed by the present invention can effectively improve the detection accuracy of small target traffic signs, and also greatly improve the real-time performance of traffic sign detection.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a real-time detection method for small-target traffic signs. Background Art

[0002] With the continuous development of autonomous driving technology, traffic sign recognition (TSR) has become one of the key components of advanced driver assistance systems (ADAS). TSR can efficiently and accurately recognize images captured by the camera and provide drivers with instructions, warnings and other information in real time. In actual road driving, the proportion of traffic signs in the image when they first enter the field of view is usually less than 1%, which belongs to small target detection. Small target traffic signs are blurred and have unclear details in the image. Therefore, the rapid and accurate recognition of small target traffic signs is of great significance to the development of assisted driving systems.

[0003] Traditional traffic sign detection methods mainly focus on color segmentation, shape, contour and other features for feature extraction, and then complete feature classification through classifiers to achieve the detection and recognition of road traffic signs. Traditional traffic sign detection algorithms use manual methods to extract features, and then use these features as input to the algorithm. These detection algorithms have high dimensions, large computational complexity, and are time-consuming and labor-intensive. In the process of traffic sign detection, there are poor real-time, stability and accuracy.

[0004] With the rapid development of deep learning, the object detection technology of improving convolutional neural models has become a hot topic in current research. As a mainstream deep learning algorithm, convolutional neural network (CNN) includes R-CNN, Fast-R-CNN, Fast-R-CNN, Mask-R-CNN, Alex Net, YOLO and other models. These models have achieved good results in object detection in different application scenarios. Object detection algorithms can be divided into two categories: two-stage detection algorithms using regional candidate boxes and single-stage detection algorithms without regional candidate boxes. The two-stage algorithm has high accuracy but slow detection speed. The improved algorithm has improved the speed to a certain extent, but due to the setting of candidate areas, the two-stage model cannot meet the real-time detection requirements of small target traffic signs during high-speed driving. Therefore, the single-stage algorithm is usually used in traffic sign recognition. In terms of single-stage object detection algorithms, although the real-time performance is improved compared with the two-stage detection algorithm, the accuracy is lower. Especially in the recognition and detection of small target traffic signs, the detection accuracy is not high, which easily leads to missed detection and false detection. In addition, due to the large number of model parameters and high computational complexity, the real-time detection is not good, and it is difficult to migrate the model to a mobile device. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention provides a real-time detection method for small target traffic signs based on improved YOLOv5s. By improving the YOLOv5s target detection model, utilizing feature enhancement and adopting the NWD measurement method, the problems of low detection accuracy of small target traffic signs, poor real-time performance, large number of model parameters, high computational complexity and unsuitability for deployment on mobile devices are solved.

[0006] The technical solution provided by the present invention comprises the following steps:

[0007] Step 1: Acquire traffic sign images to form a first data set;

[0008] Step 2: adding annotation information to the images in the first data set to form a second data set, and dividing the second data set into a training set, a validation set, and a test set;

[0009] Step 3: Build a traffic sign detection model based on improved YOLOv5s;

[0010] Step 4: Use a new loss function;

[0011] Step 5: Using the training set and the validation set to train the traffic sign detection model based on the improved YOLOv5s;

[0012] Step 6: Use the traffic sign test set to test the optimal model obtained through training, and obtain the final small target traffic sign real-time detection model.

[0013] Furthermore, in step 1, the traffic sign images in the first data set may be photographed by a digital camera, collected from the Internet, or obtained from surveillance videos.

[0014] Preferably, in step 2, the training set, the validation set and the test set may be divided in a ratio of 7:2:1.

[0015] Furthermore, the step 3 specifically includes steps 3.1 to 3.3:

[0016] Step 3.1: Combine the C3 module in the YOLOv5s model with ConvMixer to form a CSPCM module, and use the CSPCM module to replace the C3 module in the last layer of the YOLOv5s backbone network and the C3 module in the last layer of the neck network respectively;

[0017] Step 3.2: Replace the remaining C3 modules in the backbone network and the neck network of the YOLOv5s model with the lightweight convolution module C3_Faster, that is, use C3_Faster to replace the remaining C3 modules except the C3 modules of the last layer of the backbone network and the neck network that have been replaced in step 3.1;

[0018] Step 3.3: The output layer adds a small target detection head based on the existing three detection heads.

[0019] Furthermore, in step 4, the NWD measurement method is introduced into the CIoU loss function, and the CIoU loss function of YOLOv5s is optimized using the NWD measurement. The optimized loss function formula is:

[0020] L=(1-β)*(1-NWD(N a , N b ))+β*(1-CIoU) (1)

[0021] In formula (1), L is the optimized loss function, NWD is the normalized Wasserstein distance, and N a , N b Is and The Gaussian distribution of the model, a represents the real box, b represents the predicted box, cx a ,,cy a Represents the center point coordinates of the real box, w a 、h a Represents the width and height of the real box; cxb ,cy b Represents the center point coordinates of the prediction box, w b ,h b Represents the width and height of the prediction box; β is the weight ratio coefficient, CIoU is the loss function in the original YOLOv5s, and its calculation formula is:

[0022]

[0023] In formula (2), ρ 2 (b A ,b B ) represents the Euclidean distance between the center point of the real box and the predicted box, c represents the diagonal distance of the minimum bounding rectangle of the predicted box and the real box, α is the weight factor, v is the aspect ratio consistency, and IoU is the overlap ratio between the real box and the predicted box.

[0024] Furthermore, the step 5 specifically includes steps 5.1 to 5.4:

[0025] Step 5.1: Setting the training parameters of the small target traffic sign real-time detection model based on improved YOLOv5s, the model training parameters specifically include: learning rate, momentum, weight decay, optimizer, number of iterations, and batch size;

[0026] Step 5.2: Input the training set and validation set images and their corresponding labels into the improved YOLOv5s small target traffic sign real-time detection model, use the back propagation algorithm to calculate the gradient of the loss function to the model parameters, and adjust the model parameters by minimizing the loss function to gradually approach the optimal solution;

[0027] Step 5.3: Use the optimizer to update the model parameters in the direction of gradient descent until the loss function of the training set and the validation set no longer decreases, and the evaluation indicators such as accuracy P, recall R, and mAP no longer improve;

[0028] Step 5.4: Save the trained model parameters as the optimal model.

[0029] Furthermore, the step 6 specifically includes steps 6.1 to 6.3:

[0030] Step 6.1: Input the test set into the improved optimal model described in step 5;

[0031] Step 6.2: Calculate model performance indicators: The performance indicators include accuracy P (Precision), recall R, mAP, number of parameters, computational complexity (GFLOPs), and model size. The specific formula is as follows:

[0032]

[0033]

[0034]

[0035]

[0036] Among them, P is the precision, R is the recall, mAP is the average precision of all categories, AP is the average precision, m is the total number of traffic sign categories, TP is the number of positive samples correctly identified as positive samples, FP is the number of negative samples incorrectly identified as positive samples, and FN is the number of positive samples incorrectly identified as negative samples;

[0037] Step 6.3: When the performance index meets the accuracy requirements, the final small target traffic sign real-time detection model based on improved YOLOv5s is obtained.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] The present invention discloses a real-time detection method for small target traffic signs based on improved YOLOv5s, which improves the detection accuracy of small target traffic signs by introducing a CSPCM module and adding a small target detection head; adopts a C3_Faster network structure to reduce model parameters and calculation complexity, and improves the real-time performance of traffic sign detection; introduces an NWD measurement method into a CIoU loss function, adopts a new loss function, effectively avoids the large loss caused by small targets, and accelerates the convergence of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flow chart of the real-time detection method of small target traffic signs based on improved YOLOv5s of the present invention;

[0041] Figure 2 This is a schematic diagram of the model structure of the small target traffic sign real-time detection method based on improved YOLOv5s of the present invention;

[0042] Figure 3 It is a schematic diagram of the CSPCM structure;

[0043] Figure 4 This is a schematic diagram of the C3_Faster structure; DETAILED DESCRIPTION

[0044] In order to make the technical solution, structural features, achieved purposes and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific implementation methods and accompanying drawings. It should be noted that the specific embodiments described herein are only used to more clearly explain the present invention and are not used to limit the present invention.

[0045] Figure 1 This is a flow chart of a small target traffic sign real-time detection method based on improved YOLOv5s disclosed in the present invention, and its implementation process is as follows:

[0046] Step 1: Obtain traffic sign images to form a first data set; the traffic sign images in the first data set can be photographed by a digital camera, collected from the Internet, or obtained from surveillance videos;

[0047] In this embodiment, in order to better evaluate the detection effect of a small target traffic sign real-time detection method based on improved YOLOv5s disclosed in the present invention, a public data set TT100K (Tsinghua-Tencent 100K) traffic sign data set is adopted. Using the TT100K data set, the data set needs to be preprocessed: first, the traffic sign classes with less than 200 appearances in the TT100K data set are counted, and the labels corresponding to the traffic sign classes with less than 200 appearances and the pictures containing only the class are deleted, and finally a data set containing 36 types of traffic signs is made, that is, the first data set is formed.

[0048] Step 2: Use the labelme annotation tool to add annotation information to the images in the first data set. Since the public data set TT100K used in this embodiment already has annotation information, this step is skipped. Convert the label file in json format in the first data set described in this embodiment to the label file in txt format required by YOLOv5s to form a second data set, and divide the second data set into training set, validation set and test set according to 7:2:1; the divided training set includes 6598 pictures, the test set includes 1889 pictures, and the validation set includes 970 pictures. In this embodiment, the image size of the second data set is 640*640.

[0049] Step 3: Build a traffic sign detection model based on improved YOLOv5s. The improved YOLOv5s model structure is as follows: Figure 2 As shown, the model building process specifically includes steps 3.1 to 3.3:

[0050] Step 3.1: Replace the Bottleneck module in the C3 module with the ConvMixer module to form a new convolutional module CSPCM; replace the C3 module in the last layer of Backbone and the C3 module in the last layer of Neck with the new convolutional CSPCM module;

[0051] Furthermore, the structure of the ConvMixer module is as follows Figure 3As shown in the figure, it operates in a hybrid way that separates spatial and channel dimensions and maintains the same size and resolution throughout the model; in ConvMixer, the input first passes through a Depthwise Conv (Dwconv), which is a grouped convolution with the number of groups equal to the number of channels; then passes through a PointwiseConv (PWCONV), which is a 1×1 convolution; each convolution operation is followed by an activation function z′1 and a z 1+ 1BatchNorm layer. This design can effectively reduce the number of model parameters, improve the real-time performance of the model, and effectively improve the model's feature expression capabilities.

[0052] Step 3.2: Use the lightweight convolution module C3_Faster to replace the remaining C3 modules in the backbone network and the neck network of the YOLOv5s model, that is, use C3_Faster to replace the remaining C3 modules except the C3 modules of the last layer of the backbone network and the neck network that have been replaced in step 3.1;

[0053] The lightweight convolution module C3_Faster uses Pconv convolution; compared with deep convolution (DWConv) and group convolution (GConv), PConv can reduce the amount of FLOPs calculation, reduce computational redundancy and memory access, Figure 4 The structure of C3_Faster consists of a PConv layer followed by 2 Conv layers, which together appear as an inverted residual block. The intermediate layers have an expanded number of channels. After each intermediate Conv, there is a normalization and activation layer.

[0054] Step 3.3: The output layer adds a small target detection head on the basis of the existing three detection heads, which can process multi-scale feature information more efficiently, making the detector more sensitive to small targets and improving the detection performance of small target traffic signs.

[0055] Step 4: Use a new loss function, introduce the NWD measurement method into the CIoU loss function, combine CIoU with the NWD measurement, and propose a new loss function; the new loss function can effectively avoid the large loss caused by small targets and accelerate the convergence of the model. The specific steps include steps 4.1 to 4.3:

[0056] Step 4.1: Model the bounding box as a two-dimensional Gaussian distribution, and use the Gaussian distribution corresponding to the predicted target and the actual target to calculate the Wasserstein distance between the predicted box and the true box. The calculation formula is:

[0057]

[0058] In formula (1), is the Wasserstein distance, N a ,N b Is and The Gaussian distribution of the model, a represents the real box, b represents the predicted box, cx a ,,cy a Represents the center point coordinates of the real box, w a 、h a Represents the width and height of the real box; cx b ,cy b Represents the center point coordinates of the prediction box, w b ,h b Represents the width and height of the prediction box;

[0059] Step 4.2: Calculate the normalized Wasserstein distance between them, the calculation formula is:

[0060]

[0061] In formula (2), NWD(N a , N b ) is the normalized Wasserstein distance, and C is a constant closely related to the dataset. Step 4.3: According to the proportional relationship between CIoU and NWD, a new loss function is proposed, and its calculation formula is:

[0062] L=(1-β)*(1-NWD(N a , N b ))+β*(1-CIoU) (3)

[0063] In formula (3), L is the optimized loss function, β is the weight ratio coefficient, CIoU is the loss function in the original YOLOv5s, and its calculation formula is:

[0064]

[0065] In formula (4), ρ 2 (b A , b B ) represents the Euclidean distance between the center point of the real box and the predicted box, c represents the diagonal distance of the minimum bounding rectangle of the predicted box and the real box, α is the weight factor, v is the aspect ratio consistency, and Intersection over Union (IoU) is used by target detection to measure the degree of overlap between the predicted bounding box and the real bounding box. The higher the IoU value, the greater the overlap between the predicted box and the real box, and the better the detection effect. The IoU calculation formula is:

[0066]

[0067] In formula (5), B is the prediction box; GT For the real frame.

[0068] Step 5: Input the training set and the validation set into the small target traffic sign model based on the improved YOLOv5s described in step 3 for training, specifically including steps 5.1 to 5.4:

[0069] Step 5.1: Set the training parameters of the small target traffic sign real-time detection model based on improved YOLOv5s, the model training parameters include: learning rate, momentum, weight decay, optimizer, number of iterations, batch size;

[0070] In this embodiment, the optimizer is SGD, the initial learning rate 1r0 is 0.01, the momentum is 0.937, the weight decay weight_decay is 0.0005, the batch size batchsize is 16, and the iteration round Epoch is 300.

[0071] Step 5.2: Input the training set and validation set images and the corresponding labels into the improved YOLOv5s small target traffic sign real-time detection model, and use the back propagation algorithm to calculate the gradient of the loss function to the model parameters. The back propagation algorithm is an effective method for calculating gradients, which uses the chain rule to calculate the gradient of each parameter to the loss function. Specifically, back propagation calculates the gradient of each parameter layer by layer by propagating the loss function backward from the output layer. In this process, the gradient of each parameter represents the rate of change of the loss function with respect to the parameter, that is, how the loss function changes with the change of the parameter. By minimizing the loss function, the model parameters are adjusted to gradually approach the optimal solution.

[0072] Step 5.3: After calculating the gradient of the model parameters, use the optimizer to update these parameters. The optimizer updates the parameters in the opposite direction of the gradient based on the gradient information of the parameters. Parameters with larger gradients are updated at a larger pace, while parameters with smaller gradients are updated at a smaller pace. By continuously iterating and updating the model parameters, the value of the loss function can be gradually reduced. By minimizing the loss function, the values ​​of the model parameters are adjusted so that they gradually approach the optimal solution, that is, the parameter values ​​at which the loss function reaches the minimum value, the difference between the model's prediction results and the true value is minimized, and at the same time, evaluation indicators such as mAP, recall rate R, and precision rate P no longer improve.

[0073] Step 5.4: Save the trained model parameters as the optimal model.

[0074] Step 6: Use the test set to test the optimal model described in step 5, evaluate the test set test results, and meet the accuracy requirements, that is, obtain the final small target traffic sign real-time detection model based on improved YOLOv5s. Specifically, step 6 further includes steps 6.1 to 6.3:

[0075] Step 6.1: Input the test set into the optimal model described in step 5;

[0076] Step 6.2: Calculate model performance indicators: The performance indicators include precision P, recall R, mAP, number of parameters, computational complexity GFLOPs, and model size. The specific calculation formula is as follows:

[0077]

[0078]

[0079]

[0080]

[0081] Among them, P is the precision, R is the recall, mAP is the average precision of all categories, AP is the average precision, m is the total number of traffic sign categories, TP is the number of positive samples correctly identified as positive samples, FP is the number of negative samples incorrectly identified as positive samples, and FN is the number of positive samples incorrectly identified as negative samples;

[0082] Step 6.3: When the performance index meets the accuracy requirements, the final small target traffic sign real-time detection model based on improved YOLOv5s is obtained.

[0083] In this embodiment, in order to verify the effect of the improved model disclosed in the present invention, the test set is input into the small target traffic sign real-time detection model based on the improved YOLOv5s and the YOLOv5s model for testing. Evaluation indicators are calculated for the two models respectively, and the evaluation indicator data are shown in Table 1. As can be seen from Table 1, the small target traffic sign real-time detection model disclosed in the present invention is better than the original YOLOv5s model in terms of accuracy P, mAP@0.5 In terms of performance indicators such as model parameter quantity, model size, and computational complexity GFLOPs, the improved model disclosed by the present invention is lower than the original YOLOv5s model, so that the model has better lightweight indicators and is easier to deploy on mobile devices.

[0084] Table 1 Comparative experimental results

[0085]

[0086] The above description is only one embodiment of the present invention and does not limit the patent scope of the present invention. For those skilled in the art, the present invention may have various changes and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A real-time detection method for small target traffic signs based on improved YOLOv5s, characterized in that: The specific steps include: Step 1: Obtain traffic sign images to form a first data set; in the first data set, the traffic sign images can be photographed by a digital camera, collected from the Internet, or obtained from surveillance videos; Step 2: adding annotation information to the images in the first data set to form a second data set, and dividing the second data set into a training set, a validation set, and a test set; Step 3: Construct a small target traffic sign real-time detection model based on improved YOLOv5s, and the construction of the model further includes steps 3.1 to 3.3: Step 3.1: Combine the C3 module in the YOLOv5s model with ConvMixer to form a CSPCM module, and use the CSPCM module to replace the C3 module in the last layer of the YOLOv5s backbone network and the C3 module in the last layer of the neck network respectively; Step 3.2: Replace the remaining C3 modules in the backbone network and the neck network of the YOLOv5s model with the lightweight convolution module C3_Faster, that is, use C3_Faster to replace the remaining C3 modules except the C3 modules of the last layer of the backbone network and the neck network that have been replaced in step 3.1; Step 3.3: The output layer adds a small target detection head based on the existing three detection heads; Step 4: Use a new loss function. The specific method is: The NWD measurement method is introduced into the CIoU loss function, and the CIoU loss function of YOLOv5s is optimized using the NWD measurement. The optimized loss function formula is: L=(1-β)*(1-NWD(N a ,N b ))+β*(1-CIoU) (1) NWD is the normalized Wasserstein distance, N a ,N b Is and The Gaussian distribution of the model, a represents the real box, b represents the predicted box, cx a ,cy a Represents the center point coordinates of the real box, w a 、h a Represents the width and height of the real box; cx b ,cy b Represents the center point coordinates of the prediction box, w b ,h b Represents the width and height of the prediction box; β is the weight ratio coefficient, CIoU is the loss function in the original YOLOv5s, and the CIoU calculation formula is: In formula (2), ρ 2 (b A ,b B ) represents the Euclidean distance between the center point of the real box and the predicted box, c represents the diagonal distance of the minimum bounding rectangle of the predicted box and the real box, α is the weight factor, v is the aspect ratio consistency, and IoU is the overlap ratio between the real box and the predicted box; Step 5: Using the training set and the validation set to train the small target traffic sign real-time detection model based on the improved YOLOv5s, and saving the trained model as the optimal model, further including steps 5.1 to 5.4: Step 5.1: Set the training parameters of the small target traffic sign real-time detection model based on improved YOLOv5s, the model training parameters include: learning rate, momentum, weight decay, optimizer, number of iterations, batch size; Step 5.2: input the training set and validation set images and corresponding labels into the improved YOLOv5s small target traffic sign real-time detection model, use the back propagation algorithm to calculate the gradient of the loss function to the model parameters, and adjust the model parameters by minimizing the loss function to gradually approach the optimal solution; Step 5.3: Use the optimizer SGD to update the model parameters in the direction of gradient descent until the loss function of the training set and the validation set no longer decreases, and the evaluation indicators mAP, recall rate R, and precision rate P no longer increase; Step 5.4: Save the trained model parameters as the optimal model; Step 6: Use the test set to test the optimal model, evaluate the test set test results, and meet the accuracy requirements, so as to obtain the final small target traffic sign real-time detection model based on the improved YOLOv5s.

2. A small target traffic sign real-time detection method based on improved YOLOv5s according to claim 1, characterized in that: In step 2, the training set, validation set, and test set are divided according to a ratio of 7:2:

1.

3. A small target traffic sign real-time detection method based on improved YOLOv5s according to claim 1, characterized in that: The step 6 further includes steps 6.1 to 6.3: Step 6.1: Input the test set into the optimal model described in step 5; Step 6.2: Calculate the model performance indicators: accuracy P, recall R, mAP, number of parameters, computational complexity GFLOPs, model size. The specific calculation formula is as follows: Among them, P is the precision, R is the recall, mAP is the average precision of all categories, AP is the average precision, m is the total number of traffic sign categories, TP is the number of positive samples correctly identified as positive samples, FP is the number of negative samples incorrectly identified as positive samples, and FN is the number of positive samples incorrectly identified as negative samples; Step 6.3: When the performance index meets the accuracy requirements, the final small target traffic sign real-time detection model based on improved YOLOv5s is obtained.

Images