Traffic sign detection method based on improved Tiny-YOLOv4

Abstract

The invention discloses a traffic sign detection method based on improved Tiny‑YOLOv4, comprising the following steps: S1, obtaining an image training set and an image verification set for traffic sign detection; S2, improving the Tiny‑YOLOv4 target detection network structure, including: improving the feature pyramid network: using a larger scale prediction feature layer, and performing multi-scale feature fusion on it; improving the backbone feature extraction network: introducing a cross-stage local network and spatial pyramid pooling, and changing the convolution kernel size of the last convolution layer of the backbone feature extraction network from 3×3 to 1×1; S3, using a clustering algorithm to perform cluster analysis on the image training set and the image verification set to obtain a suitable anchor frame size; S4, training; S5, using the trained improved Tiny‑YOLOv4 to perform traffic sign detection. Compared with the existing traffic sign detection algorithm, the present invention can improve the detection accuracy under the premise of ensuring fast detection speed and low model memory capacity, and can meet the needs of practical applications.

Description

Technical Field

[0001] The present invention belongs to the technical field of image detection, and in particular relates to a traffic sign detection method based on improved Tiny-YOLOv4. Background Art

[0002] Traffic sign detection system is an important part of intelligent driving. It uses the camera mounted on the car to take real-time photos of traffic road scenes and transmit the captured pictures to the detection system for traffic sign detection, providing effective road traffic information for the driver. Predicting the traffic information ahead in advance can enable the driver to make timely judgments, increase their reaction time, and reduce the probability of traffic accidents.

[0003] Traffic sign detection methods are mainly divided into traditional detection methods and deep learning methods. Traditional detection methods require artificial design of features, which is time-consuming, has low detection accuracy, and requires high-quality images, which makes it difficult to meet the needs of actual application scenarios. Traffic sign detection algorithms based on deep learning can be divided into two categories: candidate region-based traffic sign detection algorithms and regression-based traffic sign detection algorithms. The candidate region-based detection algorithm is a two-stage algorithm, which first needs to screen out candidate regions, then extract features of candidate regions, and finally classify and locate. The regression-based traffic sign belongs to a one-stage algorithm, which omits the step of generating candidate regions and directly uses convolutional neural networks to extract features to complete the classification and positioning of targets. These two types of algorithms have their own advantages and disadvantages. The two-stage algorithm has good detection and positioning effects, but the detection takes a long time. The one-stage algorithm has a fast detection speed and can achieve multi-category detection, but the detection accuracy is slightly worse than the two-stage algorithm.

[0004] Although deep learning is widely used in the field of traffic sign detection and has achieved many achievements, there is still no detection algorithm that takes into account the three requirements of high traffic sign detection accuracy, fast detection speed, and small model memory capacity. Therefore, it is necessary to comprehensively consider the requirements of detection accuracy, detection speed, and model memory capacity, try to meet the needs of actual applications, and design a traffic sign detection algorithm with high detection accuracy, fast detection speed, and small model memory capacity. Summary of the invention

[0005] The purpose of the present invention is to provide a traffic sign detection method based on improved Tiny-YOLOv4, which takes into account the three requirements of high traffic sign detection accuracy, fast detection speed and small model memory capacity.

[0006] To achieve the above object, the present invention provides a traffic sign detection method based on improved Tiny-YOLOv4, comprising the following steps:

[0007] S1, preprocessing image information to form image training set, image verification set and image test set;

[0008] S2. Improve the network structure of Tiny-YOLOv4:

[0009] Improve the feature pyramid network, use a larger scale prediction feature layer, and perform multi-scale feature fusion on it;

[0010] Improve the backbone feature extraction network, introduce cross-stage local network and spatial pyramid pooling, and change the convolution kernel size of the last convolution layer of the backbone feature extraction network from 3×3 to 1×1;

[0011] S3. Use the K-means clustering algorithm to perform cluster analysis on the image training set and the image verification set to generate an anchor box that is more suitable for the traffic sign target;

[0012] S4. Use the image training set and the image verification set to train the improved Tiny-YOLOv4 target detection network;

[0013] S5. Use the image test set to test the trained improved Tiny-YOLOv4 model and obtain evaluation indicators.

[0014] Optionally, in step S1, the traffic sign dataset is divided into a training set and a validation set in a ratio of 9:1.

[0015] Optionally, step S2 includes:

[0016] S21. Improve the feature pyramid network, use larger prediction feature layers of 52×52 and 104×104, and perform multi-scale fusion on them. Starting from the deepest feature map, upsample it in turn and fuse it with the feature map of the previous layer to fuse deep features with shallow features.

[0017] S22. Improve the backbone feature extraction network, change the second convolution layer of the original network structure to a cross-stage local network, add spatial pyramid pooling in front of the last convolution layer of the backbone feature extraction network, and change the convolution kernel size of the last convolution layer from 3×3 to 1×1.

[0018] Optionally, in step S3, a K-means clustering algorithm is used to generate an anchor box that is more suitable for the traffic sign. The distance metric d(ture,pre) is defined as:

[0019] d(true,pre)=1-IOU(true,pre)

[0020] Where IOU(true,pre) is the intersection-over-union ratio between the true bounding box and the calculated anchor box (prediction box). The larger the IOU, the higher the overlap between the calculated anchor box and the true bounding box. In this way, the anchor box with a small distance to the true box will be retained.

[0021] Optionally, in step S4, when training the improved Tiny-YOLOv4 target detection network, CIOU loss is used for the coordinate regression loss of the target box. CIOU comprehensively considers factors such as the overlapping area between bounding boxes, the center point distance, the aspect ratio, and the scale change, so that the regression of the target box becomes more stable and will not have problems such as divergence during training like IOU and GIOU. The formula of CIOU can be expressed as:

[0022]

[0023] Among them, ρ 2 (b,b gt ) represents the Euclidean distance between the center point of the predicted box and the true box, b represents the predicted box, b gt represents the real box; c represents the diagonal distance of the minimum closure area that can contain both the predicted box and the real box. The formulas for α and υ are as follows:

[0024]

[0025]

[0026] Where W and h represent the width and height of the prediction box respectively, w gt and h gt Represent the width and height of the real box respectively. The CIOU loss function can be expressed as:

[0027]

[0028] Optionally, in step S5, the training set is input into the trained detection network to obtain evaluation indicators, including: detection accuracy, recall rate, F1 value, average detection precision mAP, detection speed FPS and model memory capacity of the improved Tiny-YOLOv4 detection model, and the formula is as follows:

[0029]

[0030]

[0031]

[0032]

[0033] Among them, TP represents the number of samples that are actually positive samples and predicted correctly, FN represents the number of samples that are actually positive samples but predicted incorrectly, FP represents the number of samples that are actually negative samples but predicted incorrectly, and TN represents the number of samples that are actually negative samples and predicted correctly. Precision is mainly used to indicate whether the prediction result is accurate. The higher the value, the fewer false detections of the target. Recall is mainly used to indicate whether the prediction of the target is comprehensive. The higher the value, the fewer missed detections of the target. AP represents the area under the PR curve drawn with Recall as the horizontal axis and Precision as the vertical axis, which can measure the performance of the model in each category.

[0034] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0035] The present invention uses a larger scale prediction feature map in the feature pyramid network, and integrates shallow features with deep features, thereby enhancing the feature expression ability of the network and improving the detection accuracy of small target traffic signs and dense traffic signs; at the same time, the backbone feature extraction network is deepened, and cross-stage local networks and spatial pyramid pooling are introduced to enhance the feature extraction ability of the network. Compared with the existing traffic sign detection algorithm, the present invention can improve the detection accuracy while ensuring fast detection speed and low model memory capacity, taking into account the three requirements of high traffic sign detection accuracy, fast detection speed, and small model memory capacity, and meeting the needs of practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of the traffic sign detection method based on the improved Tiny-YOLOv4.

[0037] Figure 2 It is a structural diagram of the cross-stage local network.

[0038] Figure 3 It is a structural diagram of spatial pyramid pooling.

[0039] Figure 4 It is a network structure diagram of the traffic sign detection algorithm based on the improved Tiny-YOLOv4 of the present invention.

[0040] Figure 5(a) is a comparison of the AP values ​​of the Tiny-YOLOv4 detection model indicator before and after improvement.

[0041] Figure 5(b) is a comparison of the AP values ​​of the prohibition sign of the Tiny-YOLOv4 detection model before and after the improvement.

[0042] Figure 5(c) is a comparison of the AP values ​​of the warning signs of the Tiny-YOLOv4 detection model before and after the improvement. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0044] The invention discloses a traffic sign detection method based on improved Tiny-YOLOv4, belonging to the field of image detection technology. The method comprises: obtaining image information of a traffic sign data set; preprocessing the image information to form an image training set, an image verification set and an image test set; improving the network structure of Tiny-YOLOv4: improving the feature pyramid network, using a larger scale prediction feature layer, and performing multi-scale feature fusion on it; improving the backbone feature extraction network, introducing a cross-stage local network and spatial pyramid pooling, and changing the convolution kernel size of the last convolution layer of the backbone feature extraction network from 3×3 to 1×1; using a K-means clustering algorithm to perform cluster analysis on an image training set and an image verification set to obtain an anchor frame suitable for the data set; using the image training set and the image verification set to train the improved Tiny-YOLOv4 target detection network; using the image test set to test the trained improved Tiny-YOLOv4 model to obtain evaluation indicators. The improved Tiny-YOLOv4 in the present invention can improve the detection accuracy of small target traffic signs, and the model memory capacity of the improved Tiny-YOLOv4 algorithm is only 10.8MB, which greatly reduces the number of model parameters. The comprehensive performance of the improved algorithm is better than other traffic sign detection algorithms, and can perform traffic sign detection tasks more effectively.

[0045] The traffic sign detection method based on improved Tiny-YOLOv4 in the embodiment of the present invention is as follows: Figure 1 As shown, the following steps are included:

[0046] S1. Preprocess image information to form image training set, image verification set and image test set.

[0047] This example uses the CCTSDB dataset, which is divided into a training set of 15,724 images and a test set of 400 images. It mainly detects three types of traffic signs, namely, instruction signs, warning signs, and prohibition signs. The training set is divided into a training set and a validation set at a ratio of 9:1, and finally forms an image training set, an image validation set, and an image test set.

[0048] S2. Design the network structure of the traffic sign detection algorithm based on the improved Tiny-YOLOv4, such as Figure 4As shown, it includes two parts: backbone feature extraction network and feature pyramid network.

[0049] Improve the backbone feature extraction network: Figure 2 As shown in the figure, the second layer of the original network, DarknetConv2D_BN_Leaky, is replaced with a cross-stage local network, namely the Resblock_body module. This increases the network depth without increasing the computational burden, enhances the network's feature extraction capability, and does not reduce the detection speed much, nor does it increase the number of model parameters much. Figure 3 As shown in the figure, a spatial pyramid pooling module is added before the last convolution layer of the backbone feature extraction network to expand the network's receptive field and fuse local and global features. In order to balance speed and accuracy, the convolution kernel size of the last convolution layer of the network is changed from 3×3 to 1×1, which can reduce the amount of calculation and the number of model parameters.

[0050] Improved feature pyramid network: The two prediction feature layers of the original network are not effective for small target detection and dense traffic sign detection, so the prediction feature layers with scales of 13×13 and 26×26 are deleted and replaced with prediction feature layers with larger scales of 52×52 and 104×104 to improve the detection accuracy of small target traffic signs and dense traffic signs. In order to make the prediction feature layer contain both location detail information and high-level semantic information, the deep feature maps are upsampled in turn, enlarged to the same size as the shallow feature maps, and then feature fusion and splicing are performed to enhance the detection capability of the network.

[0051] S3. Before training the improved Tiny-YOLOv4 target detection network, use the K-means clustering algorithm to perform cluster analysis on the image training set and the validation set to obtain a suitable anchor box size. In this embodiment, six anchor boxes are obtained through cluster analysis. The anchor boxes corresponding to the 104×104 prediction feature layer are (8, 23), (8, 15), (12, 20), and the anchor boxes corresponding to the 52×52 prediction feature layer are (17, 30), (29, 43), and (56, 79).

[0052] S4. When training the improved Tiny-YOLOv4 target detection network, CIOU loss is used for the coordinate regression loss of the target box. CIOU comprehensively considers factors such as the overlapping area between bounding boxes, the center point distance, the aspect ratio, and the scale change, making the regression of the target box more stable and not having problems such as divergence during training like IOU and GIOU. The CIOU loss function can be expressed as:

[0053]

[0054] Among them, ρ2 (b,b gt ) represents the Euclidean distance between the center point of the predicted box and the true box, b represents the predicted box, b gt represents the real box; c represents the diagonal distance of the minimum closure area that can contain both the predicted box and the real box. The formulas for α and υ are as follows:

[0055]

[0056]

[0057] Where W and h represent the width and height of the prediction box respectively, w gt and h gt Represent the width and height of the ground-truth box respectively.

[0058] During training, the size of the input image is 416×416, the Adam optimizer is used, the initial learning rate is 0.0001, the training is 150 epochs, the batch size is 8, and the dynamic learning rate and early stopping mechanism are used.

[0059] S5. Input the training set into the above-trained detection network to obtain evaluation indicators, including: detection accuracy, recall rate, F1 value, average detection precision mAP, detection speed FPS and model memory capacity of the improved Tiny-YOLOv4 detection model.

[0060] Test Example:

[0061] The method is tested on the CCTSDB dataset using six indicators: Precision, Recall, F1, mAP, FPS, and model memory capacity. The method is compared with the Cascaded R-CNN target detection algorithm with high detection accuracy, the Strong Tiny-YOLOv3 target detection algorithm with small model content capacity, the YOLOv4 target detection algorithm, and the Tiny-YOLOv4 target detection algorithm.

[0062] Table 1 Comparison results between the present invention and four excellent methods

[0063]

[0064] As can be seen from Table 1, the Cascaded R-CNN model achieves the best results in Precision and F1, but the detection speed of this model is the slowest; the YOLOv4 model achieves the best results in Recall and mAP, but the detection speed of this model is too slow compared with the improved Tiny-YOLOv4, and the mAP of the improved Tiny-YOLOv4 model is only 1.09% lower than that of the improved Tiny-YOLOv4, which is not a big difference, while the detection speed is about 5 times that of the improved Tiny-YOLOv4, and the model memory capacity is reduced by 233.2MB compared with that of the improved Tiny-YOLOv4; the Tiny-YOLOv4 model achieves the best results in FPS, but the FPS of the improved Tiny-YOLOv4 model is not much different from that of the improved Tiny-YOLOv4 model, and the model memory capacity is reduced by 11.8MB compared with that of the improved Tiny-YOLOv4, and the mAP is increased by 13.4% compared with that of the improved Tiny-YOLOv4, and the detection effects on Precision, Recall, and F1 are better than those of the improved Tiny-YOLOv4; The Tiny-YOLOv3 model achieves the best results in terms of model memory capacity, but the improved Tiny-YOLOv4 model has a similar model memory capacity, a detection speed that is about 6 times faster, and a 3.23% increase in the mAP value compared to the improved Tiny-YOLOv4 model.

[0065] Figure 5(a) is a comparison of the AP values ​​of the indication signs of the Tiny-YOLOv4 detection model before and after the improvement. Figure 5(b) is a comparison of the AP values ​​of the prohibition signs of the Tiny-YOLOv4 detection model before and after the improvement. Figure 5(c) is a comparison of the AP values ​​of the warning signs of the Tiny-YOLOv4 detection model before and after the improvement. The detection accuracy of the improved detection algorithm is 13.4% higher than that of the original model, the model memory capacity is reduced by 11.8MB, and the detection speed reaches 200FPS. By comparison, the improved Tiny-YOLOv4 algorithm has better detection effects on small target traffic signs and dense traffic signs than the original algorithm.

[0066] Although the improved Tiny-YOLOv4 model did not achieve the optimal results in any single indicator, the comprehensive evaluation of detection accuracy, detection speed and model memory capacity showed that the effect of the improved Tiny-YOLOv4 model was better than that of other detection models.

[0067] The parts not described in the specification are all prior art or common knowledge. This embodiment is only used to illustrate the invention, and is not used to limit the scope of the invention. The equivalent substitution and other modifications made by those skilled in the art to the invention are considered to fall within the scope of protection of the claims of the invention.

[0068] It will be easily understood by those skilled in the art that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A traffic sign detection method based on improved Tiny-YOLOv4, characterized in that: The following steps are involved: S1. Obtain image training set and image verification set for traffic sign detection; S2. Improve the Tiny-YOLOv4 target detection network structure, including: Improve the feature pyramid network: use a larger scale prediction feature layer and perform multi-scale feature fusion on it. Starting from the deepest feature map, upsample it in turn and perform feature fusion with the feature map of the previous layer to fuse deep features with shallow features. Improve the backbone feature extraction network: introduce cross-stage local network and spatial pyramid pooling, replace the second layer of the original network with the cross-stage local network, add spatial pyramid pooling before the last convolution layer of the backbone feature extraction network, and finally change the convolution kernel size of the last convolution layer of the backbone feature extraction network from 3×3 to 1×1; S3. Use a clustering algorithm to perform cluster analysis on the image training set and the image verification set to obtain a suitable anchor box size; S4. Use the image training set and the image verification set to train the improved Tiny-YOLOv4 target detection network; S5. Use the trained improved Tiny-YOLOv4 for traffic sign detection; Among them, the coordinate regression loss of the target box adopts CIOU loss: Where IOU is IOU(tura,pre), which represents the intersection-over-union ratio of the real box and the predicted box; ρ 2 (b,b gt ) represents the Euclidean distance between the center point of the predicted box and the true box, b represents the predicted box, b gt represents the real box; c represents the diagonal distance of the minimum closure area that contains both the predicted box and the real box; The formulas for α and υ are as follows: Where W and h represent the width and height of the prediction box respectively, w gt and h gt Represent the width and height of the ground-truth box respectively.

2. The traffic sign detection method based on improved Tiny-YOLOv4 according to claim 1, characterized in that: The original prediction feature layers with scales of 13×13 and 26×26 are deleted and replaced with prediction feature layers with scales of 52×52 and 104×104.

3. The traffic sign detection method based on improved Tiny-YOLOv4 according to claim 1, characterized in that: After the training is completed, the improved Tiny-YOLOv4 target detection network is tested using the test set to obtain evaluation indicators.

4. The traffic sign detection method based on improved Tiny-YOLOv4 according to claim 3, characterized in that: Evaluation indicators include: detection accuracy, recall rate, F1 value, average detection precision mAP, detection speed FPS and model memory capacity.

5. The traffic sign detection method based on improved Tiny-YOLOv4 according to claim 1, characterized in that: Traffic signs include direction signs, warning signs, and prohibition signs.

6. The traffic sign detection method based on improved Tiny-YOLOv4 according to claim 1, characterized in that: Clustering algorithms include K-means.

Images