Small target traffic sign identification method based on improved YOLOv8

Abstract

The invention discloses a small target traffic sign recognition method based on improved YOLOv8, and the method comprises the steps: constructing a data set, selecting sample pictures from the data set, carrying out the labeling and data enhancement, and dividing the data set into a training set and a test set; the YOLOv8 model is improved, and a multi-scale attention mechanism EMA is embedded in a backbone network of the YOLOv8 model; an up-sampling operator in a neck network of the YOLOv8 model is replaced by a DySample up-sampling operator; replacing an original loss function CIoU of the YOLOv8 model with a WIoUv3 loss function; and inputting the training set into the improved YOLOv8 model for training, after training is completed, detecting the detection performance of the improved YOLOv8 model by using the test set to obtain an identification result, and evaluating the performance of the improved YOLOv8 model by using various indexes. According to the small target traffic sign identification method based on the improved YOLOv8, the identification precision and the identification speed of the small target traffic sign can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic sign detection and recognition, and in particular to a small target traffic sign recognition method based on improved YOLOv8. Background Art

[0002] Traffic signs are a very important part of road driving. They guide and regulate the driver's driving behavior and ensure the safety of the vehicle during driving. Traffic sign detection and recognition helps the autonomous driving system to monitor and interpret various traffic signs on the road in real time, such as speed limit signs, prohibition signs, and instruction signs, and then provide necessary information for autonomous driving decisions. In unmanned driving technology, due to the performance of the camera equipment, small target traffic signs are often encountered. Therefore, the ability to quickly and accurately detect small target traffic signs will help road safety driving and driving decisions, and has high practical significance and application value.

[0003] Traffic sign detection and recognition methods based on deep learning achieve more accurate, robust and efficient traffic sign detection through multi-layer network structures and large amounts of data training. According to whether it is necessary to generate candidate frames first, methods based on deep learning can be divided into two categories: two-stage methods and one-stage methods. The two-stage method first generates candidate regions and then performs feature extraction and classification, while the one-stage method performs feature extraction, position regression and category classification in one step.

[0004] The typical two-stage method Faster R-CNN introduces the idea of ​​SPP (Spatial pyramid pooling), maps the region of interest to a fixed-size feature map through ROI-Pooling (Region of interest pooling), eliminates unnecessary calculation steps, and thus speeds up feature extraction. At the same time, it abandons selective search and uses RPN (Region proposal networks) to automatically generate candidate regions, effectively solving the computational complexity problem of multi-target detection.

[0005] The Chinese patent "A traffic sign detection method based on regional attention" (application number: 202110425906.8) proposes using Faster R-CNN to detect traffic signs, but this method has poor detection effect when the traffic signs are small in the image.

[0006] The one-stage detection algorithm combines feature extraction, position regression and category classification into one step, uses the entire input image as the input of the network, and directly regresses the position of the bounding box in the output layer to predict the category to which it belongs. It is more suitable for use in the field of traffic sign recognition and detection with high real-time performance requirements.

[0007] The Chinese patent "Traffic Sign Recognition Model Training Method and System Based on YOLOv4" (Application Number: 202110838499.3) uses the one-stage detection algorithm YOLOv4 to identify traffic signs, but this method has low detection and recognition accuracy for traffic signs.

[0008] In summary, current traffic sign recognition methods often ignore the fact that traffic signs are too small in images, and have poor recognition effects on small traffic signs. At the same time, the unimproved deep learning network has low accuracy in target detection and recognition. Summary of the invention

[0009] In order to solve the above technical problems, the present invention provides a small target traffic sign recognition method based on improved YOLOv8, which can effectively improve the recognition accuracy and recognition speed of small target traffic signs. The method implementation flow chart is as follows: Figure 1 shown.

[0010] The technical solution adopted by the present invention is:

[0011] The small target traffic sign recognition method based on improved YOLOv8 includes the following steps:

[0012] Step 1: Build a dataset, select sample images from the dataset for annotation and data augmentation, and divide the dataset into a training set and a test set;

[0013] Step 2: Improve the YOLOv8 model by embedding the multi-scale attention mechanism EMA in the backbone network of the YOLOv8 model; replace the upsampling operator in the neck network of the YOLOv8 model with the DySample upsampling operator; replace the original loss function CIoU of the YOLOv8 model with the WIoUv3 loss function;

[0014] Step 3: Input the training set into the improved YOLOv8 model for training. After the training is completed, use the test set to test the detection performance of the improved YOLOv8 model to obtain the recognition results, and use various indicators to evaluate the performance of the improved YOLOv8 model.

[0015] In the step 1, the selected data set is the TT100K traffic sign data set, which classifies each type of traffic sign; the small target traffic signs in the sample image are labeled using the YOLO format in the labeling software LabelImg, and a TXT label file is automatically generated, in which the category of the small target traffic sign and the coordinates of the marking box are stored.

[0016] The selected data augmentation is mosaic data augmentation, which is to randomly scale, randomly crop and splice 4 pictures together for training.

[0017] In step 2, a multi-scale attention mechanism EMA is embedded in front of the SPPF spatial pyramid pooling module of the YOLOv8 model backbone network. The multi-scale attention mechanism EMA divides any given input feature map into G sub-features along the channel dimension to learn different semantics, and uses three parallel paths to extract the attention weights of the grouped feature maps. In the structure of the multi-scale attention mechanism EMA, two parallel paths are 1x1 branches and the third path is a 3x3 branch. In order to capture the dependencies between channels and reduce the computational complexity of the EMA attention mechanism, the two 1×1 branches use one-dimensional global average pooling to encode the channels along the image width X and height Y directions, while the 3×3 branch only uses a 3×3 convolution to capture more detailed multi-scale spatial structure information. The channel features encoded along the width and height directions are spliced ​​in the height direction and share a 1×1 convolution. After decomposing the output of the 1×1 convolution into two vectors, the activation function Sigmoid is used to fit the two-dimensional binomial distribution on the linear convolution, and finally the output of the two parallel path 1×1 branches is selected by the Re-weight adaptive feature variable.

[0018] Replace the upsampling operator in the original YOLOv8 model with the DySample upsampling operator. The DySample upsampling operator performs upsampling from the perspective of point sampling.

[0019] The position of the target in the image is predicted by bounding box regression. Bounding box regression uses the overlap between the predicted bounding box and the true bounding box as the loss function, also known as the intersection over union (IoU). w and h represent the width and height of the predicted bounding box respectively; w gt and h gt represents the width and height of the ground-truth bounding box; W i and H i Respectively represent the width and height of the overlapping rectangle of the predicted bounding box and the true bounding box; W g and H g is the width and height of the minimum bounding rectangle of the predicted and true bounding boxes.

[0020] The WIoUv3 loss function is introduced to achieve smoother convergence. WIoUv3 introduces a non-monotonic focusing factor β based on WIoUv1. The size of β is directly proportional to the quality of the anchor box. The function expression of WIoUv3 is as follows:

[0021] L WIoUv3 =rL WIoUV1

[0022]

[0023] Among them, L WIoUv3 is the WIoUv3 loss function, α and δ are hyperparameters, r is the gradient gain, L * IoU is a monotone focusing function. When β = δ and r = 1, the anchor box will obtain the highest gradient gain.

[0024] The present invention provides a small target traffic sign recognition method based on improved YOLOv8, and the technical effects are as follows:

[0025] 1) The present invention adopts the mosaic data enhancement method, which enriches the data set by splicing through random scaling, random cropping, etc., making the model more robust. In the last ten rounds of training, the mosaic data enhancement is turned off, allowing the model to focus more on the details of the real image, effectively improving the model's performance in small target traffic sign recognition and generalization ability.

[0026] 2) The present invention proposes to embed a multi-scale attention mechanism EMA in the backbone network of the YOLOv8 network model, which can integrate contextual information of different scales to make the model more focused on the detection and recognition of small target traffic signs.

[0027] 3) The present invention proposes to use the DySample upsampling operator to replace the original upsampling operator of YOLOv8 in the neck part of YOLOv8. The DySample upsampling operator performs upsampling from a point sampling perspective. This upsampling method has lower model parameters and delay, effectively reducing the amount of model calculation and detection speed.

[0028] 4) The present invention replaces the loss function with WIoUv3. When the aspect ratio of the predicted bounding box is linearly proportional to the aspect ratio of the true bounding box, the original CIoU loss function of the model will cause relatively large fluctuations in the convergence process, thereby affecting the optimization of the network. The introduction of WIoUv3 can solve the problem of CIoU. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 The present invention is a flowchart for implementing the method of the present invention.

[0030] Figure 2 This is a schematic diagram of the LabelImg software.

[0031] Figure 3 This is the image after mosaic data enhancement.

[0032] Figure 4 This is the overall structure diagram of the improved YOLOv8 of the present invention.

[0033] Figure 5EMA structure diagram added for the present invention.

[0034] Figure 6 It is the intersection-over-union (IoU) relationship diagram.

[0035] Figure 7 is a comparison chart of the improved results;

[0036] Among them, part a in Figure 7 is the original YOLOv8 detection result; part b is the improved YOLOv8 detection result. DETAILED DESCRIPTION

[0037] The small target traffic sign recognition method based on improved YOLOv8 includes the following steps:

[0038] Step 1: Build a dataset, select sample images from the dataset for annotation and data augmentation, and divide the dataset into a training set and a test set;

[0039] Step 2: Improve the YOLOv8 model by embedding the multi-scale attention mechanism EMA in the backbone network of the YOLOv8 model to enhance the feature information extraction capability; replace the original upsampling operator in the neck network of the YOLOv8 model with the DySample upsampling operator to reduce the number of network parameters and speed up the model detection; replace the original loss function CIoU of the YOLOv8 model with the WIoUv3 loss function to make the model learn the target box shape more stably and further improve the model detection accuracy;

[0040] Step 3: Input the training set into the improved YOLOv8 model for training. After the training is completed, use the test set to test the detection performance of the improved YOLOv8 model to obtain the recognition results, and use various indicators to evaluate the performance of the improved YOLOv8 model.

[0041] In the step 1, the selected data set is the TT100K traffic sign data set of Tsinghua University. The TT100K traffic sign data set has more than 10,000 traffic sign sample images, and each type of traffic sign is classified; the small target traffic signs in the sample images are labeled using the YOLO format in the labeling software LabelImg, and a TXT label file is automatically generated. The TXT label file stores the category of the small target traffic sign and the coordinates of the marking box. The labeling schematic diagram is as follows: Figure 2 As shown;

[0042] In step 1, the selected data enhancement is mosaic data enhancement, which can enhance data diversity and model robustness by randomly scaling, randomly cropping and splicing four pictures together for training, which is beneficial to improving the model's detection performance for small targets. Figure 3 shown.

[0043] The present invention selects 3000 pictures from the TT100K traffic sign dataset and divides them into a training set and a test set in a ratio of 8:2. The picture resolution is 640×640. The training set is used for model training, and the test set is used to evaluate model performance.

[0044] In step 2, the improved YOLOv8 model network structure is as follows Figure 4 As shown in the figure, the present invention embeds a multi-scale attention mechanism EMA in front of the SPPF spatial pyramid pooling module of the YOLOv8 model backbone network. The multi-scale attention mechanism EMA is an efficient attention mechanism with a parallel substructure. The multi-scale attention mechanism EMA divides any given input feature map into G sub-features along the channel dimension to learn different semantics, and uses three parallel paths to extract the attention weights of the grouped feature maps. The structure of the multi-scale attention mechanism EMA is as follows Figure 5 shown. Figure 5 In the figure, two parallel paths are 1x1 branches and the third path is a 3x3 branch. In order to capture the dependencies between channels and reduce the computational complexity of the EMA attention mechanism, the two 1×1 branches use one-dimensional global average pooling to encode the channels along the image width (X) and height (Y) directions, while the 3×3 branch only uses a 3×3 convolution to capture more detailed multi-scale spatial structure information. The channel features encoded along the width and height directions are concatenated in the height direction and share a 1×1 convolution. After decomposing the output of the 1×1 convolution into two vectors, the activation function Sigmoid is used to fit the two-dimensional binomial distribution on the linear convolution. Finally, after the Re-weight adaptive feature variable selection, it is used as the output of the two parallel path 1×1 branches. In this way, the EMA attention mechanism not only encodes the information between channels to adjust the importance of different channels, but also retains the precise spatial structure information in the channel.

[0045] The feature upsampling in the YOLOv8 model network structure is a key factor in gradually restoring the feature resolution in the prediction model. The upsampling operation enables the model to effectively detect objects of various sizes. The present invention replaces the upsampling operator in the original YOLOv8 model with the DySample upsampling operator. The DySample upsampling operator performs upsampling from the perspective of point sampling. Compared with the previous kernel-based dynamic upsampling operator, the DySample upsampling operator can complete sampling more efficiently and quickly.

[0046] The most commonly used upsampling operators in the prediction model are neighboring and bilinear interpolation. These two interpolation methods follow fixed rules to interpolate upsampled values, which will make the prediction model limited and lead to the loss of key image details. The upsampling operator used in the original YOLOv8 is based on neighboring interpolation, so the present invention uses the DySample upsampling operator to reduce the number of model parameters while maintaining a high detection accuracy.

[0047] In object detection algorithms, accurate object positioning is crucial. Usually, bounding box regression can be used to predict the location of the object in the image. Bounding box regression uses the overlap between the predicted bounding box and the true bounding box as a loss function, also known as the intersection over union (IoU). The spatial relationship between the predicted bounding box and the true bounding box is as follows: Figure 6 As shown, Figure 6 The upper left bounding box is the real bounding box, the lower right bounding box is the predicted bounding box, w and h represent the width and height of the predicted bounding box respectively; w gt and h gt represents the width and height of the ground-truth bounding box; W i and H i Respectively represent the width and height of the overlapping rectangle of the predicted bounding box and the true bounding box; W g and H g It is the width and height of the minimum bounding rectangle of the predicted bounding box and the true bounding box. The YOLOv8 model uses the CIoU loss function, which combines the overlapping area, center point distance and aspect ratio in the bounding box regression process to improve the accuracy of regression positioning. The calculation formula is as follows:

[0048]

[0049] Where, L CIoU is the CIoU loss function, IoU is the overlapping area between the predicted bounding box and the true bounding box, and its value is between 0 and 1. gt denote the center points of the predicted bounding box and the true bounding box, respectively, and ρ 2 (b,b gt ) represents the Euclidean distance between the center points of the two bounding boxes, c represents the diagonal of the minimum circumscribed rectangle formed when the predicted bounding box and the true bounding box intersect, α is a balance parameter, and ν is used to evaluate the aspect ratio. The formula shows that when the aspect ratio of the predicted bounding box is linearly proportional to the aspect ratio of the true bounding box, that is, ν is equal to 0. In this case, the penalty term of the aspect ratio does not work, resulting in relatively large fluctuations in the convergence process, thereby affecting the optimization of the network. Therefore, in order to solve the above network optimization problems, the present invention introduces the WIoUv3 loss function to achieve smoother convergence, so that the target box has better positioning accuracy.

[0050] There are three versions of the WIoU loss function, namely WIoUv1, WIoUv2, and WIoUv3. The formula of the WIoUv1 loss function is as follows:

[0051]

[0052] L WIoUν1 =R WIoU L IoU

[0053]

[0054] Where, L IoU is the IoU loss function, R WIoU is the distance attention, L WIoUv1 is the loss function of WIoUv1. WIoUv3 introduces a non-monotonic focusing factor β based on WIoUv1. The size of β is directly proportional to the quality of the anchor box. The function expression of WIoUv3 is as follows:

[0055] L WIoUv3 =rL WIoUV1

[0056]

[0057]

[0058] Among them, L WIoUv3 is the WIoUv3 loss function, α and δ are hyperparameters, r is the gradient gain, L * IoU is a monotone focusing function. When β = δ and r = 1, the anchor box will obtain the highest gradient gain.

[0059] In step 3, the Adam optimizer is used in the training of the improved YOLOv8 model to gradually fit the model to the real data distribution. The Adam optimizer is an adaptive optimization algorithm that can adjust the learning rate according to the historical gradient information and normalize the parameter update so that each parameter update has a similar magnitude, thereby improving the training effect. The training process iteration cycle is 300 epochs, the momentum is 0.937, the weight decay is 0.0005, the initial learning rate is 0.01, the cyclic learning rate is 0.1, the iteration batch size is 16, and the other hyperparameters are all default values.

[0060] To further illustrate the technical solution of the present invention, the following is a specific description in conjunction with embodiments:

[0061] In this embodiment, a multi-scale attention mechanism EMA is embedded in the backbone network of the YOLOv8 model to retain the weights of the small target feature channels and suppress other irrelevant background weights, thereby effectively extracting the key features of the small target. The DySample upsampling operator is used to replace the upsampling module of the original YOLOv8 neck network in the neck network of the YOLOv8 model. The DySample upsampling operator performs sampling from the perspective of point sampling, has a faster detection speed, and does not affect the accuracy of the model in small target traffic sign recognition. In terms of loss function, the WIoUv3 loss function is used to replace the CIoU loss function in the original YOLOv8, thereby effectively improving the model's learning ability and positioning ability for the target box, thereby further improving the detection accuracy of the model.

[0062] The evaluation indicators of this embodiment use mean average precision (mAP), the number of frames per second (FPS) of detected pictures, and the amount of computation (GFLOPs), which are three common evaluation indicators in target detection methods to evaluate the performance of the present invention. Mean average precision (mAP) is related to precision (Precision) and recall (Recall), which are usually used to evaluate the ability of the model to predict specific categories. The amount of computation (GFLOPs) is the number of floating-point operations of the model in one second. The calculation formulas for mean average precision (mAP), precision (Precision) and recall (Recall) are as follows:

[0063]

[0064] Where P is the average precision, N is the recognition type, TP means that the model predicts a positive example as a positive example, FN means that the model predicts a positive example as a negative example, and FP means that the model predicts a negative example as a positive example.

[0065] To verify the effectiveness of the present invention, the present invention and the original YOLOv8 were tested on the same test set, and the results of the comparison of various performance indicators are shown in Table 1 below.

[0066] Table 1 Comparison results of various performance indicators of the test set

[0067]

[0068] As can be seen from the table, compared with the original YOLOv8, the performance of the present invention in small target traffic sign recognition is better, and the mAP value reaches 88.2%. On the basis of the original YOLOv8, the precision (Precision) is improved by 3.9%, the recall rate (Recall) is improved by 6.1%, and the number of frames per second (FPS) of the detected picture is increased by 22 frames. The floating-point operation amount is also lower than that of the original YOLOv8, which shows that the present invention has a faster detection speed while improving the detection accuracy and has lower hardware requirements.

[0069] It can be seen from the experimental results that the present invention optimizes the backbone network, the neck network, and the loss function on the basis of the original YOLOv8 method, effectively improving the detection accuracy of the YOLOv8 network model for small target traffic signs. The detection speed of the improved method is significantly improved compared with the original YOLOv8 method, and the number of detection frames per second is also greatly improved, which can meet the real-time requirements while improving the detection accuracy. The detection results are shown in Figure 7, where part a in Figure 7 is the original YOLOv8 detection result, and part b in Figure 7 is the detection result of the present invention. As shown in Figure 7, the method of the present invention has higher recognition accuracy for small target traffic signs, has good practicality in different scenarios, and has broad application prospects.

Claims

1. A small target traffic sign recognition method based on improved YOLOv8, characterized by The following steps are involved: Step 1: Build a dataset, select sample images from the dataset for annotation and data augmentation, and divide the dataset into a training set and a test set; Step 2: Improve the YOLOv8 model by embedding the multi-scale attention mechanism EMA in the backbone network of the YOLOv8 model; replace the upsampling operator in the neck network of the YOLOv8 model with the DySample upsampling operator; replace the original loss function CIoU of the YOLOv8 model with the WIoUv3 loss function; Step 3: Input the training set into the improved YOLOv8 model for training. After the training is completed, use the test set to test the detection performance of the improved YOLOv8 model to obtain the recognition results, and use various indicators to evaluate the performance of the improved YOLOv8 model.

2. According to claim 1, the small target traffic sign recognition method based on improved YOLOv8 is characterized in that: In the step 1, the selected data set is the TT100K traffic sign data set, which classifies each type of traffic sign; the small target traffic signs in the sample image are labeled using the YOLO format in the labeling software LabelImg, and a TXT label file is automatically generated, in which the category of the small target traffic sign and the coordinates of the marking box are stored.

3. The small target traffic sign recognition method based on improved YOLOv8 according to claim 2 is characterized in that: The selected data augmentation is mosaic data augmentation, which is to randomly scale, randomly crop and splice 4 pictures together for training.

4. The small target traffic sign recognition method based on improved YOLOv8 according to claim 1 is characterized in that: In the step 2, a multi-scale attention mechanism EMA is embedded in front of the SPPF spatial pyramid pooling module of the YOLOv8 model backbone network. The multi-scale attention mechanism EMA divides any given input feature map into G sub-features along the channel dimension to learn different semantics, and uses three parallel paths to extract the attention weights of the grouped feature maps; In the structure of the multi-scale attention mechanism EMA, two parallel paths are 1x1 branches and the third path is a 3x3 branch. In order to capture the dependencies between channels and reduce the computational complexity of the EMA attention mechanism, the two 1×1 branches use one-dimensional global average pooling to encode the channels along the image width X and height Y directions, while the 3×3 branch only uses one 3×3 convolution to capture more detailed multi-scale spatial structure information; the channel features encoded along the width and height directions are concatenated in the height direction and share a 1×1 convolution. After decomposing the output of the 1×1 convolution into two vectors, the activation function Sigmoid is used to fit the two-dimensional binomial distribution on the linear convolution, and finally, after Re-weight adaptive feature variable selection, it is used as the output of the two parallel path 1×1 branches.

5. The small target traffic sign recognition method based on improved YOLOv8 according to claim 1 is characterized in that: In the step 2, the upsampling operator in the original YOLOv8 model is replaced with the DySample upsampling operator; the DySample upsampling operator performs upsampling from the perspective of point sampling; The position of the target in the image is predicted by bounding box regression. Bounding box regression uses the overlap between the predicted bounding box and the true bounding box as the loss function, also known as the intersection over union (IoU); w and h represent the width and height of the predicted bounding box, respectively; w gt and h gt represents the width and height of the ground-truth bounding box; W i and H i Respectively represent the width and height of the overlapping rectangle of the predicted bounding box and the true bounding box; W g and H g is the width and height of the minimum bounding rectangle of the predicted and true bounding boxes.

6. The small target traffic sign recognition method based on improved YOLOv8 according to claim 1 is characterized in that: In step 2, the WIoUv3 loss function is introduced to achieve smoother convergence. WIoUv3 introduces a non-monotonic focusing factor β based on WIoUv1. The size of β is directly proportional to the quality of the anchor frame. The function expression of WIoUv3 is as follows: L WIoUv3 =rL WIoUV1 Among them, L WIoUv3 is the WIoUv3 loss function, α and δ are hyperparameters, r is the gradient gain, L * IoU is a monotonic focusing function; when β = δ, r = 1, the anchor box will obtain the highest gradient gain.

Images