Improved yolov5n model for traffic signal detection

Abstract

An improved YOLOv5n model for traffic signal lamp detection, the steps of the optimization method comprising: S1, obtaining the image to be detected, performing data preprocessing, and dividing into a training set and a test set; S2, replacing the Convolution convolution with an involution convolution in the convolution kernel of 3 and the step of 2 in the head layer of the YOLOv5n model; S3, improving the Bottleneck in the C3 module of the YOLOv5n model, and introducing a depth separable convolution network and a FReLU activation function to form an MN_Bottleneck; S4, training the detection model using the data set, detecting the traffic signal lamp using the trained detection model, and evaluating the detection result. The present application makes the model lightweight, the training and reasoning speed faster, improves the robustness of the model, makes the model keep a relatively stable prediction result for the interference or noise in the input data, and makes it more suitable for complex and variable real scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection in deep learning, and particularly relates to an improved YOLOv5n model for traffic signal lamp detection. BACKGROUND

[0002] Target detection is an important problem in the field of computer vision, which refers to automatically detecting target objects appearing in a picture or video sequence and positioning and classifying them. In computer vision applications, target detection is a common task, such as intelligent monitoring, autonomous driving, unmanned aerial vehicle and robot navigation, medical image analysis, etc. In recent years, with the rise and development of deep learning, deep learning-based target detection methods have made great progress. For example: Faster R-CNN, YOLO, SSD, Mask R-CNN, etc. These deep learning-based target detection methods have made great progress in target detection accuracy and speed, and have been widely used in various computer vision applications.

[0003] YOLOv5n is a target detection model, which is a member of the YOLOv5 series. Compared with the original YOLOv5, it greatly improves the detection speed while maintaining high accuracy. YOLOv5n uses a series of optimization strategies, such as using lightweight models under the premise of small precision loss, channel attention mechanism, dense connection module, etc., so as to perform well in target detection tasks. Specifically, YOLOv5n uses CSPDarknet53 as the backbone network, uses channel attention mechanism to enhance the expression ability of the model and reduce the amount of calculation. In addition, YOLOv5n performs pyramid multi-scale feature extraction on the input image, combines low-level features and high-level features to improve detection accuracy. In addition, YOLOv5n also uses dense connection module to better utilize the information of the previous layers, increases the depth and complexity of the model, and further improves the accuracy.

[0004] Although YOLOv5n performs better than other versions of YOLOv5, it still has some shortcomings: YOLOv5n relies on the quality and diversity of the dataset. If the dataset is too single or too small, it may lead to insufficient generalization ability of the model. For some extreme cases such as object occlusion, deformation, insufficient light, etc., the detection effect of YOLOv5n will also decrease. YOLOv5n needs longer training time compared to other versions, because it has more layers and larger network capacity, so it needs more computing resources and time. SUMMARY

[0005] The technical problem of the long training time, the slightly slow processing speed and the high computing power of the YOLOv5n model is solved by providing an improved YOLOv5n model for traffic signal lamp detection, which makes the model lightweight, faster in training and reasoning, improves the robustness of the model, and makes the model maintain relatively stable prediction results for interference or noise in the input data, so that it is more suitable for complex and variable real scenes.

[0006] The improved YOLOv5n model for traffic signal lamp detection makes the model lightweight, faster in training and reasoning, improves the robustness of the model, and makes the model maintain relatively stable prediction results for interference or noise in the input data, so that it is more suitable for complex and variable real scenes.

[0007] S1, obtain the image data set to be detected, perform data preprocessing, and divide the data set into a training set and a test set;

[0008] S2, replace the Convolution convolution with a step length of 2 in the head layer of the YOLOv5n model with an involution convolution;

[0009] S3, improve the Bottleneck in the C3 module of the YOLOv5n model, introduce a depth separable convolution network and a FReLU activation function to form an MN_Bottleneck;

[0010] S4, train the detection model using the data set, use the trained detection model to detect traffic signal lamps, and evaluate the detection results.

[0011] Further, the image data set to be detected adopts a traffic signal lamp data set S2TLD, and various road scenes and typical picture data are obtained by searching for keywords of four categories of "red", "yellow", "green" and "wait_on", to obtain a picture data set; the picture data set is cleaned, and the crawled data is removed damaged images, repeated images and non-keyword images.

[0012] Further, the training set and the test set in step 1 are divided after the image data is cleaned, and the data set is labeled, and the data set is divided into a training set and a test set according to 8:2.

[0013] Further, the head layer of the YOLOv5n model in step 2 is replaced with an involution convolution instead of a convolution module with consistent input and output channels; the specific operation mode is:

[0014] Step 1: Modify the common file and add the involution module to it;

[0015] Step 2: Modify the YOLO file and add the involution module in the parse_model function of YOLO;

[0016] The involution convolution decomposes the input tensor into two sub-tensors, performs a nonlinear transformation on one of the sub-tensors, and then combines the two sub-tensors; this operation can be represented by the following formula:

[0017]

[0018] where x j is the jth element of the input tensor, y i is the ith element of the output tensor, w ij and b ij are the weights and biases in the involution convolution, respectively, and k is the total number of blocks; the form of this nonlinear transformation is similar to the ReLU activation function, which is performed on the sub-tensor.

[0019] Further, the Bottleneck in the C3 module of the improved YOLOv5n model in step 3 is introduced in the Bottleneck of the original YOLOv5n model C3 module, and the specific operation mode is:

[0020] Step a: Modify the common file and add the Conv_FReLU, MN_Bottleneck, and MN_C3 modules in it;

[0021] Step b: Modify the YOLO file and add the MN_C3 module in the parse_model function of YOLO;

[0022] Step c: Create a yolov5n-MN.yaml file under the models folder;

[0023] Step d: Change the original yaml file in train.py to the yolov5n-MN.yaml file in the previous step to train.

[0024] Further, the MN_Bottleneck in step a is composed of a 1x1 convolution, a DW convolution, and a 1x1 convolution; the DW convolution is a depth separable convolution that performs convolution operations on each input channel with an independent convolution kernel; this operation is represented by the following formula:

[0025]

[0026] In the formula, m is the convolution kernel in the mth row, n is the convolution kernel in the nth column, x (i*8+m),(j*8+n),k is the pixel value of the input feature map on the (i*8+m)th row, (j*8+n)th column, kth channel, and w is the weight value.

[0027] Further, the MN_Bottleneck selects the FRelu activation function, solves the SiLu function space insensitivity problem, makes ordinary convolution also has the ability to capture complex visual layout, and makes the model have the ability of pixel-level modeling; the specific mathematical formula of the SiLu function is:

[0028]

[0029] In the formula, e is a natural exponent.

[0030] Further, the FReL in step a is a shape-adjustable activation function, which expands the ReLU function by adding scientific system parameters, so that FReLU can learn more feature representations, and the mathematical formula of FReLU is:

[0031] FReLU(x)=max(0,x)+min(0,α(e x-β -1))#(4)

[0032] In the formula, max(0,x) represents the ReLU function, min(0,α(e x-β -1)) is the expansion part of FReLU, and alpha and beta are learnable parameters for controlling the shape of FReLU.

[0033] Further, the data set training detection model in step S4 is to input the improved YOLOv5 algorithm to the data set image, and the optimal weight file of the traffic signal lamp target detection is obtained through continuous iteration training; the essence is to input the position, size and category information of the target object into the model for learning through a large amount of labeled data; in the learning process, the model will continuously adjust its internal parameters, so that the target object in the input image can be detected more accurately.

[0034] Beneficial effects

[0035] The improved YOLOv5n model for traffic signal lamp detection provided by the application has the following beneficial effects compared with the prior art:

[0036] (1) The application replaces the convolution in the head layer of the YOLOv5n model with involution convolution, which is lighter and more efficient than convolution, and is more concise than self-attention in form, and can achieve double improvement of precision and efficiency on various visual task models, and because involution convolution requires the input channel and the output channel to be consistent, the convolution module with consistent input channel and output channel is replaced in the head layer of the YOLOv5n model, thereby reducing the burden of the model.

[0037] (2) The application adopts residual edge design in the MN_C3 in the backbone layer, which can not only effectively alleviate the problem of gradient disappearance, but also increase the nonlinear expression ability of the network, and the residual edge is not needed in the neck layer MN_C3, because there are multiple branches, the depth of the network is shallow, and the problem of gradient disappearance is light, so the residual edge is not needed, and the residual edge is not used, thereby reducing the network parameter quantity and improving the running efficiency of the model.

[0038] (3) The application adopts DW convolution, i.e., depth separable convolution; has the advantages of small parameter quantity, fast calculation speed, strong spatial feature extraction ability, high effectiveness, improved model performance, etc. The DW convolution is introduced in the bottleneck of the original YOLOv5n model C3 module, which can be applied to devices with limited computing resources. In terms of activation function, FReLU is adopted, because the ReLU and PReLU are extended to 2D activation function with only a little increase in calculation burden, the problem of spatial insensitivity in the activation function is solved, the regular (ordinary) convolution also has the ability to capture complex visual layout, and the model has the ability of pixel-level modeling. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 It is an operation flow diagram of the application.

[0040] Figure 2 It is a schematic diagram of the YOLOv5n-MN model in the application.

[0041] Figure 3 It is a structure diagram of MN_C3 in the application.

[0042] Figure 4 It is a structure diagram of MN_Bottleneck1 in the application.

[0043] Figure 5 It is a structure diagram of MN_Bottleneck2 in the application.

[0044] Figure 6 It is a model parameter diagram in the embodiment.

[0045] Figure 7 Model training duration graph for the embodiment.

[0046] Figure 8 Practical effect graph for case 1.

[0047] Figure 9 Model average frame rate graph for case 2.

[0048] Figure 10 YOLOv5n frame rate graph for case 2.

[0049] Figure 11 YOLOv5n-MN frame rate graph for case 2. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. The described embodiments are only some of the embodiments of the present application, not all. Without departing from the design concept of the present application, various modifications and improvements to the technical solutions of the present application made by those skilled in the art shall fall within the protection scope of the present application.

[0051] Embodiment 1:

[0052] As shown in Figure 1 An improved YOLOv5n model for traffic signal detection makes the model lightweight, faster in training and inference, improves the robustness of the model, and enables the model to maintain relatively stable prediction results for disturbances or noise in input data, making it more suitable for complex and variable real-world scenarios. The specific optimization method steps include:

[0053] S1, obtain the image data set to be detected, perform data preprocessing, and divide it into a training set and a test set.

[0054] The image data set to be detected uses the traffic signal data set (S2TLD) released by Shanghai Jiaotong University, searches for various road scenes and typical data with keywords such as "red", "yellow", "green", and "wait_on", and collects a total of 5786 image data.

[0055] The image data set is cleaned, and operations such as removing damaged images, removing duplicate images, and removing non-keyword images are performed on the crawled data. After cleaning the image data, the data set is labeled, and the data set is divided into a training set and a test set according to an 8:2 ratio.

[0056] S2, replace the Convolution convolution kernel with 3 and step 2 in the head layer of the YOLOv5n model with involution convolution; the specific operation mode is:

[0057] First step: modify the common file and put the involution module into it;

[0058] Second step: modify the YOLO file and add the involution module in the parse_model function of YOLO.

[0059] In the field of autonomous driving, embedded devices are generally used, and the processor performance of embedded devices is usually limited. However, the autonomous driving system needs to perform complex image processing and decision-making calculations, so efficient algorithms and optimized code are needed to achieve a balance between real-time performance and accuracy. Therefore, the primary goal of the model is to be lightweight.

[0060] Involution convolution is a new convolution operation that adopts a different idea from traditional convolution. It decomposes the input tensor into two sub-tensors and performs a nonlinear transformation on one of them, and then combines the two sub-tensors. This operation can be represented by the following formula:

[0061]

[0062] In the formula: x j is the jth element of the input tensor, y i is the ith element of the output tensor, w ij and b ij are the weights and biases of involution convolution, respectively, and k is the total number of blocks; the form of this nonlinear transformation is similar to the ReLU activation function, which is performed on the sub-tensor.

[0063] Involution convolution is more lightweight and efficient than convolution, and more concise in form than self-attention, which can achieve double improvement in accuracy and efficiency on various visual task models. Because involution convolution requires the input channel and the output channel to be consistent, it replaces the convolution module with consistent input and output channels in the head layer of the YOLOv5n model, reducing the burden of the model.

[0064] The structure of MN_C3 is shown in Figure 3 , but in the backbone layer, MN_C3 adopts the design of residual edges, as shown in Figure 4As shown, not only can the problem of gradient disappearance be effectively alleviated, but also the nonlinear expression capability of the network can be increased. In the neck layer MN_C3, a residual edge is not needed, as shown in FIG. 5, because there are multiple branches, the depth of the network is shallow, the problem of gradient disappearance is light, and therefore a residual edge is not needed, and moreover, not using a residual edge can reduce the amount of network parameters and improve the running efficiency of the model.

[0065] S3, improving the Bottleneck in the C3 module of the YOLOv5n model, is introducing a DW convolution in the Bottleneck of the original C3 module of the YOLOv5n model to form an MN_Bottleneck composed of a depth separable convolution network and a FReLU activation function. The specific operation mode is as follows:

[0066] Step a: modify the common file and add Conv_FReLU, MN_Bottleneck and MN_C3 module therein;

[0067] Step b: modify the YOLO file and add the MN_C3 module in the parse_model function of YOLO;

[0068] Step c: create a yolov5n-MN.yaml file under the models folder;

[0069] Step d: change the original yaml file in train.py to the yolov5n-MN.yaml file in the previous step to train.

[0070] DW (depth separable convolution) convolution has the advantages of less parameter amount, fast calculation speed, strong spatial feature extraction capability, high effectiveness and improved model performance. Introducing DW convolution in the Bottleneck of the original C3 module of the YOLOv5n model can be applied to devices with limited computing resources. In terms of activation function, FReLU is used because it extends ReLU and PReLU into a 2D activation function with only a little bit of additional computational burden, solves the problem of spatial insensitivity in the activation function, enables regular (ordinary) convolution to also have the ability to capture complex visual layouts, and enables the model to have the ability of pixel-level modeling.

[0071] Introducing a depth separable convolution network into the Bottleneck network of the YOLOv5 C3 feature extraction layer forms an MN_Bottleneck, which is composed of a 1x1 convolution, a DW convolution and a 1x1 convolution.

[0072] DW convolution: Depthwise Separable Convolution, is a convolution operation commonly used in lightweight neural network design. DW convolution performs convolution operation on each input channel with an independent convolution kernel, thereby realizing the extraction of spatial information without designing information interaction between different channels. The operation can be represented by the following formula:

[0073]

[0074] In the formula, m is the mth row of the convolution kernel, n is the nth column of the convolution kernel, x (i*8+m),(j*8+n),k is the pixel value of the input feature map at the (i*8+m)th row, (j*8+n)th column, and kth channel, and w is the weight value.

[0075] MN_Bottleneck selects FRelu activation function, which is intended to solve the SiLu function space insensitive problem, so that ordinary convolution also has the ability to capture complex visual layout, and the model has the ability of pixel-level modeling.

[0076] SiLu function, also known as Sigmoid-Weighted Liner Unit or Swish activation function, is a commonly used neural network activation function, and its specific mathematical formula is as follows:

[0077]

[0078] In the formula, e is the natural exponential.

[0079] Although SiLU function performs better in some scenarios, it also has some disadvantages, such as non-monotonicity, high computational cost, and gradient disappearance. Overall, although SiLU function performs well in some scenarios, its advantages and disadvantages need to be weighed in practical applications.

[0080] FReLU is a Flexible Rectified Linear Unit, which is an adjustable activation function that can adaptively adjust the shape of the function according to different inputs to adapt to different data features. It is an extension of ReLU function and can solve the problem of dead neurons in ReLU function. It expands the ReLU function by adding a scientific system parameter, so that FReLU can learn more feature representations. The mathematical formula of FReLU is as follows:

[0081] FReLU(x)=max(0,x)+min(0,α(e x-β -1))#(4)

[0082] In the above formula, max(0, x) represents the ReLU function, min(0, a(e x-β -1)) is the extended part of FReLU, and a and b are learnable parameters for controlling the shape of FReLU.

[0083] The biggest advantage of FReLU compared to SiLU is that it has stronger non-linear representation ability. In the activation function, the stronger the non-linear representation ability, the stronger the fitting ability of the neural network. FReLU introduces two learnable parameters, which makes it have stronger non-linear representation ability and can better adapt to complex data distribution. In addition, FReLU also has an advantage that it can set the activation value of negative numbers to zero, which can effectively avoid the problem of gradient disappearance, making the network training more stable.

[0084] S4, train the detection model using the dataset, use the trained detection model to detect traffic signal lights, and evaluate the detection results.

[0085] The improved YOLOv5 algorithm is inputted with the dataset image, and the optimal weight file for traffic signal lamp target detection is obtained through continuous iteration training. The essence is to input the position, size and category information of the target object into the model for learning through a large number of labeled data. In the learning process, the model will continuously adjust its internal parameters, so that it can more accurately detect the target object in the input image.

[0086] The Predicted-True implementation data reflects the probability of the algorithm predicting correctly, and in this experiment, it reflects the correct prediction of the algorithm for the four categories of "Green", "Red", "Yellow" and "Wait_on".

[0087] The Predicted-True implementation data reflects the probability of the algorithm predicting correctly, and in this experiment, it reflects the correct prediction of the algorithm for the four categories of "Green", "Red", "Yellow" and "Wait_on". The data analysis of the probability of the algorithm predicting correctly is shown in Table 1:

[0088] Table 1 Accuracy of each class in the dataset

[0089] Predicted-True Green Red Yellow Wait_on YOLOv5n 0.94 0.90 0.79 0.92 YOLOv5n-MN 0.92 0.90 0.67 0.92

[0090] The Predicted-Recall experimental data reflects the mean precision (AP). The area enclosed by the Predicted-Recall values ​​is the mean precision. mAP@0.5 is the average of the AP calculated for each class. mAP@0.5:0.95 represents the average mAP over different IoU thresholds (from 0.5 to 0.95, with a step size of 0.05) (0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95). The data analysis of the mean precision for each class in the dataset is shown in Table 2 below:

[0091] Table 2 shows the average accuracy of each category in the dataset.

[0092]

[0093] F1 considers both precision and recall, and uses a unified single-value evaluation metric to assess the model's training performance. The F1-Confidence experimental data reflects the differences in algorithm performance; a higher F1 value indicates better algorithm performance. The data analysis of the overall model metric F1 is shown in Table 3 below:

[0094] Table 3 Model Composite Index F1

[0095] F1-Confidence allclasses YOLOv5n 0.409-0.86 YOLOv5n-MN 0.393-0.79

[0096] By modifying the model, the number of model parameters decreased significantly, and the training time also improved markedly. Data analysis of model parameter count and training time is as follows: Figure 6 and Figure 7 As shown. Case 1:

[0097] One hundred traffic light images were randomly sampled as a validation set. Predictions were made using these images, and the average confidence scores were calculated. The values ​​are shown in Table 4.

[0098] Table 4. Average confidence level of model validation

[0099]

[0100] The average confidence score for the "Yellow" class is relatively lower than that of other classes because there are fewer "Yellow" class images compared to other classes during model training. Practical results are as follows... Figure 3 As shown.

[0101] Case 2:

[0102] We selected a 30-minute video and analyzed the average frame rate of the Raspberry Pi when processing complex images under high load, such as... Figure 8 As shown in the image. The YOLOv5n frame rate graph is as follows. Figure 9As shown, the YOLOv5n-MN frame rate diagram is as follows Figure 10 As shown.

[0103] The above merely illustrates the specific embodiments of the present application, but the protection scope of the present application is not limited to this, any change, replacement and improvement within the technical scope disclosed by the present application are within the protection scope of the present application.

Claims

1. An improved YOLOv5n model for traffic signal detection, characterized in that: Lighten the model, make the training and inference faster, improve the robustness of the model, and make the model keep relatively stable prediction results for interference or noise in the input data, making it more suitable for complex and variable real scenes; The steps of the specific optimization method include: S1, obtaining the image data set to be detected, performing data preprocessing, and dividing into a training set and a test set; S2, replacing the Convolution convolution with a convolution kernel of 3 and a step of 2 in the head layer of the YOLOv5n model with an involution convolution; S3, improving the Bottleneck in the C3 module of the YOLOv5n model, introducing a depth separable convolution network and a FReLU activation function to form an MN_Bottleneck; The improvement of the Bottleneck in the C3 module of the YOLOv5n model is to introduce a DW convolution in the Bottleneck of the original YOLOv5n model C3 module, and the specific operation mode is: Step a: modify the common file, and add Conv_FReLU, MN_Bottleneck and MN_C3 module in it; Step b: modify the YOLO file, and add the MN_C3 module in the parse_model function of YOLO; Step c: create a yolov5n-MN.yaml file under the models folder; Step d: change the original yaml file in train.py to the yolov5n-MN.yaml file in the previous step for training; S4, training the improved YOLOv5n detection model on the data set, using the trained detection model to detect traffic signal lights, and evaluating the detection results.

2. The improved YOLOv5n model for traffic signal detection according to claim 1, wherein: The image data set to be detected adopts the traffic signal lamp data set S2TLD, and various road scenes and typical picture data are obtained by searching with the keywords of "red", "yellow", "green" and "wait_on"; the picture data set is cleaned, and the crawled data is removed damaged images, repeated images and non-keyword images.

3. The improved YOLOv5n model for traffic signal detection according to claim 2, wherein: The division into a training set and a test set in step 1 is to label the data set after image data cleaning, and divide the data set into a training set and a test set according to 8:

2.

4. The improved YOLOv5n model for traffic signal detection according to claim 1, wherein: The YOLOv5n model head layer in step 2 is replaced with an involution convolution instead of a convolution module with consistent input and output channels; the specific operation mode is: First step: modify the common file and put the involution module into it; Second step: modify the YOLO file, and add the involution module in the parse_model function of YOLO; The involution convolution decomposes the input tensor into two sub-tensors, and performs nonlinear transformation on one of the sub-tensors, and then combines the two sub-tensors; this operation can be represented by the following formula: ； where: is the element of the input tensor, is the element of the output tensor, and are the weights and biases in the involution convolution, respectively, is the total number of blocks; the form of this non-linear transformation is similar to the ReLU activation function, performed on the sub-tensors.​​ 5. The improved YOLOv5n model for traffic signal detection according to claim 1, wherein: The MN_Bottleneck in step a is composed of a 1x1 convolution, a DW convolution and a 1x1 convolution; the DW convolution is a depth separable convolution, which convolves each input channel with an independent convolution kernel; the operation is represented by the following formula: ； In the formula, m is the mth row of the convolution kernel, and n is the nth column of the convolution kernel, is the pixel value of the input feature map at the (i*8+m)th row, the (j*8+n)th column, and the kth channel, and w is a weight value.

6. The improved YOLOv5n model for traffic signal detection according to claim 5, wherein: The MN_Bottleneck selects the FRelu activation function, solves the SiLu function space insensitivity problem, makes ordinary convolution also has the ability to capture complex visual layout, and makes the model have the ability of pixel-level modeling; the specific mathematical formula of the SiLu function is: ； In the formula, e is the natural exponential.

7. The improved YOLOv5n model for traffic signal detection according to any one of claims 5-6, characterized in that: The FReLU in step a is an adjustable shape activation function, which expands the ReLU function by adding a scientific system parameter, so that FReLU can learn more feature representations, and the mathematical formula of FReLU is: ； In the above formula, denotes the ReLU function, is the extended part of FReLU, and are learnable parameters for controlling the shape of FReLU.

8. The improved YOLOv5n model for traffic signal detection according to claim 1, wherein: The data set training detection model in step S4 is to input the improved YOLOv5 algorithm into the data set image, and obtain the optimal weight file of the traffic signal lamp target detection through continuous iteration training; The essence is to input the position, size and category information of the target object into the model for learning through a large amount of labeled data; In the learning process, the model will continuously adjust its internal parameters, so that it can more accurately detect the target object in the input image.

Images