Improved lightweight traffic sign detection method

Abstract

The invention relates to the technical field of image processing, in particular to an improved lightweight traffic sign detection method, which comprises the following steps: collecting sign image data, and preprocessing an image; the method comprises the following steps: replacing a backbone network of YOLOv8n by using an improved MobileViT, replacing a common convolution of the MobileViT by using an ODConv convolution, and introducing a PSAModule block; carrying out deep fusion on the improved MobileViT feature extraction information by utilizing an AFPN (Attenuous Fabry-Perot Networking) to obtain an improved YOLOv8n network; a K-fold cross validation method is adopted to train the improved YOLOv8n network; and evaluating the improved YOLOv8n network by using the evaluation index. The problem that an existing traffic sign detection model cannot give consideration to accuracy and efficiency at the same time is solved.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to an improved and lightweight traffic sign detection method. Background Art

[0002] With the accelerated development of urbanization, the road traffic network is becoming increasingly complex, which puts higher requirements on traffic management; traditional traffic sign detection systems usually rely on large computing equipment and complex algorithm models, which not only increases the cost of the system, but also limits its application in edge computing environments. Especially with the rapid development of Internet of Things (IoT) technology, intelligent transportation systems (ITS) are increasingly inclined to adopt distributed architectures to improve response speed and data processing capabilities. Therefore, the development of a lightweight traffic sign detection method has become an urgent need.

[0003] The publication number is CN118570767A. The SPD-Conv convolution replaces the backbone convolution layer, and the SPD-Conv convolution module is used to enhance the backbone network's ability to extract small target features. The attention mechanism of the C2f-ECA module is used to improve detection accuracy. The main focus is on improving the accuracy of small target detection, and the lightweight problem is not considered.

[0004] The shortcoming of the existing lightweight methods is that the feature extraction information is insufficient, resulting in a decrease in model accuracy and the inability to take into account both accuracy and model size at the same time; Liu Jun et al.'s traffic sign detection algorithm based on improved YOLOv4 uses YOLOv4 to perform real-time detection of traffic signs, with a lightweight network Mobilenetv2 as the backbone feature extraction network. The model size is 56.8Mbit. Although the network is lightweight, the size is still large and it is not easy to deploy in small embedded or mobile devices. Summary of the invention

[0005] In view of the shortcomings of the existing methods, the present invention aims to solve the problem that the existing traffic sign detection model cannot take into account both accuracy and efficiency at the same time.

[0006] The technical solution adopted by the present invention is: an improved lightweight traffic sign detection method comprises the following steps:

[0007] Step 1: Collect sign image data and pre-process the image;

[0008] As a preferred implementation of the present invention, the preprocessing includes: slicing the sign image using SAHI.

[0009] Step 2: Use the improved MobileViT to replace the YOLOv8n backbone network, use the ODConv convolution to replace the ordinary convolution of MobileViT, and introduce the PSAModule block; use AFPN to deeply fuse the improved MobileViT feature extraction information to obtain the improved YOLOv8n network;

[0010] As a preferred embodiment of the present invention, the improved MobileViT includes: ODConv, a first MV2Block layer, a second MV2Block layer, a third MV2Block layer, a first IMobileViTBlock layer, a fourth MV2Block layer, a second IMobileViTBlock layer, a fifth MV2Block layer, a third IMobileViTBlock layer, a sixth MV2Block layer, a PSAModule, a fourth IMobileViTBlock layer and Conv connected in sequence.

[0011] As a preferred implementation manner of the present invention, the IMobileViTBlock layer replaces Conv in the MobileViTBlock layer with ODConv.

[0012] As a preferred implementation of the present invention, the Conv convolution in AFPN is replaced by ODConv convolution.

[0013] As a preferred embodiment of the present invention, the YOLOv8n backbone network replacement also includes: EfficientNet, ShuffleNetv2, FasterNet, MobileNetv3.

[0014] Step 3: Use K-fold cross-validation method to train the improved YOLOv8n network;

[0015] Step 4: Use evaluation indicators to evaluate the improved YOLOv8n network;

[0016] As a preferred implementation of the present invention, the evaluation indicators include: precision, recall, F1 and map.

[0017] As a preferred embodiment of the present invention, an improved lightweight traffic sign detection system includes: a memory for storing instructions executable by a processor; and a processor for executing the instructions to implement an improved lightweight traffic sign detection method.

[0018] As a preferred embodiment of the present invention, a computer readable medium stores a computer program code, and when the computer program code is executed by a processor, an improved lightweight traffic sign detection method is implemented.

[0019] Beneficial effects of the present invention:

[0020] 1. In view of the problem that the traditional traffic sign detection system is not conducive to deployment on embedded devices due to complex network and large amount of calculation, by improving the lightweight traffic sign detection model, the model calculation amount is greatly reduced, which is more conducive to deployment on embedded devices;

[0021] 2. Through ablation experiments and comparison with existing models, the detection speed of the model of the present invention is significantly improved on the basis of a certain improvement in detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is the improved lightweight feature extraction network MobileViT model of the present invention;

[0023] Figure 2 It is a module structure diagram of MobileViT Block of the present invention;

[0024] Figure 3 is a structural diagram of the AFPN of the present invention;

[0025] Figure 4 It is a structural diagram of the PSAModule of the present invention;

[0026] Figure 5 is a structural diagram of ODConv of the present invention;

[0027] Figure 6 It is a detection effect diagram of the present invention. DETAILED DESCRIPTION

[0028] The present invention is further described below in conjunction with the accompanying drawings and embodiments. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore it only shows the components related to the present invention.

[0029] like Figure 1 As shown, an improved lightweight traffic sign detection method includes the following steps:

[0030] Step 1: Collect sign image dataset and perform SAHI slice preprocessing;

[0031] Based on the public dataset CTSDB (Chinese Traffic Sign Database) of Chinese traffic sign images, this paper studies the traffic sign detection method based on improved YOLOv8n for unmanned driving with sign identification information as the object; CTSDB is a database specifically for Chinese road traffic signs. CTSDB contains a large number of Chinese-specific traffic sign images, which are collected from actual road environments and cover a variety of different weather conditions and lighting conditions, making the system developed based on this dataset highly adaptable and robust.

[0032] The specific implementation steps of SAHI (Sice Aware HyperScale Infrastructure) slice preprocessing technology are as follows:

[0033] First, define the size of the slice (H s ,W s ) and the size of the overlapping area (O h ,O w ), calculate the number of slices N h and N w :

[0034]

[0035] Generate slice coordinates (y,x):

[0036] (y,x)=(i·(H s -O h ),j·(W s -O w )) (2)

[0037] Next, each slice is cropped and preprocessed;

[0038] Each slice is then independently input into the object detection or segmentation model for inference to generate a prediction result;

[0039] Finally, the prediction results of all slices are merged to generate the final complete prediction result by processing the overlapping areas and performing non-maximum suppression (NMS).

[0040] This process not only effectively solves the problems of memory limitations and computing resources, but also ensures the efficiency and accuracy of object detection and classification tasks.

[0041] Step 2: Replace the YOLOv8n backbone network with the improved MobileViT, use ODConv convolution to replace the ordinary convolution of MobileViT, and introduce the PSAModule block; use AFPN to deeply fuse the feature extraction information of the improved MobileViT;

[0042] Among them, the MobileViT model includes ODConv, the first MV2Block layer, the second MV2Block layer, the third MV2Block layer, the first IMobileViTBlock, the fourth MV2Block layer, the second IMobileViTBlock, the fifth MV2Block layer, the third IMobileViTBlock, the sixth MV2Block layer, the PSAModule, the fourth IMobileViTBlock and Conv connected in sequence; Figure 1 The middle green is the improvement point of the present invention.

[0043] Among them, the third MV2Block layer and the first IMobileViTBlock constitute the C2 layer; the fourth MV2Block layer and the second IMobileViTBlock constitute the C3 layer; the fifth MV2Block layer and the third IMobileViTBlock constitute the C4 layer; the sixth MV2Block layer, PSAModule and the fourth IMobileViTBlock constitute the C5 layer.

[0044] Compared with the existing MobileViT model, the Conv convolution after the input image is first replaced with the ODConv convolution; this is because there are small targets in the traffic sign image. ODConv can simultaneously consider the four dimensions of the number of convolution kernels, the spatial kernel size, the number of input channels and the number of output channels, and use the multi-dimensional attention mechanism to learn four types of attention along the four dimensions of the kernel space in parallel, so that the traffic signs can be accurately located in complex scenes; during driving, affected by the distance, it is necessary to obtain the traffic sign information ahead in advance, and the target sign to be detected is usually small; since the convolution kernel of ordinary convolution is static, it is easy to cause parameter redundancy, and ordinary convolution only considers local pixel relationships and cannot capture global information, which is not conducive to quickly and accurately detecting smaller traffic sign targets in complex scenes; ODConv uses a parallel strategy to learn all dimensions of the space where the convolution kernel is located, so that the model can extract features more accurately and enhance the network's feature extraction ability for small targets.

[0045] Secondly, compared with the existing MobileViT model that adds a second MV2Block layer, the existing MobileViT model only has the first MV2Block layer, and the present invention adds a second MV2Block layer; since traffic signs are small targets in the detection picture, it is difficult to detect them in complex scenes, and the design of MV2Block adopts an inverted residual structure and a linear bottleneck, which can effectively capture local features in the image, and can also adapt to features of different scales by adjusting its internal structure (such as expansion rate, number of channels, etc.), so as to better prepare input to the subsequent Transformer part, which helps the model learn more levels of feature representation, so an additional MV2Block layer is added to the MobileViT model to enable the model to learn more levels of feature representation.

[0046] Secondly, a C2 layer structure is added. The existing MobileViT model does not have a C2 layer structure, which is a cascade of two MV2Block layers. The present invention changes it to MV2Block and IMobileViTBlock. By adding the C2 layer structure as the basis for forming the small target prediction layer P2, the network model's small target detection performance is enhanced.

[0047] Secondly, compared with the MobileViTBlock in the existing MobileViT model, the improvement of IMobileViTBlock is that Conv in MobileViTBlock is replaced by ODConv. The structure is as follows Figure 2 As shown in the figure, the Conv in MobileViTBlock is replaced by ODConv, which aims to model the local and global information in the input tensor with fewer parameters. Since the design of ODConv adopts weight sharing and parallel computing to effectively utilize computing resources, it is possible to maintain or even reduce the computational cost even if directional processing is added, so the Conv in MobileViTBlock is replaced by ODConv.

[0048] Finally, compared with the C5 layer of the existing MobileViT model, the PSAModule block is inserted between the MobileViTBlock and the MV2Block. The PSA mechanism can be used to efficiently extract more fine-grained multi-scale spatial information of traffic signs and establish longer-range channel dependencies. After embedding this module in the neural network, the network performance can be effectively improved while reducing the amount of calculation; Figure 4As shown in the figure, the PSAModule block first processes the feature map of size H×W×C through the Squeeze and Concat (SPC) module in the channel dimension to obtain a multi-scale feature map; then uses the SEweight module to extract the channel attention vector from the feature maps of different scales; then, the channel attention vector is recalibrated through the softmax function; finally, the recalibrated weight is combined with the corresponding feature map through element-level multiplication operation to generate a refined feature map rich in multi-scale feature information.

[0049] like Figure 5 As shown in the figure, ODConv uses a generative network to dynamically generate convolution kernels based on the local information of the input feature map; this process enables the convolution kernel to adaptively adjust its shape and size to better match features of different scales; in this way, ODConv can not only effectively capture multi-scale information, but also enhance the expressiveness of the model, enabling the network to learn complex patterns and structures more flexibly; in addition, ODConv optimizes the design of the generative network and the parameter sharing strategy to ensure that computational efficiency is maintained while improving performance.

[0050] The lightweight extraction network of the YOLOv8n backbone network can also include: EfficientNet, ShuffleNetv2, FasterNet, and MobileNetv3.

[0051] like Figure 3 As shown in the figure, in AFPN (Adaptive Feature Pyramid Network), the feature map generation and fusion process includes a bottom-up path, a top-down path, and a lateral connection; first, the input image is extracted layer by layer through a series of convolutional layers to form a feature map sequence C1, C2, ..., C5 from low to high levels; then, starting from the highest level feature map C5, the resolution of the previous level feature map C4 is restored through an upsampling operation, and C4 is convolved 1x1 to match the number of channels, and then the upsampled feature map is fused with the feature map after 1x1 convolution through an addition operation , generating a new feature map P4; this process is recursive layer by layer from high to low until feature maps P2,…,P5 of all levels are generated; in addition, AFPN introduces a weight learning module to generate dynamic weights wi for each level of feature maps, and performs weighted fusion of feature maps through Pi=wi·Pi+(1-wi)·Ci′, further enhancing the flexibility and adaptability of feature fusion; finally, these processed multi-scale feature maps P2,…,P5 are sent to the detection head to generate candidate boxes and classification scores, thereby achieving efficient target detection.

[0052] The present invention improves AFPN by replacing the Conv convolution in AFPN with ODConv convolution, further enhancing the extraction capability of the network; since ODConv adopts parallel computing, the computational complexity of the model is greatly reduced, thereby better realizing the lightweight of the network and improving the detection efficiency of traffic signs.

[0053] Step 3: Use the K-fold cross-validation method to train the improved model to address the problems of sparse data set samples and imbalanced categories;

[0054] The advantages of K-fold cross-validation are: by dividing the dataset into K mutually exclusive subsets, it ensures that each sample has the opportunity to appear as a test set once, thereby making better use of data resources and improving data utilization efficiency; each subset is used as a test set in turn, which helps to reduce the model's dependence on a specific training set and reduce the risk of overfitting; through multiple training and testing processes, and taking the average of the results to evaluate the model performance, it can effectively reduce performance fluctuations caused by different data divisions and improve the robustness of model evaluation.

[0055] Specific steps:

[0056] 1. Dataset partitioning: randomly divide the original dataset into K subsets, ensuring that the number of samples in each subset is similar; usually, the value of K is 5 or 10, but other values ​​can also be selected in some cases;

[0057] 2. Training and testing: Perform K rounds of iterations. In each round, select one subset as the test set and the remaining K-1 subsets as the training set. Then use the training set to train the model and evaluate the model performance on the test set.

[0058] 3. Evaluation indicators: Calculate the performance indicators of the model on each round of test sets, such as accuracy, precision, recall, F1 score, etc.;

[0059] 4. Performance evaluation: After completing k rounds of iterations, average the results of K performance indicators to obtain the final performance evaluation result of the model.

[0060] An improved lightweight traffic sign detection model is trained using the dataset.

[0061] Step 4: Use precision, recall, F1 and map value indicators to evaluate and improve the lightweight traffic sign detection model.

[0062] Experimental results:

[0063] Taking the CTSDB traffic dataset as the research object, in order to test the effectiveness of the improved lightweight traffic sign detection method, this study uses the PyTorch 1.8.0 deep learning framework, Pycharm compiler, and python3.7 programming language on NVIDIA Tesla V100 32G, operating system Ubuntu 16.04, processor Intel Xeon Gold 5117, and memory 32GB platform to train the data set model. The SGD optimizer is used in the experiment, the random seed is set to 2, the learning rate is 0.01, the SGD optimizer momentum is 0.937, the weight decay is 0.0005, the iteration is 300 rounds, and the batch-size is 16.

[0064] In order to analyze the performance of the model, the experiment uses precision P (Precision), recall R (Recall), F1 and map value indicators to evaluate the detection effect of the model.

[0065] In order to deeply analyze and explore the superiority of the improved model of the present invention, a fusion experiment was carried out to fuse the YOLOv8n model, the improved MobileViT structure, ODConv, the AFPN module and the PSAModule. The results of the fusion experiment are shown in Table 1.

[0066] Table 1 Fusion test

[0067]

[0068]

[0069] According to the lightweight model ablation experiment results in Table 1, it can be seen that the improved traffic detection model has an accuracy P and average precision mAP of 8.4 and 3.3 percentage points higher than the YOLOv8n network model respectively; therefore, the improved method can improve the detection accuracy of traffic detection signs while achieving the goal of lightweight.

[0070] To verify the effectiveness of the model of the present invention, the mainstream lightweight backbone network algorithms EfficientNet, ShuffleNetv2, FasterNet, and MobileNetv3 were used to train the model under the same data set and the same configuration. The present method has better performance. The results of the comparative test are shown in Table 2.

[0071] Table 2 Comparative test

[0072]

[0073] Note: AOP is the abbreviation of AFPN, ODConv, and PSAModule.

[0074] Through the experimental verification of the above method, it can be known that the method YOLOv8n-MobileViT_AOP proposed in the present invention is more effective in lightweight traffic detection; its average precision map is 4.3, 5.7, 4.5, 2.6 and 2.4 percentage points higher than the original YOLOv8n model and the other four lightweight models respectively; the precision P is increased by 8.4, 7.7, 6.2, 4.8, and 5.0 percentage points respectively; the recall rate R is increased by 9.7, 5.6, 2.2, 10.6 and 9.8 percentage points respectively. At the same time, the size of the lightweight model is compressed by 27% compared with the original YOLOv8n; it can be seen that the model of the present invention is not only greatly reduced, but also the accuracy is significantly improved; indicating that the method proposed in the present invention is effective. Finally, the model of the present invention is deployed in the JETSONAGX ORIN CLB development kit. It takes an average of 24ms to detect a 2048×2048 pixels image, which means that 41.7 frames of images can be processed per second, realizing real-time detection of traffic indication information. This shows that the model of the present invention can effectively realize real-time judgment of traffic information in the process of unmanned driving on the JETSONAGX ORIN CLB. Figure 6 This is the actual detection effect diagram.

[0075] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. An improved and lightweight traffic sign detection method, characterized in that: The following steps are involved: Step 1: Collect sign image data and pre-process the image; Step 2: Use the improved MobileViT to replace the YOLOv8n backbone network, use the ODConv convolution to replace the ordinary convolution of MobileViT, and introduce the PSAModule block; use AFPN to deeply fuse the improved MobileViT feature extraction information to obtain the improved YOLOv8n network; Step 3: Use K-fold cross-validation method to train the improved YOLOv8n network; Step 4: Use evaluation indicators to evaluate the improved YOLOv8n network.

2. The improved lightweight traffic sign detection method according to claim 1, characterized in that: The improved MobileViT includes: ODConv, the first MV2Block layer, the second MV2Block layer, the third MV2Block layer, the first IMobileViTBlock layer, the fourth MV2Block layer, the second IMobileViTBlock layer, the fifth MV2Block layer, the third IMobileViTBlock layer, the sixth MV2Block layer, PSAModule, the fourth IMobileViTBlock layer and Conv, which are connected in sequence.

3. The improved lightweight traffic sign detection method according to claim 2, characterized in that: The IMobileViTBlock layer replaces Conv in the MobileViTBlock layer with ODConv.

4. The improved lightweight traffic sign detection method according to claim 1, characterized in that: Replace the Conv convolution in AFPN with ODConv convolution.

5. The improved lightweight traffic sign detection method according to claim 1, characterized in that: The backbone networks that replace YOLOv8n also include: EfficientNet, ShuffleNetv2, FasterNet and MobileNetv3.

6. The improved lightweight traffic sign detection method according to claim 1, characterized in that: Evaluation indicators include: precision, recall, F1 and map.

7. The improved and lightweight traffic sign detection method according to claim 1, characterized in that: The preprocessing includes: using SAHI to slice the sign image.

8. Improved lightweight traffic sign detection system, characterized in that: include: a memory for storing instructions executable by a processor; A processor, used for executing instructions to implement the improved lightweight traffic sign detection method as described in any one of claims 1-7.

9. A computer readable medium storing a computer program code, characterized in that: When the computer program code is executed by a processor, the computer program code implements the improved lightweight traffic sign detection method as described in any one of claims 1 to 7.

Images