Traffic sign small target detection method based on improved SSPDO-YOLO

By improving the YOLOv8 model to SSPDO-YOLO, using the feature integration module of C2f-SMPCGLU and P2 detection layer, the accuracy and speed problems of YOLOv8 when detecting small targets in complex environments are solved, and more efficient traffic sign detection is achieved.

CN118942071APending Publication Date: 2024-11-12HUAIYIN INSTITUTE OF TECHNOLOGY

Patent Information

Application Number
CN202411057491.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-02
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The YOLOv8 algorithm lacks the accuracy and speed of long-distance detection of traffic signs in complex road environments, especially when the on-board camera detects small targets too slowly.

Method used

Using the improved SSPDO-YOLO model, by replacing the C2f module with the C2f-SMPCGLU module, adding the P2 detection layer and using the SPDConv and CSPOmniKernel modules, combining the convolution and gating mechanism, effective learning from global to local feature representation is achieved.

Benefits of technology

The performance and efficiency of traffic sign small target detection is improved, the amount of model calculation is reduced, and the detection accuracy is maintained.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118942071A_ABST
    Figure CN118942071A_ABST
Patent Text Reader

Abstract

The invention relates to the field of target detection, in particular to a traffic sign small target detection method based on improved SSPDO-YOLO, and the method comprises the steps: collecting various traffic sign pictures through a vehicle-mounted monitor, and carrying out the preprocessing of the pictures, and obtaining a data set; a network framework is improved based on a YOLOv8 model, and an improved SSPDO-YOLO model is obtained; the method comprises the following steps: 1) replacing a C2f module of an original YOLOv8 model with a C2f-SMPCGLU module; 2) adding a P2 detection layer, enabling the P2 detection layer to pass through an SPDConv module, fusing the P2 detection layer with a P3 detection layer, and performing feature integration by using a CSPOmniKernel module; and performing traffic sign target detection on the processed data set by using an SSPDO-YOLO model. The method can ensure that the precision is equivalent to that of an existing algorithm, meanwhile, the model is simpler, the calculated amount is remarkably reduced, global to local feature representation can be learned more effectively, and finally the detection performance of small targets such as traffic signs is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of target detection, and in particular to a traffic sign small target detection method based on improved SSPDO-YOLO. Background Art

[0002] With the continuous advancement of science and technology and the gradual popularization of driverless technology, the rapid and accurate detection of traffic signs is a key factor in ensuring the safety of driverless vehicles. Nowadays, the YOLOv8 algorithm is mainly used for traffic sign detection. On the basis of inheriting the high-efficiency performance of the previous version, it further improves the recognition accuracy and speed by improving the network architecture and optimizing the algorithm. YOLOv8 adopts a new backbone network architecture, Anchor-Free detection head and loss function, which can more effectively process the detailed features of traffic signs, thereby achieving higher recognition accuracy in complex traffic scenes.

[0003] But it has the following disadvantages:

[0004] Due to the complexity of factors such as actual road weather and environment, the YOLOv8 algorithm cannot guarantee accuracy in actual applications. During the unmanned driving process, the on-board camera detects traffic signs at a long distance, and the YOLOv8 algorithm detects too slowly because the detection target is too small. Summary of the invention

[0005] Purpose of the invention: In response to the problems pointed out in the background technology, the present invention discloses a method for detecting small targets of traffic signs based on improved SSPDO-YOLO, so that the model can more effectively learn feature representation from global to local, and ultimately improve the detection performance of small targets such as traffic signs.

[0006] Technical solution: The present invention discloses a traffic sign small target detection method based on improved SSPDO-YOLO, comprising the following steps:

[0007] Step 1: Collect various traffic sign images using vehicle-mounted monitoring and pre-process the images to obtain a data set;

[0008] Step 2: Improve the network framework based on the YOLOv8 model to obtain an improved SSPDO-YOLO model; the specific improvements of the SSPDO-YOLO model are as follows:

[0009] 1) Use the C2f-SMPCGLU module to replace the C2f module of the original YOLOv8 model;

[0010] 2) Add the P2 detection layer. The P2 detection layer first passes through the SPDConv module, and then is fused with the P3 detection layer and the CSPOMniKernel module is used for feature integration.

[0011] Step 3: Use the SSPDO-YOLO model to perform traffic sign target detection on the processed dataset.

[0012] Furthermore, the C2f-SMPCGLU module first adopts the SMPConv module, and then adds the CGLU module after the SMPConv module, combining convolution and gating mechanisms;

[0013] 3. The method for detecting small traffic signs based on improved SSPDO-YOLO according to claim 2, characterized in that the SMPConv module implements a continuous convolution operator. In one dimension, the continuous convolution is expressed as:

[0014] (f*SMP)(x)=C=1NcRfc(τ)SMPc(xT)dt

[0015] Among them, f:R→RNc is the input function, fc and SMPc represent the cth element of the input, the convolution operator generates a function to calculate the filter response by summing over the entire Nc channels, one SMP represents one convolution operator, and multiple SMPs are used to implement a convolution layer to generate multiple output channels.

[0016] Furthermore, the CGLU module adds a minimal form of 3×3 depth convolution before the activation function of the GLU gating branch to make its structure conform to the gated channel attention, converting it into a gated channel attention mechanism based on nearest neighbor features.

[0017] Furthermore, the SPDConv module that the P2 detection layer passes through includes a space-to-depth layer SPD and a non-stride convolution layer Conv. The space-to-depth layer SPD downsamples the feature map and adds a non-stride convolution layer Conv thereafter.

[0018] Furthermore, the CSPOMniKernel module is improved by using the CSP idea and the OmniKernel based on AAAI2024 to obtain the CSPOMniKernel module, as follows:

[0019] The OmniKernel is divided into three branches: local branch, large branch and global branch. The results of the three branches are fused by addition and integrated by another 1×1 convolution.

[0020] Large branch: Apply deep convolution with kernel size K×K to pursue a large receptive field, influenced by strip-based self-attention, and also use 1×K and K×1 deep convolutions to obtain contextual information of strip shape;

[0021] Global branch: In the inference stage, the global modeling function is added in the global branch through dual-domain processing, where the global branch includes the dual-domain channel attention module DCAM and the frequency-based spatial attention module FSAM;

[0022] Local branch: local signal integration is performed by using 1×1 depthwise convolutional layers.

[0023] Beneficial effects:

[0024] 1. When ordinary convolution in the original YOLOv8 model faces small target detection objects such as traffic signs, the diversity and complexity of its features will cause the model to be unable to capture effective features. The present invention improves the YOLOv8 model to obtain the SSPDO-YOLO model, first using SMPConv, which has the latest adaptive point movement mechanism, so as to better capture local features and improve the flexibility and accuracy of feature extraction. Then, CGLU (Convolutional GLU) is added after SMPConv, which combines convolution and gating mechanisms, and can selectively pass information channels, thereby improving the effectiveness and flexibility of feature extraction. Compared with the previous MLP-based continuous convolution (using one neural network for one convolution layer), SMPConv has independent parameters for each convolution filter in a layer. It provides more freedom for each filter and makes the learned filter have stronger descriptive power. By adding the minimum form of 3×3 deep convolution before the activation function of the GLU gated branch, its structure can conform to the design concept of gated channel attention and transform it into a gated channel attention mechanism based on nearest neighbor features.

[0025] 2. For small target detection tasks such as traffic signs, the P3, P4 and P5 detection layers in the original YOLOv8 model do not seem to be very useful. The traditional improvement method is to add a small target detection layer, but this will cause a significant increase in the amount of model calculation after the addition and it is more time-consuming. The present invention is based on PAFPN and uses the P2 feature layer to obtain features rich in small target information through SPDConv and gives them to P3 for fusion. Then, the CSP idea and the OmniKernel based on AAAI2024 are used to improve the CSPOMniKernel for feature integration, effectively learning feature representations from global to local, and ultimately improving the detection performance of small targets.

[0026] 3. The SPD layer of SPDConv downsamples the feature map, retaining all the information in the channel dimension, so there is no information loss. We are inspired by the image conversion technique, which rescales the original image before feeding it to the neural network, but we generalize it to the downsampling of feature maps within and throughout the network. In addition, we add a convolution operation after each SPD, using scientific parameters to reduce the number of channels. The method we propose is both general and unified, that is, SPD can be applied to most CNN architectures and replaces stride convolution and pooling in the same way. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is the SSPDO-YOLO model diagram of the present invention;

[0028] Figure 2 This is a specific improvement diagram of the target detection layer of the present invention;

[0029] Figure 3 This is a C2f-SMPCGLU module diagram of the present invention;

[0030] Figure 4 It is the CGLU module diagram of the present invention;

[0031] Figure 5 It is the branch diagram of the OmniKernel module of the present invention;

[0032] Figure 6 This is the CSPOMniKernel module diagram of the present invention. DETAILED DESCRIPTION

[0033] In order to overcome the defects of the prior art, this paper provides a small target detection method for traffic signs based on improved SSPDO-YOLO. The technical solutions adopted by the present invention are: a small target detection method for traffic signs based on improved SSPDO-YOLO, comprising:

[0034] S001 collects various traffic sign images using vehicle-mounted monitoring and preprocesses the images to obtain a data set. Various traffic sign images are collected using vehicle-mounted monitoring, these images are classified and processed, and then the images are preprocessed using the Mosaic data enhancement method, and finally annotated to obtain a data set.

[0035] S002 improves its network framework based on the YOLOv8 algorithm and obtains the improved SSPDO-YOLO model. Figure 1 , the SSPDO-YOLO model is as follows:

[0036] S0021 uses the new module C2f-SMPCGLU to replace the C2f module of the original model.

[0037] When ordinary convolution in the original YOLOv8 model faces small target detection objects such as traffic signs, the diversity and complexity of its features will cause the model to be unable to capture effective features. Therefore, we first adopted SMPConv, which has the latest adaptive point movement mechanism, so as to better capture local features and improve the flexibility and accuracy of feature extraction. Then, CGLU (Convolutional GLU) was added after SMPConv, which combines convolution and gating mechanisms, and can selectively pass information channels, improving the effectiveness and flexibility of feature extraction.

[0038] C2f-SMPCGLU module such as Figure 2 As shown, SMPConv is used to implement the continuous convolution operator. In one dimension, the continuous convolution can be expressed as:

[0039] (f*SMP)0=c=1NcRfC(T)SMPC(x-τ)dt

[0040] f:R→RNc is the input function, fc and SMPc represent the cth element of the input. The convolution operator generates a function that calculates the filter response by summing over the entire Nc channels. One SMP represents one convolution operator, and multiple SMPs are used to implement a convolution layer to generate multiple output channels. Compared with the previous MLP-based continuous convolution (using one neural network for one convolution layer), SMPConv has independent parameters for each convolution filter in a layer. It provides more freedom for each filter and makes the learned filters more descriptive.

[0041] The specific structure of the CGLU module is as follows Figure 3 By adding a minimal 3×3 depth convolution before the activation function of the GLU gating branch, its structure can be made consistent with the design concept of gated channel attention and transformed into a gated channel attention mechanism based on nearest neighbor features.

[0042] S0022 adds a P2 detection layer, which first passes through SPDConv and then is fused with P3; the CSPOMniKernel module is used for feature integration.

[0043] For small target detection tasks such as traffic signs, the P3, P4, and P5 detection layers in the original YOLOv8 model do not seem to be very useful. The traditional improvement method is to add a small target detection layer, but this will greatly increase the amount of model calculation after the addition and be more time-consuming. We make improvements based on PAFPN, use the P2 feature layer through SPDConv to obtain features rich in small target information and give them to P3 for fusion, and then use the CSP idea and the OmniKernel based on AAAI2024 to improve the CSPOMniKernel for feature integration, effectively learning feature representations from global to local, and ultimately improving the detection performance of small targets.

[0044] SPDConv is a spatial-to-depth layer followed by only one non-strided convolutional layer. The SPD layer downsamples the feature map X while retaining all the information in the channel dimension, so there is no information loss. We are inspired by image conversion techniques that rescale the original image before feeding it to the neural network, but we generalize it to downsample feature maps within and across networks. In addition, we add a convolution operation after each SPD to reduce the number of channels using scientific parameters. The method we propose is both general and unified, that is, SPD can be applied to most CNN architectures and replaces strided convolution and pooling in the same way.

[0045] OmniKernel is divided into three branches (local branch, large branch and global branch), and the results of the three branches are then fused by addition and integrated by another 1×1 convolution. Large branch: We apply depthwise convolution with kernel size K×K to pursue a large receptive field. Inspired by strip-based self-attention, we also use 1×K and K×1 depthwise convolutions to obtain contextual information of strip shapes. Global branch: During the inference stage, the input degraded image is much larger than the training image (256). Therefore, the 63×63 kernel cannot cover the global image. To alleviate this problem, we add global modeling capabilities in the global branch through dual-domain processing. Among them, the global branch includes a dual-domain channel attention module (DCAM) and a frequency-based spatial attention module (FSAM). Local branch: Inspired by the fact that local information plays an important role in image restoration, in addition to capturing a large receptive field, a very simple but effective local branch is used to integrate local signals by using 1×1 depthwise convolution layers.

[0046] S003 uses the SSPDO-YOLO model to perform traffic sign target detection on the data set processed by step S001.

[0047] In order to verify the effectiveness of the SSPDO-YOLO model, the detection structures of various mainstream target detection networks were reproduced on the enhanced TT100K dataset and compared with YOLOv8s and our model. The comparison results are shown in Table 1:

[0048] Table 1 Comparison with other models on the TT100K dataset

[0049]

[0050] It can be seen from the data in Table 1 that the computational complexity of the model of the present invention is 29.7. Compared with other models, the present invention greatly reduces the computational complexity of the model while ensuring the accuracy is comparable to that of the existing models.

[0051] Then, the SSPDO-YOLO model was used to conduct experiments on another open source traffic sign dataset, CCTSDB 2021. Compared with different mainstream algorithms on this dataset, the results are shown in Table 2 below:

[0052] Table 2 Comparison with other models on the CCTSDB 2021 dataset

[0053]

[0054] It can be seen from the experiments on the above two open source traffic sign datasets that the present invention can ensure that the accuracy is comparable to that of the existing algorithms while making the model simpler and significantly reducing the amount of calculation.

[0055] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent transformation or modification made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A traffic sign small target detection method based on improved SSPDO-YOLO, characterized in that: The steps include: Step 1: Collect various traffic sign images using vehicle-mounted monitoring and pre-process the images to obtain a data set; Step 2: Improve the network framework based on the YOLOv8 model to obtain an improved SSPDO-YOLO model; the specific improvements of the SSPDO-YOLO model are as follows: 1) Use the C2f-SMPCGLU module to replace the C2f module of the original YOLOv8 model; 2) Add the P2 detection layer. The P2 detection layer first passes through the SPDConv module, and then is fused with the P3 detection layer and the CSPOMniKernel module is used for feature integration. Step 3: Use the SSPDO-YOLO model to perform traffic sign target detection on the processed dataset.

2. The traffic sign small target detection method based on improved SSPDO-YOLO according to claim 1, characterized in that: The C2f-SMPCGLU module first adopts the SMPConv module, and then adds a CGLU module after the SMPConv module, combining convolution and gating mechanisms.

3. The traffic sign small target detection method based on improved SSPDO-YOLO according to claim 2 is characterized in that: The SMPConv module implements the continuous convolution operator. In one dimension, the continuous convolution is expressed as: (f*SMP)(x)=c=1NcRfc(τ)SMPc(x-τ)dτ Among them, f:R→RNc is the input function, fc and SMPc represent the cth element of the input, the convolution operator generates a function to calculate the filter response by summing over the entire Nc channels, one SMP represents one convolution operator, and multiple SMPs are used to implement a convolution layer to generate multiple output channels.

4. The traffic sign small target detection method based on improved SSPDO-YOLO according to claim 2 is characterized in that: The CGLU module adds a minimal form of 3×3 depth convolution before the activation function of the GLU gating branch to make its structure conform to the gated channel attention, converting it into a gated channel attention mechanism based on nearest neighbor features.

5. The traffic sign small target detection method based on improved SSPDO-YOLO according to claim 1, characterized in that: The SPDConv module that the P2 detection layer passes through includes a space-to-depth layer SPD and a non-stride convolution layer Conv. The space-to-depth layer SPD downsamples the feature map and adds a non-stride convolution layer Conv thereafter.

6. The traffic sign small target detection method based on improved SSPDO-YOLO according to claim 1, characterized in that: The CSPOMniKernel module is improved by using the CSP idea and the OmniKernel based on AAAI2024, as follows: The OmniKernel is divided into three branches: local branch, large branch and global branch. The results of the three branches are fused by addition and integrated by another 1×1 convolution. Large branch: Apply deep convolution with kernel size K×K to pursue a large receptive field, influenced by strip-based self-attention, and also use 1×K and K×1 deep convolutions to obtain contextual information of strip shape; Global branch: In the inference stage, the global modeling function is added in the global branch through dual-domain processing, where the global branch includes the dual-domain channel attention module DCAM and the frequency-based spatial attention module FSAM; Local branch: local signal integration is performed by using 1×1 depthwise convolutional layers.

Citation Information

Patent Citations

  • Traffic sign board detection method based on improved YOLOv8s

    CN117392640A

Cited By

  • Power transmission line multi-defect detection method and system based on improved YOLOv8s

    CN119559479A

  • Aerial image small target detection method and system based on improved YOLOv8 algorithm, medium and processor

    CN119625269A

  • Infrared small target detection method and system based on YOLOv8-SO algorithm

    CN120747476A