RCSOSA-AFGC traffic sign detection method and system fused with lightweight detection head

Abstract

The invention discloses an RCSOSA-AFGC traffic sign detection method and system fused with a lightweight detection head. The traffic sign detection method comprises the following steps: acquiring a traffic sign data set; constructing a traffic sign detection model based on YOLOv10: replacing a C2f module in a backbone network with a C2fQBOEMA module, and adding an AFGC attention mechanism module behind a PSA self-attention module; a C2f module in the neck network is replaced by an RCSOSA module; an original detection head v10Detect is replaced by an MDBF detection head; training a traffic sign detection model through the data set; and carrying out traffic sign detection by using the trained traffic sign detection model. According to the invention, on the premise of ensuring the detection precision, the light weight of the model is realized, so that deployment on limited equipment is facilitated.

Description

Technical Field

[0001] The present invention relates to traffic sign image detection, and in particular to a RCSOSA-AFGC traffic sign detection method and system integrating a lightweight detection head. Background Art

[0002] The main task of traffic sign detection is to quickly and accurately identify and locate traffic signs in various complex traffic environments. In recent years, with the rapid development of deep learning, object detection methods based on convolutional neural networks (CNNs) have made significant progress, especially real-time object detection algorithms represented by YOLO (You Only Look Once), which have performed well in traffic sign detection.

[0003] In order to improve the accuracy of traffic sign detection, the prior art usually increases the model depth and improves the feature expression ability by strengthening multi-scale feature fusion and other means, which may take up a lot of memory. In some cases where the device is limited, memory consumption may become a limiting factor. When dealing with special weather and long-distance target detection, adding detection heads to improve the accuracy of small target detection will increase the total number of model parameters and memory usage, which is not conducive to low-end device applications. For example, CN111274970A discloses a traffic sign detection method based on an improved YOLOv3 algorithm, which replaces the DarkNet-53 in the original YOLOv3 algorithm by designing a feature extraction network that maintains high resolution, thereby improving the detection accuracy of small-sized traffic sign targets. The model size issue is not taken into account, resulting in deployment difficulties.

[0004] In summary, in autonomous driving vehicles or embedded devices, computing resources and storage space are limited. Deep learning models such as YOLO usually contain a large number of parameters and complex calculations. Direct application may cause problems such as computing delays, device heating, and increased power consumption. Summary of the invention

[0005] Purpose of the invention: The purpose of the present invention is to provide a RCSOSA-AFGC traffic sign detection method and system that integrates a lightweight detection head and achieves model lightweighting while ensuring detection accuracy.

[0006] Technical solution: A RCSOSA-AFGC traffic sign detection method integrating a lightweight detection head of the present invention comprises:

[0007] (1) Obtain traffic sign dataset;

[0008] (2) Build a traffic sign detection model based on YOLOv10, including:

[0009] In the YOLOv10 backbone network, the C2f module is replaced by the C2f_QB_OEMA module, and the AFGC attention mechanism module is added after the PSA self-attention module; the C2f_QB_OEMA module is formed by replacing the Bottleneck in the C2f module with QB_OEMA, which is a module obtained by adding the OptimizedEMA attention mechanism to the Quicker_Block;

[0010] In the YOLOv10 neck network, the C2f module is replaced by the RCSOSA module;

[0011] For the detection head of YOLOv10, replace the original detection head v10Detect with the MDBF detection head;

[0012] (3) Train the traffic sign detection model using the dataset;

[0013] (4) Use the trained traffic sign detection model to perform traffic sign detection.

[0014] Furthermore, in step (1), the CCTSDB traffic sign dataset is used.

[0015] Further, in step (2), the backbone network includes a first Conv module, a second Conv module, a third Conv module, a first C2f_QB_OEMA module, a second C2f_QB_OEMA module, a third C2f_QB_OEMA module, a fourth C2f_QB_OEMA module, an SPPF module, a PSA self-attention module and an AFGC attention mechanism module; the neck network includes a fourth Conv module, a C2fCIB module, a first RCSOSA module, a second RCSOSA module and a third RCSOSA module; the detection head part includes a first MDBF detection head, a second MDBF detection head, a third MDBF detection head, a fourth MDBF detection head, a fifth MDBF detection head and a sixth MDBF detection head;

[0016] The original image is used as the input of the first Conv module, the output of the first Conv module is used as the input of the second Conv module, the output of the second Conv module is used as the input of the first C2f_QB_OEMA module, the output of the first C2f_QB_OEMA module is used as the input of the third Conv module, the output of the third Conv module is used as the input of the second C2f_QB_OEMA module, one output of the second C2f_QB_OEMA module is down-sampled by SCDown and used as the input of the third C2f_QB_OEMA module, one output of the third C2f_QB_OEMA module is down-sampled by SCDown and used as the input of the fourth C2f_QB_OEMA module, the output of the fourth C2f_QB_OEMA module is used as the input of the SPPF module, the output of the SPPF module is used as the input of the PSA self-attention module, and one output of the PSA self-attention module is used as the input of the AFGC attention mechanism module;

[0017] The output of the AFGC attention mechanism module is upsampled by Upsample, and then concatenated with the other output of the third C2f_QB_OEMA module, and then used as the input of the first RCSOSA module; one output of the first RCSOSA module is upsampled by Upsample, and then concatenated with the other output of the second C2f_QB_OEMA module, and then used as the input of the second RCSOSA module; one output of the second RCSOSA module is used as the input of the first MDBF detection head and the second MDBF detection head respectively; another output of the second RCSOSA module is used as the input of the fourth Conv module, and the output of the fourth Conv module is concatenated with the other output of the first RCSOSA module, and then used as the input of the third RCSOSA module; one output of the third RCSOSA module is used as the input of the third MDBF detection head and the fourth MDBF detection head respectively; another output of the third RCSOSA module is downsampled by SCDown, and then concatenated with the other output of the PSA self-attention module, and then used as the input of the C2fCIB module; the output of the C2fCIB module is used as the input of the fifth MDBF detection head and the sixth MDBF detection head respectively.

[0018] Furthermore, in the QB_OEMA module, the input data is first input into the channel adjustment layer. When the number of input channels is inconsistent with the target number of channels, a 1x1 convolution is used to transform the number of channels. When they are consistent, a residual connection operation is performed. Then the data enters the spatial mixing layer to mix the feature maps in spatial dimensions. Then, the data enters the MLP module and the OptimizedEMA attention mechanism module in turn. The MLP module first uses depthwise separable convolution for feature extraction, and then uses 1x1 convolution to adjust the number of channels. Then, the data enters the SE module to recalibrate the channel features through compression and excitation. Finally, the residual connection is used to skip the connection to transfer errors and avoid gradient explosion.

[0019] Furthermore, in the OptimizedEMA attention mechanism module, the input feature map is first grouped according to the channels dimension to reduce computational complexity and allow different groups to focus on different feature patterns; then the spatial dependencies in the horizontal and vertical directions are captured respectively, and the information in the two directions is fused using 1x1 convolution; then further feature enhancement is performed, the sigmoid function is used to generate attention maps, GroupNorm ensures the stability of feature distribution, and then 3x3 convolution is used to capture local context information; finally, cross-attention is performed to calculate the correlation between features, adaptively adjust feature weights, and enhance the representation ability of key areas.

[0020] Furthermore, the AFGC attention mechanism mainly implements the following steps:

[0021] 1) The global average pooling layer compresses the spatial dimension and reduces the size of the input feature map to 1x1; given a feature map F∈R C×H×W , where C, H, and W represent the number, length, and width of channels respectively, and a channel descriptor U∈R is generated by global average pooling. C , the nth channel element of U is recorded as:

[0022]

[0023] In formula (1), F n (i, j) is the value of the nth channel feature map at position (i, j), GAP(X) is the global average pooling function, through which the shape of the feature map F can be transformed from C×H×W to C×1×1;

[0024] 2) One-dimensional convolution layer with adaptive kernel size, where both input and output channels are 1, and its function is to adaptively calculate the convolution kernel size based on the number of channels;

[0025] 3) Define a conv2d full convolution layer with input and output channels of channel and convolution kernel size of 1, which is used to complete the feature mapping between channels; secondly, define a Sigmoid activation function for subsequent feature weight normalization;

[0026] 4) Then comes the forward propagation part:

[0027] First, the input is globally averaged pooled to obtain a feature map of size (1, channel, 1); then the features are processed using a dual-branch structure, with the first branch using a one-dimensional convolution and the second branch using a fully connected layer; after the two branches are processed, the attention weights are obtained by interactive fusion through the calculated matrix multiplication. In order to obtain local channel information while ensuring a small number of model parameters, the interaction matrix B for local channel interaction is used, and B is set as follows: [b1, b2, b3, ..., b k ]:

[0028]

[0029] Among them, U represents the channel descriptor, U 1c represents local information, k represents the number of adjacent channel numbers;

[0030] In order to obtain global channel information and enhance the representation ability of global information, the diagonal matrix D is used to capture the dependencies between channels as global information. Let D = [d1, d2, d3, ..., d c ]:

[0031]

[0032] Among them, U gc is the global information, c is the number of channels;

[0033] In order to promote the effective interaction of global and local information, the global information obtained through the diagonal matrix is ​​combined with the local information obtained through the interaction matrix, and the correlation between the two at different granularities is captured through mutual correlation operations, which is formalized as follows:

[0034]

[0035] In formula (4), M represents the correlation matrix;

[0036] Then, in order to accurately assign feature weights and reduce computational complexity, an adaptive fusion strategy is adopted; row and column information are extracted from the correlation matrix and its transpose as weight vectors of global and local information; dynamic fusion is achieved through a learnable factor, as shown in formulas (5) to (7);

[0037]

[0038]

[0039]

[0040] in, is the fused global channel weight, is the local channel weight after fusion, C is the number of channels; σ(X) represents the sigmoid activation function; θ is an input variable, usually a learnable parameter or an intermediate variable, which is used to control the fusion ratio of global and local information after the sigmoid activation function;

[0041] Then, the sum is taken in the channel dimension to obtain the final attention weights out1 and out2, and they are mapped to the (0,1) interval through the sigmoid function; then, the two attention weights are combined using a mixing layer, and the mixed attention features are further convolved and activated using the sigmoid function in one dimension; finally, the element-by-element multiplication result of the input and the calculated attention feature map is returned to enhance the channel features; finally, the obtained weight is multiplied by the input feature map as follows:

[0042]

[0043] Among them, F is the input feature map, F * is the final output feature map.

[0044] Furthermore, the RCSOSA module mainly includes a scaling factor e, two 1x1 convolutional layers and a sequence module composed of n SR modules stacked together; the input feature first undergoes a 1x1 convolution through conv1 to adjust the number of channels and generate an intermediate feature map; then the intermediate feature map passes through the SR module, undergoes n SR operations, performs channel division, depth convolution and channel shuffling; finally, after a 1x1 convolution through conv2, the number of channels of the output feature map is adjusted to c2.

[0045] Furthermore, the SR module includes a depth-wise separable convolution Conv2d, a batch normalization layer BatchNorm2d, and an activation function SiLU; the input feature map x is first divided into two parts along the channel dimension: x1 and x2, and x2 is processed by convolution, batch normalization, and activation function to obtain a feature map; then x1 and the processed x2 are concatenated on the channel dimension to form the output; the output features undergo a channel shuffle operation.

[0046] Furthermore, the input end of the MDBF detection head receives three feature maps P3, P4, and P5. P3 is responsible for detecting small targets, P4 is responsible for detecting medium targets, and P5 is responsible for detecting large targets. Each input feature is first processed by 1×1 ConvGN dimensionality reduction, and then feature extraction is performed through two shared 3×3 ConvGNs. Finally, it is divided into two branches, one2one and one2many. Each branch has two tasks: regression and classification.

[0047] Based on the same inventive concept, the present invention provides an RCSOSA-AFGC traffic sign detection system integrating a lightweight detection head, including a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the above-mentioned traffic sign detection method.

[0048] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0049] (1) In view of the limited computing power of low-end devices, the present invention combines OptimizedEMA and Quicker_Block to form Quicker_Block_OptimizedEMA, and replaces the c2f module in the YOLOv10 backbone network with c2f_Quicker_Block_OptimizedEMA as the main feature extraction method, making the YOLOv10 backbone network more lightweight while maintaining the accuracy basically unchanged, thereby achieving the technical effect of speeding up feature extraction while achieving model lightweight.

[0050] (2) The present invention adds an AFGC (Adaptive fine-grained channel attention) attention mechanism module after the PSA self-attention module of the YOLOv10 backbone network, uses the correlation matrix to capture the correlation between global and local information, promotes the interaction between them at different granularities, and achieves more effective feature weight allocation, thereby solving the problem of feature loss that may be caused during the lightweight process of the backbone network, and improving the accuracy without increasing the model calculation amount; at the same time, this technical means can further enhance the feature extraction capability, reduce the interference of blurred background on traffic sign detection, and suppress redundant and interfering information; thereby achieving the technical effect of improving the accuracy of model detection and classification, increasing the model's attention to important features, and reducing the complexity of the model.

[0051] (3) The present invention replaces the C2f module in the neck network with the RCSOSA module as the main feature fusion and processing means of the neck network, solves the problem that the information extracted by the neck network is not comprehensive and rich enough, and replaces the complex convolution with the standard depth-separable convolution; these technical means can ensure the reuse of features and enhance the information flow between different channels between adjacent layer features, so as to enable high-precision and fast inference in traffic sign detection. In this way, high accuracy is achieved, and the inference throughput is improved on the GPU, and the computing time is saved on the CPU; redundant calculations and memory access are reduced, so that spatial features can be extracted more effectively.

[0052] (4) The present invention proposes an MDBF (Multi-DualBranch FusionDetect) detection head to replace the original v10Detect. The dual-branch structure of the MDBF detection head, and the use of shared convolution and Group Norm normalization functions, solve the problems of low model performance, huge model parameters, and low training efficiency. It can then be deployed on restricted devices and solve problems such as computing delays, device heating, and increased power consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flow chart of a RCSOSA-AFGC traffic sign detection method integrating a lightweight detection head provided by an embodiment of the present invention;

[0054] Figure 2 is a schematic diagram of a traffic sign detection model in an embodiment of the present invention;

[0055] Figure 3 is a schematic diagram of the Quicker_Block_OptimizedEMA module in an embodiment of the present invention;

[0056] Figure 4 is a schematic diagram of the OptimizedEMA attention mechanism module in an embodiment of the present invention;

[0057] Figure 5 is a schematic diagram of a RCSOSA module in an embodiment of the present invention;

[0058] Figure 6 : is a schematic diagram of the AFGC attention mechanism module in an embodiment of the present invention;

[0059] Figure 7 is a flowchart of processing a single feature map of an MDBF detection head in an embodiment of the present invention;

[0060] Figure 8 is an overall structural diagram of the MDBF detection head in an embodiment of the present invention;

[0061] Fig. 9 is a curve diagram of the traffic sign detection model training process in an embodiment of the present invention;

[0062] Fig.10 It is a diagram showing the effect of detection on the CCTSDB dataset in an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The present invention will be further described below in conjunction with the accompanying drawings.

[0064] like Figure 1 As shown, an embodiment of the present invention provides a RCSOSA-AFGC traffic sign detection method integrating a lightweight detection head, comprising the following steps:

[0065] (1) Obtain traffic sign dataset;

[0066] The CCTSDB traffic sign dataset is used, including speed limit signs, prohibition signs, warning signs, instruction signs, etc., which contains a total of 16,356 training sets and 1,500 test set data.

[0067] (2) Build a traffic sign detection model based on YOLOv10;

[0068] In the YOLOv10 backbone network Backbone, the C2f module is replaced by the C2f_QB_OEMA module, and the AFGC attention mechanism module is added after the PSA self-attention module; the C2f_QB_OEMA module is formed by replacing the Bottleneck in the C2f module with QB_OEMA (Quicker_Block_OptimizedEMA, referred to as QB_OEMA), which is a module obtained by adding the OptimizedEMA attention mechanism to Quicker_Block; the C2f_QB_OEMA module is responsible for fusing the shallow feature map with the deep feature map to extract rich semantic information.

[0069] In the YOLOv10 neck network Neck (feature fusion network), the C2f module is replaced by the RCSOSA module.

[0070] The detection head part of YOLOv10, Detect, replaces the original detection head v10Detect with the MDBF (Multi-DualBranch FusionDetect) detection head.

[0071] The specific structure of the traffic sign detection model is as follows:

[0072] like Figure 2As shown, the backbone network includes a first Conv module, a second Conv module, a third Conv module, a first C2f_QB_OEMA module, a second C2f_QB_OEMA module, a third C2f_QB_OEMA module, a fourth C2f_QB_OEMA module, an SPPF module, a PSA self-attention module and an AFGC attention mechanism module; the neck network includes a fourth Conv module, a C2fCIB module, a first RCSOSA module, a second RCSOSA module and a third RCSOSA module; the detection head part includes a first MDBF detection head, a second MDBF detection head, a third MDBF detection head, a fourth MDBF detection head, a fifth MDBF detection head and a sixth MDBF detection head.

[0073] The original image is used as the input of the first Conv module, the output of the first Conv module is used as the input of the second Conv module, the output of the second Conv module is used as the input of the first C2f_QB_OEMA module, the output of the first C2f_QB_OEMA module is used as the input of the third Conv module, the output of the third Conv module is used as the input of the second C2f_QB_OEMA module, one output of the second C2f_QB_OEMA module is down-sampled by SCDown and used as the input of the third C2f_QB_OEMA module, one output of the third C2f_QB_OEMA module is down-sampled by SCDown and used as the input of the fourth C2f_QB_OEMA module, the output of the fourth C2f_QB_OEMA module is used as the input of the SPPF module, the output of the SPPF module is used as the input of the PSA self-attention module, and one output of the PSA self-attention module is used as the input of the AFGC attention mechanism module;

[0074] The output of the AFGC attention mechanism module is upsampled by Upsample, and then concatenated with the other output of the third C2f_QB_OEMA module, and then used as the input of the first RCSOSA module; one output of the first RCSOSA module is upsampled by Upsample, and then concatenated with the other output of the second C2f_QB_OEMA module, and then used as the input of the second RCSOSA module; one output of the second RCSOSA module is used as the input of the first MDBF detection head and the second MDBF detection head respectively; another output of the second RCSOSA module is used as the input of the fourth Conv module, and the output of the fourth Conv module is concatenated with the other output of the first RCSOSA module, and then used as the input of the third RCSOSA module; one output of the third RCSOSA module is used as the input of the third MDBF detection head and the fourth MDBF detection head respectively; another output of the third RCSOSA module is downsampled by SCDown, and then concatenated with the other output of the PSA self-attention module, and then used as the input of the C2fCIB module; the output of the C2fCIB module is used as the input of the fifth MDBF detection head and the sixth MDBF detection head respectively.

[0075] 1. QB_OEMA module

[0076] Combination Figure 3 ,The Quicker_Block_OptimizedEMA module is mainly composed of a channel adjustment layer, a spatial mixing layer, an MLP module, an OptimizedEMA attention mechanism module and a SE module.

[0077] The input data X first enters the channel adjustment layer. Its main function is to use 1x1 convolution to transform the number of channels when the number of input channels is inconsistent with the target channel number. When they are consistent, a residual connection operation is performed; then it enters the spatial mixing layer, which mixes the spatial dimensions of the feature map; then it enters the MLP module and the OptimizedEMA attention mechanism module in turn. The function of the MLP module is to first use depthwise separable convolution for feature extraction, and then use 1x1 convolution to adjust the number of channels. Using depthwise separable convolution can reduce the computational complexity. The function of the OptimizedEMA attention mechanism module is combined below Figure 4 A detailed introduction is given; then the SE module is entered to recalibrate channel features through compression (compressing spatial information into channel descriptors) and excitation (learning the correlation between channels and recalibrating); finally, the residual connection is used to skip the connection and transfer errors to avoid gradient explosion. Replacing the standard convolution with a depth-wise separable convolution can reduce the number of parameters while maintaining expressiveness. The introduction of the SE module can enhance the focus on small distant signs or partially occluded signs.

[0078] The addition of the OptimizedEMA attention mechanism further enhances the feature extraction capability. Figure 4 As shown in the figure, the input feature map is first grouped by channels to reduce computational complexity and allow different groups to focus on different feature patterns, which is particularly suitable for targets with multi-scale features such as traffic signs. Then, the spatial dependencies in the horizontal and vertical directions are captured respectively, which is particularly conducive to detecting the shape features of traffic signs (circles, triangles, rectangles, etc.), and then the information in the two directions is fused using 1x1 convolution. Then, feature enhancement is performed, and attention maps are generated using the sigmoid function. GroupNorm (group normalization) ensures the stability of feature distribution, and then 3x3 convolution is used to capture local context information. Finally, cross attention is performed to calculate the correlation between features, adaptively adjust feature weights, and enhance the representation ability of key areas. EMA can better handle multi-scale features through grouping and multi-dimensional attention, and improve the detection ability of small targets. The bidirectional pooling operation helps capture the geometric features of traffic signs, and the spatial attention mechanism strengthens the perception of shape boundaries and improves the robustness of partially occluded signs. Traffic sign detection often faces complex backgrounds, and the attention mechanism can suppress background interference and enhance the focus on key areas. The grouping operation reduces the amount of calculation, which is suitable for real-time detection needs and can run efficiently on restricted devices.

[0079] 2. AFGC Attention Mechanism Module

[0080] In order to compensate for the feature loss problem that may be caused by the lightweight process of the backbone network and control the overall accuracy from decreasing, the AFGC (Adaptive fine-grained channel attention) attention mechanism module is added after the PSA self-attention module of the YOLOv10 backbone network. AFGC uses the correlation matrix to capture the correlation between global and local information, promotes the interaction between them at different granularities, and achieves more effective feature weight allocation.

[0081] The AFGC attention mechanism mainly implements the following steps:

[0082] 1) The first is the global average pooling layer, which compresses the spatial dimension and reduces the size of the input feature map to 1x1. The use of lightweight one-dimensional convolution and global average pooling improves performance while maintaining low computational overhead. Given a feature map F∈R C×H×W , where C, H, and W represent the number, length, and width of channels respectively, and a channel descriptor U∈R is generated by global average pooling. C , the nth channel element of U is recorded as:

[0083]

[0084] In formula (1), F n (i,j) is the value of the nth channel feature map at position (i,j), GAP(X) is the global average pooling function, through which the shape of the feature map F can be transformed from C×H×W to C×1×1.

[0085] 2) The second is a one-dimensional convolution layer with adaptive kernel size. The number of input and output channels is 1. Its function is to adaptively calculate the convolution kernel size based on the number of channels. Through the adaptive channel attention mechanism, the detailed features of traffic signs can be better captured, and traffic signs of different sizes and shapes can be fully paid attention to.

[0086] 3) The third is to define a conv2d full convolution layer, the number of input and output channels are both channel, the convolution kernel size is 1 (equivalent to point-by-point convolution), the role is to complete the feature mapping between channels. Secondly, define a Sigmoid activation function for subsequent feature weight normalization.

[0087] 4) Then comes the forward propagation part, such as Figure 6 As shown:

[0088] First, the input is globally averaged pooled to obtain a feature map of size (1, channel, 1); then the features are processed using a dual-branch structure, the first branch is processed using a one-dimensional convolution, and the second branch is processed using a fully connected layer; after the two branches are processed, the attention weights are obtained by interactive fusion through the calculated matrix multiplication. In order to obtain local channel information while ensuring a small number of model parameters, the interaction matrix B for local channel interaction is used, and B is set as follows: [b1, b2, b3, ..., b k ]:

[0089]

[0090] Among them, U represents the channel descriptor, U 1c represents local information, and k represents the number of adjacent channel numbers.

[0091] In order to obtain global channel information and enhance the representation ability of global information, the diagonal matrix D is used to capture the dependencies between channels as global information. Let D = [d1, d2, d3, ..., d c ]:

[0092]

[0093] Among them, U gc is the global information, and c is the number of channels.

[0094] In order to promote the effective interaction of global and local information, the global information obtained through the diagonal matrix is ​​combined with the local information obtained through the interaction matrix, and the correlation between the two at different granularities is captured through mutual correlation operations, which is formalized as follows:

[0095]

[0096] In formula (4), M represents the correlation matrix.

[0097] Then, in order to accurately assign feature weights and reduce computational complexity, an adaptive fusion strategy is adopted. The row and column information is extracted from the correlation matrix and its transpose as the weight vector of global and local information. Dynamic fusion is achieved through a learnable factor, as shown in formulas (5) to (7);

[0098]

[0099]

[0100]

[0101] in, is the fused global channel weight, is the local channel weight after fusion, C is the number of channels; σ(X) represents the sigmoid activation function; θ is an input variable, usually a learnable parameter or an intermediate variable, which is used to control the fusion ratio of global and local information after the sigmoid activation function.

[0102] Then, the sum is taken in the channel dimension to obtain the final attention weights out1 and out2, and they are mapped to the interval (0,1) through the sigmoid function; the dual-branch structure provides complementary feature representation, which can better handle traffic sign detection in complex scenes and has strong adaptability; then the two attention weights are combined using a mixed layer, and the mixed attention features are further convolved and activated by a one-dimensional convolution and a sigmoid function. This step is used to enhance the expressiveness of the features, effectively highlight the key features of traffic signs, and suppress the interference of background noise. Finally, the element-by-element multiplication result of the input and the calculated attention feature map is returned to achieve channel feature enhancement. Finally, the obtained weight is multiplied by the input feature map as follows:

[0103]

[0104] Among them, F is the input feature map, F * is the final output feature map.

[0105] RCSOSA Module

[0106] like Figure 5 As shown in Figure 2, the RCSOSA module mainly contains a scaling factor (e), two 1x1 convolutional layers, and a sequence module ( Figure 5 Only one SR module is shown in the figure). The input feature first passes through conv1 for a 1x1 convolution to adjust the number of channels and generate an intermediate feature map. The intermediate feature map then passes through the SR module and undergoes three SR operations to perform channel division, depthwise convolution, and channel shuffling. Finally, after a 1x1 convolution in conv2, the number of channels of the output feature map is adjusted to c2. The computational overhead and model capacity of the network can be flexibly adjusted through the parameters n (the number of times the SR module is stacked) and e (the scaling factor of the intermediate channel). This helps to optimize under different detection requirements, such as efficient traffic sign detection on mobile devices.

[0107] The SR module contains a depthwise separable convolution (Conv2d) and a batch normalization layer (BatchNorm2d), followed by an activation function (SiLU). The advantage of depthwise separable convolution is that it reduces computational complexity, retains information within the channel, and accelerates inference speed; the batch normalization layer is used to stabilize the training process and accelerate convergence; SiLU, as an activation function, can retain negative information and facilitate gradient flow. Specifically: the input feature map x is first divided into two parts along the channel dimension: x1 and x2. x2 is processed by convolution, batch normalization, and activation function to obtain a feature map. Then x1 and the processed x2 are concatenated in the channel dimension to form the output. The output features are subjected to the channel shuffle (channel_shuffle) operation. The random mixing operation of the channel helps to exchange information across channels and promote feature fusion. Through the channel_shuffle operation in the SR module, the module reorganizes features in the channel dimension, enhancing the ability to exchange information across channels. This is beneficial for fine-grained targets (long-distance micro traffic signs) and can improve the expressiveness of features.

[0108] 4. MDBF detection head

[0109] The following new features and improvements are provided:

[0110] The dual branch structure used (one-to-many and one-to-one detection):

[0111] One-to-Many (multi-detection): extract features through shared convolutional layers and perform multi-task output, so that common feature representations can be used to improve the performance of different detection tasks.

[0112] One-to-One: Provides more accurate output for specific detection tasks. Separate branches can optimize network structure and parameters for specific tasks.

[0113] The main purpose of using shared convolutional layers is to share convolution kernels (weights) between different layers or different network structures. It is used for multi-task learning or model compression. By sharing convolutional layers, multiple tasks can share the same feature extractor, thereby reducing model parameters and improving training efficiency.

[0114] Conv_GN applies group normalization after the convolution operation. Group normalization is a method of normalizing features, which is suitable for small batch data scenarios. It solves the problem of batch normalization being unstable in small batches and effectively reduces statistical noise in small batches.

[0115] The end-to-end mode integrates the processing flow and performs training and reasoning directly from input to output to improve efficiency.

[0116] By sharing convolutional layers, feature concatenation, and context information processing, the model not only enhances its ability to adapt to complex scenes, but also improves the overall detection accuracy and performance. By effectively combining features from different levels and scales, the model can better capture important information in the image and improve the performance of the detection task.

[0117] Figure 7 The processing flow chart for a single feature map contains the following parts:

[0118] Initialization part: Set the number of categories (nc) and the number of detection layers (nl), initialize multiple convolutional layers for feature processing, including one-to-one (one2one) and one-to-many (one2many) detection branches.

[0119] Forward propagation process: The shared convolution layer processes the input features, performs one-to-one and one-to-many predictions respectively, and the training and inference modes have different processing flows.

[0120] Inference processing: decoding bounding box predictions, processing classification scores, post-processing operations including TopK selection.

[0121] Figure 8 The structure diagram of the detection head is shown below:

[0122] The input end receives three feature maps (P3, P4, and P5). P3 is responsible for detecting small targets, P4 is responsible for detecting medium targets, and P5 is responsible for detecting large targets. These are different scale features from the backbone network. Each input feature is first processed by 1×1 ConvGN dimensionality reduction, and then feature extraction is performed through two shared 3×3 ConvGNs. Finally, it is divided into two branches, one2one and one2many. Each branch has two tasks: regression (Conv_Reg) and classification (Conv_Cls).

[0123] (3) Train the traffic sign detection model using the dataset;

[0124] Input the training set image data in the dataset into the network model for training, save the model parameters with the highest accuracy on the validation set during the training process, and name the file best.pt.

[0125] The experimental environment is configured to use the Ubuntu 20.04 operating system, the NVIDIA GeForce RTX 4060 graphics card, and the deep learning framework PyTorch. The specific configuration is shown in Table 1 below:

[0126] Table 1 Experimental environment configuration table

[0127]

[0128] The training set image data in the traffic sign dataset is input into the network model, and GPU training is used to set the improved network model parameters, with 300 training rounds, momentum of 0.937, initial learning rate of 0.01, minimum learning rate of 0.0001, weight decay coefficient of 0.00005, SGD optimizer, and batch-size of 16. During the model training process, the model parameters with the best accuracy will be saved and named best.pt.

[0129] Fig. 9The figure is the curve of the changes of various indicators during 300 rounds of model training. After the model training is completed, the Precision of various signs in the validation set is 89.3%, and the mAP is 80.5%. Compared with the original YOLOv10n, the algorithm of the present invention has improved the Precision of various signs by 2.0%, and the mAP has remained basically unchanged. The number of parameters has been reduced by 64.6% and the amount of calculation (GFLOPs) has been reduced by 46.1%. The model size has been reduced from 6MB to 1.9MB, a decrease of 68.3%. Among them, Precision represents the accuracy of the model. The larger the index, the better the model recognition effect; mAP represents the quality of the model in all categories. The larger the index, the better the model network performance. The algorithm index comparison table is shown in Table 2, which proves the feasibility of the present invention.

[0130] Table 2 Comparison of experimental results of different models

[0131]

[0132] (4) Use the trained traffic sign detection model to perform traffic sign detection.

[0133] The video signal is acquired through the camera, the video signal is intercepted into an image and image preprocessing is performed, the size of the input image is scaled and adjusted to 640×640 pixels, and the image is detected for speed limit signs, prohibition signs, warning signs, instruction signs, etc. using the trained traffic sign detection model, and the detection results are displayed on the monitor.

[0134] Fig.10 It is a detection effect diagram of the present invention.

[0135] The present invention has good feature extraction capability, small model size, small calculation amount, strong real-time performance, can meet the requirements of deploying traffic sign detection, and provides a technical basis for traffic sign detection.

[0136] An embodiment of the present invention also provides a RCSOSA-AFGC traffic sign detection system integrating a lightweight detection head. The system may include a processor and a memory. The memory stores computer instructions. The processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method described in the above embodiment and can achieve technical effects consistent with the above method.

[0137] The memory may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The device may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the memory may be used to read and write non-removable, non-volatile magnetic media (commonly referred to as a "hard drive"). A program / utility having a set (at least one) of program modules may be stored in, for example, the memory, such program modules including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include the implementation of a network environment. The program modules typically perform the functions and / or methods in the embodiments described herein.

[0138] The processor executes various functional applications and data processing by running the programs stored in the memory, such as implementing the method provided in the embodiment of the present invention.

Claims

1. A RCSOSA-AFGC traffic sign detection method integrating a lightweight detection head, characterized in that: include: (1) Obtain traffic sign dataset; (2) Build a traffic sign detection model based on YOLOv10, including: In the YOLOv10 backbone network, the C2f module is replaced by the C2f_QB_OEMA module, and the AFGC attention mechanism module is added after the PSA self-attention module; the C2f_QB_OEMA module is formed by replacing the Bottleneck in the C2f module with QB_OEMA, which is a module obtained by adding the OptimizedEMA attention mechanism to the Quicker_Block; In the YOLOv10 neck network, the C2f module is replaced by the RCSOSA module; For the detection head of YOLOv10, replace the original detection head v10Detect with the MDBF detection head; (3) Train the traffic sign detection model using the dataset; (4) Use the trained traffic sign detection model to perform traffic sign detection.

2. The RCSOSA-AFGC traffic sign detection method according to claim 1, characterized in that: In step (1), the CCTSDB traffic sign dataset is used.

3. The RCSOSA-AFGC traffic sign detection method according to claim 1, characterized in that: In step (2), the backbone network includes a first Conv module, a second Conv module, a third Conv module, a first C2f_QB_OEMA module, a second C2f_QB_OEMA module, a third C2f_QB_OEMA module, a fourth C2f_QB_OEMA module, an SPPF module, a PSA self-attention module and an AFGC attention mechanism module; the neck network includes a fourth Conv module, a C2fCIB module, a first RCSOSA module, a second RCSOSA module and a third RCSOSA module; the detection head part includes a first MDBF detection head, a second MDBF detection head, a third MDBF detection head, a fourth MDBF detection head, a fifth MDBF detection head and a sixth MDBF detection head; The original image is used as the input of the first Conv module, the output of the first Conv module is used as the input of the second Conv module, the output of the second Conv module is used as the input of the first C2f_QB_OEMA module, the output of the first C2f_QB_OEMA module is used as the input of the third Conv module, the output of the third Conv module is used as the input of the second C2f_QB_OEMA module, one output of the second C2f_QB_OEMA module is down-sampled by SCDown and used as the input of the third C2f_QB_OEMA module, one output of the third C2f_QB_OEMA module is down-sampled by SCDown and used as the input of the fourth C2f_QB_OEMA module, the output of the fourth C2f_QB_OEMA module is used as the input of the SPPF module, the output of the SPPF module is used as the input of the PSA self-attention module, and one output of the PSA self-attention module is used as the input of the AFGC attention mechanism module; The output of the AFGC attention mechanism module is upsampled by Upsample, and then concatenated with the other output of the third C2f_QB_OEMA module, and then used as the input of the first RCSOSA module; one output of the first RCSOSA module is upsampled by Upsample, and then concatenated with the other output of the second C2f_QB_OEMA module, and then used as the input of the second RCSOSA module; one output of the second RCSOSA module is used as the input of the first MDBF detection head and the second MDBF detection head respectively; another output of the second RCSOSA module is used as the input of the fourth Conv module, and the output of the fourth Conv module is concatenated with the other output of the first RCSOSA module, and then used as the input of the third RCSOSA module; one output of the third RCSOSA module is used as the input of the third MDBF detection head and the fourth MDBF detection head respectively; another output of the third RCSOSA module is downsampled by SCDown, and then concatenated with the other output of the PSA self-attention module, and then used as the input of the C2fCIB module; the output of the C2fCIB module is used as the input of the fifth MDBF detection head and the sixth MDBF detection head respectively.

4. The RCSOSA-AFGC traffic sign detection method according to claim 1, characterized in that: In the QB_OEMA module, the input data is first input into the channel adjustment layer. When the number of input channels is inconsistent with the target number of channels, a 1x1 convolution is used to transform the number of channels. When they are consistent, a residual connection operation is performed. Then the data enters the spatial mixing layer to mix the feature maps in spatial dimensions. Then, the data enters the MLP module and the OptimizedEMA attention mechanism module in turn. The MLP module first uses depthwise separable convolution for feature extraction, and then uses 1x1 convolution to adjust the number of channels. Then, the data enters the SE module to recalibrate the channel features through compression and excitation. Finally, the residual connection is used to skip the connection to transfer errors and avoid gradient explosion.

5. The RCSOSA-AFGC traffic sign detection method according to claim 4, characterized in that: In the OptimizedEMA attention mechanism module, the input feature map is first grouped according to the channels dimension to reduce computational complexity and allow different groups to focus on different feature patterns. Then, the spatial dependencies in the horizontal and vertical directions are captured respectively, and the information in the two directions is fused using 1x1 convolution. Then, further feature enhancement is performed, and the sigmoid function is used to generate attention maps. GroupNorm ensures the stability of feature distribution, and 3x3 convolution is used to capture local context information. Finally, cross-attention is performed to calculate the correlation between features, adaptively adjust feature weights, and enhance the representation ability of key areas.

6. The RCSOSA-AFGC traffic sign detection method according to claim 1, characterized in that: The AFGC attention mechanism mainly implements the following steps: 1) The global average pooling layer compresses the spatial dimension and reduces the size of the input feature map to 1x1; given a feature map F∈R C×H×W , where C, H, and W represent the number, length, and width of channels respectively, and a channel descriptor U∈R is generated by global average pooling. C , the nth channel element of U is recorded as: In formula (1), F n (i, j) is the value of the nth channel feature map at position (i, j), GAP(X) is the global average pooling function, through which the shape of the feature map F can be transformed from C×H×W to C×1×1; 2) One-dimensional convolution layer with adaptive kernel size, where both input and output channels are 1, and its function is to adaptively calculate the convolution kernel size based on the number of channels; 3) Define a conv2d full convolution layer with input and output channels of channel and convolution kernel size of 1, which is used to complete the feature mapping between channels; secondly, define a Sigmoid activation function for subsequent feature weight normalization; 4) Then comes the forward propagation part: First, the input is globally averaged pooled to obtain a feature map of size (1, channel, 1); then the features are processed using a dual-branch structure, with the first branch using a one-dimensional convolution and the second branch using a fully connected layer; after the two branches are processed, the attention weights are obtained by interactive fusion through the calculated matrix multiplication. In order to obtain local channel information while ensuring a small number of model parameters, the interaction matrix B for local channel interaction is used, and B is set as follows: [b1, b2, b3, ..., b k ]: Among them, U represents the channel descriptor, U 1c represents local information, k represents the number of adjacent channel numbers; In order to obtain global channel information and enhance the representation ability of global information, the diagonal matrix D is used to capture the dependencies between channels as global information. Let D = [d1, d2, d3, ..., d c ]: Among them, U gc is the global information, c is the number of channels; In order to promote the effective interaction of global and local information, the global information obtained through the diagonal matrix is ​​combined with the local information obtained through the interaction matrix, and the correlation between the two at different granularities is captured through mutual correlation operations, which is formalized as follows: In formula (4), M represents the correlation matrix; Then, in order to accurately assign feature weights and reduce computational complexity, an adaptive fusion strategy is adopted; row and column information are extracted from the correlation matrix and its transpose as weight vectors of global and local information; dynamic fusion is achieved through a learnable factor, as shown in formulas (5) to (7); in, is the fused global channel weight, is the local channel weight after fusion, C is the number of channels; σ(X) represents the sigmoid activation function; θ is an input variable, usually a learnable parameter or an intermediate variable, which is used to control the fusion ratio of global and local information after the sigmoid activation function; Then, the sum is taken in the channel dimension to obtain the final attention weights out1 and out2, and they are mapped to the (0,1) interval through the sigmoid function; then, the two attention weights are combined using a mixing layer, and the mixed attention features are further convolved and activated using the sigmoid function in one dimension; finally, the element-by-element multiplication result of the input and the calculated attention feature map is returned to enhance the channel features; finally, the obtained weight is multiplied by the input feature map as follows: Among them, F is the input feature map, F * is the final output feature map.

7. The RCSOSA-AFGC traffic sign detection method according to claim 1, characterized in that: The RCSOSA module mainly consists of a scaling factor e, two 1x1 convolutional layers and a sequence module consisting of n SR modules stacked together; the input feature first undergoes a 1x1 convolution through conv1 to adjust the number of channels and generate an intermediate feature map; then the intermediate feature map passes through the SR module, undergoes n SR operations, performs channel division, depth convolution and channel shuffling; finally, after a 1x1 convolution through conv2, the number of channels of the output feature map is adjusted to c2.

8. The RCSOSA-AFGC traffic sign detection method according to claim 7, characterized in that: The SR module contains a depth-wise separable convolution Conv2d, a batch normalization layer BatchNorm2d, and an activation function SiLU; the input feature map x is first divided into two parts along the channel dimension: x1 and x2, and x2 is processed by convolution, batch normalization, and activation function to obtain a feature map; then x1 and the processed x2 are concatenated in the channel dimension to form the output; The output features undergo channel shuffle operation.

9. The RCSOSA-AFGC traffic sign detection method according to claim 1, characterized in that: The input end of the MDBF detection head receives three feature maps P3, P4, and P5. P3 is responsible for detecting small targets, P4 is responsible for detecting medium targets, and P5 is responsible for detecting large targets. Each input feature is first processed by 1×1 ConvGN dimensionality reduction, and then feature extraction is performed through two shared 3×3 ConvGNs. Finally, it is divided into two branches, one2one and one2many. Each branch has two tasks: regression and classification.

10. A RCSOSA-AFGC traffic sign detection system integrating a lightweight detection head, characterized in that: The system comprises a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the traffic sign detection method as described in any one of claims 1 to 9.

Images