Foggy day traffic sign detection method based on attention modeling and multi-branch aggregation

Abstract

The invention discloses a foggy day traffic sign detection method based on attention modeling and multi-branch aggregation, and the method comprises the steps: providing a C2fDCAM module in a backbone network, and improving the fusion capability of shallow texture and deep semantics through a multi-branch convolution and feature interaction enhancement mechanism; transformerARSBlock is introduced into an RT-DETR encoder, and through fusion of adaptive sparse attention modeling and a spatial enhancement feedforward structure, the global correlation modeling capability of traffic sign features and the discriminative expression of a key area in a foggy environment are enhanced; meanwhile, an MBRepConvC3 module is introduced, and multi-direction enhancement modeling is carried out on local features of edge blur and directional structure feature weakening under the foggy weather condition through a multi-branch reparameterized convolution structure. Experimental results show that the method significantly improves the detection precision and robustness in a foggy day traffic sign detection task, maintains high real-time performance, and has a good engineering application prospect.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, specifically to a method for detecting traffic signs in foggy weather based on attention modeling and multi-branch aggregation. Background Technology

[0002] With the rapid development of intelligent transportation systems and autonomous driving technology, traffic sign detection, as a crucial link in road environment perception, directly determines the rationality of vehicle decisions and driving safety. Currently, mainstream object detection algorithms have achieved high recognition performance in normal scenarios such as sunny days, but they still face severe technical challenges under complex weather conditions such as fog, rain, and backlighting. The impact of foggy conditions is particularly prominent: fog leads to a decrease in overall image contrast, severe attenuation of color information, and blurred boundaries between traffic signs and the background. This makes it difficult for detection models to capture effective discriminative features during the feature extraction stage, directly affecting the accuracy of subsequent object recognition.

[0003] This image degradation problem has a significant impact on various detection algorithms: traditional handcrafted feature-driven detection methods, such as cascaded classifiers based on Haar features, HOG+SVM frameworks, and Deformable Part Models (DPM), suffer from limited feature representation capabilities and poor robustness to environmental noise. In low-visibility environments, they are prone to problems such as a sharp drop in detection accuracy, an increase in false detection rate, and insufficient scene adaptability. On the other hand, deep learning models based on convolutional neural networks, such as the YOLO series and the RT-DETR with the Transformer structure, perform well in conventional scenes, but still suffer from pain points such as blurred features, inaccurate target boundary localization, and missed detection of small targets in foggy environments.

[0004] More importantly, the negative impact of low visibility environments can be compounded by the multi-scale distribution characteristics of traffic signs. Signs in the distance are small and easily obscured by fog, while signs in the foreground may be affected by light reflection or partially obscured. Traditional multi-scale feature fusion structures are prone to information imbalance and loss of details when fusing cross-layer information, making it difficult to take into account both high-level semantic features and low-level texture features, resulting in insufficient stability of detection results.

[0005] To address these issues, some studies have introduced attention mechanisms or context enhancement modules to improve the model's ability to perceive small and low-contrast targets. However, these methods often increase computational complexity and inference latency, failing to meet the real-time requirements of traffic scenarios. On the other hand, while RT-DETR has the advantage of end-to-end detection, its feature interaction and semantic modeling capabilities are still limited under complex weather conditions, making it difficult to effectively extract subtle structural features affected by fog.

[0006] In summary, existing detection algorithms generally suffer from insufficient detection accuracy due to decreased image contrast and blurred edges, inadequate multi-scale feature fusion, and limited real-time performance when dealing with traffic sign detection tasks in foggy weather. Therefore, there is an urgent need for a traffic sign detection method specifically designed for low-visibility environments like fog, which can effectively enhance the model's ability to recognize low-contrast and small-scale traffic sign features while maintaining real-time performance, thereby improving the robustness and reliability of the traffic sign detection system in foggy scenarios. Summary of the Invention

[0007] Objective: To address the issues of decreased contrast, blurred edge structures, and degraded target features in traffic sign images under foggy conditions, leading to missed detections of small targets, insufficient multi-scale feature fusion, and limited feature modeling performance, this invention proposes a foggy traffic sign detection method based on attention modeling and multi-branch aggregation. While maintaining lightweight and real-time performance, the method enhances the feature fusion capability of low-contrast and small-scale traffic signs by designing a C2f_DCAM module to achieve cross-layer feature interaction and channel recalibration. A Transformer_ARSBlock module is introduced into the RT-DETR encoder, which strengthens the global correlation modeling capability of traffic sign features and the discriminative representation of key regions under foggy conditions by fusing adaptive attention modeling and spatial enhancement feedforward structures. Simultaneously, an MBRepConvC3 module is introduced, utilizing a multi-branch reparameterizable convolutional structure to perform multi-directional enhancement modeling of weakened edge and directional structural features under foggy conditions, improving the discriminativeness of local structural features without significantly increasing inference computational complexity, thereby improving the accuracy and robustness of foggy traffic sign detection.

[0008] Technical solution: This invention discloses a method for detecting traffic signs in foggy weather based on attention modeling and multi-branch aggregation, comprising the following steps:

[0009] Beneficial effects:

[0010] 1. This invention introduces the C2f_DCAM module into the backbone network. Through dual-branch feature preservation and multi-layer DCAMBlock stacking structure, it fully mines the small target features and multi-scale information of traffic signs in foggy scenes, effectively alleviating the problem of detail loss caused by decreased contrast. At the same time, by combining residual fusion mechanism and 1×1 convolution channel control method, it enhances the basic feature expression ability while maintaining low computational overhead, providing high-quality multi-scale feature input for subsequent feature modeling.

[0011] 2. This invention constructs the Transformer_ARSBlock module to make targeted improvements to the Transformer feature modeling unit in the RT-DETR encoder. It introduces adaptive sparse attention modeling and spatial augmentation feedforward structure. While maintaining the stability of the original global modeling framework, it significantly enhances the global correlation modeling ability between traffic sign features under foggy conditions and the discriminative expression of key regions, thereby improving the semantic modeling stability and detection generalization performance of the model in low-contrast environments.

[0012] 3. This invention introduces the MBRepConvC3 module in the encoder stage of RT-DETR to address the problems of blurred edges and weakened directional structural features of traffic signs in foggy conditions. It uses a multi-branch reparameterizable convolutional structure to enhance local features in multiple directions, significantly improving the model's ability to identify the local structure and geometric features of traffic signs. At the same time, by using a reparameterizable deployment method, the multi-branch structure is equivalently compressed into a single convolution in the inference stage, balancing detection accuracy and real-time performance without significantly increasing the computational complexity of inference, thus improving the overall robustness of the model in foggy traffic sign detection tasks. Attached Figure Description

[0013] Figure 1 This is a flowchart of the improved fog traffic sign detection method based on RT-DETR of the present invention;

[0014] Figure 2 This is a diagram showing the overall algorithm network structure of the improved RT-DETR method of this invention.

[0015] Figure 3 This is a structural diagram of the C2f_DCAM module based on the improved RT-DETR method of this invention;

[0016] Figure 4 This is a structural diagram of the DCAMBlock module based on the improved RT-DETR method of this invention;

[0017] Figure 5 This is a structural diagram of the Transformer_ARSBlock module based on the improved RT-DETR method of this invention;

[0018] Figure 6 The flowchart and FRFN module diagram are shown below for the ASSA module based on the improved RT-DETR method of this invention.

[0019] Figure 7 This is a structural diagram of the SEFN module based on the improved RT-DETR method of this invention;

[0020] Figure 8This is a structural diagram of the MBRepConvC3 module based on the improved RT-DETR method of this invention;

[0021] Figure 9 This is a graph showing the changes in training and validation loss and performance metrics of the model of this invention.

[0022] Figure 10 The images shown are from the improved RT-DETR detection dataset in this embodiment of the invention. Detailed Implementation

[0023] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0024] This invention discloses a foggy traffic sign detection method based on attention modeling and multi-branch aggregation, comprising the following steps:

[0025] Step 1: Select a suitable dataset of foggy traffic signs and preprocess it. Then, divide the dataset into training, testing and validation sets according to a certain ratio.

[0026] In step 1, the images in the foggy image dataset are preprocessed using various enhancement methods. For example, the Mosaic enhancement strategy randomly stitches multiple images together in the same frame to simulate the distribution of targets at different scales. The images are randomly flipped and their brightness is adjusted to enhance the model's adaptability to lighting, blur, and noise. Finally, the dataset is divided into training set, validation set, and test set.

[0027] Step 2: As Figure 2 The diagram shows the overall network structure of the improved RT-DETR algorithm. In the backbone network of RT-DETR, a C2f_DCAM module is introduced. The C2f_DCAM module enhances features through multiple layers of DCAMBlocks. Each DCAMBlock contains convolutional mapping, a convolutional attention mechanism, and an MLP feedforward network. It also achieves adaptive fusion of multi-scale features through residual connection operations, thereby enhancing the feature extraction capability of small targets under foggy conditions.

[0028] like Figure 3 As shown, after receiving the input features, the C2f_DCAM module first uses a 1×1 convolution layer to linearly map the input features in the channel dimension, transforming them into the set number of intermediate channels.

[0029] The mapped features are divided into two sub-features in the channel dimension. Each sub-feature is processed in one of the n DCAMBlock structures corresponding to the diagram, so that local spatial information is gradually accumulated through multi-level convolution enhancement and attention enhancement.

[0030] like Figure 4 As shown, within the DCAMBlock module, for any input feature x, a 1×1 convolution is performed to obtain feature x0, denoted as:

[0031]

[0032] in, express Convolution operation; The main branch features are generated after convolution, and at the same time It is also used as a residual branch for subsequent feature overlay.

[0033] Subsequently, Simultaneous batch normalization is performed, and the data is then input into the convolutional attention structure. For example... Figure 4 As shown, this module consists of vertical and horizontal convolutional branches, used to extract local dependencies of features in both directions. Each branch uses a 7×1 convolutional kernel, and its calculation is as follows:

[0034]

[0035] The other branch uses a 1×7 convolution kernel, and its calculation is expressed as follows:

[0036]

[0037] in, and These represent the core size as follows: , directional convolution; and These are local feature maps in the vertical and horizontal directions, respectively.

[0038] The results of the two-directional convolutions are divided according to the attention head, and the directional attention weights are obtained by Softmax operation respectively:

[0039]

[0040]

[0041] in, This is a normalization operation; and This is a weighted graph of attention in two directions.

[0042] The attention weights then undergo shape restoration to ensure their spatial dimensions match the input features. The restored attention weights are then linearly mapped again using a convolutional kernel aligned with the aforementioned direction. The calculation can be expressed as:

[0043]

[0044]

[0045] The outputs from both directions are element-wise summed to form the final result of the convolutional attention:

[0046]

[0047] in, This represents the attention features after fusion.

[0048] Then, output attention The residual branch, after being processed by synchronous batch normalization, is added for the first time to obtain the residual result of the first stage. .

[0049] The obtained features The input is fed into a feedforward network structure, MLP. For example... Figure 4 The MLP module structure diagram shown first examines the input features. After performing SyncBatchNorm, the data undergoes 3×3 convolution and activation function processing, followed by local spatial modeling and channel transformation. The expression is as follows:

[0050]

[0051] in, for Convolution operation; This represents the output characteristics of the MLP.

[0052] Finally, the MLP output and Perform a second residual superposition to obtain the final output of DCAMBlock:

[0053]

[0054] in This is the final output of the DCAMBlock.

[0055] The outputs of multiple DCAMBlocks are concatenated with the original input features along the channel dimension and integrated through a 1×1 convolution layer. This effectively fuses the fine-grained information generated by multi-level convolutional attention enhancement and feedforward transformation, resulting in C2f_DCAM output features with stronger spatial structure perception and robustness. These features are suitable for enhancing feature expression intensity in weak texture scenarios such as foggy weather and small targets.

[0056] Step 3: To address the issue of weakened feature relationships and unstable global semantic modeling caused by reduced contrast of traffic sign targets in foggy conditions, this invention specifically improves the Transformer feature modeling unit in the RT-DETR model, proposing a Transformer_ARSBlock feature reconstruction module that integrates adaptive attention modeling and a spatially enhanced feedforward structure. This module, while maintaining the stability of the original Transformer residual connections and layer normalization framework, enhances the global correlation modeling capability and discriminative representation of key regions of traffic sign features under foggy conditions by introducing an adaptive sparse attention modeling unit and a spatially enhanced feedforward network. See also Figure 2 The output features of the topmost C2f_DCAM module of the backbone network are processed by convolution and then enter the Transformer_ARSBlock feature reconstruction module for further processing.

[0057] In this embodiment, as Figure 5 As shown, the Transformer_ARSBlock module is set to take the input feature x∈R as input. H ×W×C As input, the features are subjected to attention modeling and spatial augmentation reconstruction.

[0058] The input features are first fed into the Adaptive Attention Modeling (AST) module to model the feature relationship between traffic sign targets and background areas in foggy scenes.

[0059] In the AST module, the input feature x first enters the ASSA submodule. For example... Figure 6 As shown, the ASSA submodule performs layer normalization on the input features to stabilize the feature distribution, and the result is expressed as follows:

[0060]

[0061] in, Presentation layer normalization operation.

[0062] The normalized features are then used to generate query features, key features, and value features through linear projection:

[0063]

[0064] in, is a learnable parameter matrix.

[0065] Subsequently, the attention response is calculated by the correlation between query features and key features: .

[0066] The ASSA module employs a two-branch adaptive sparse modeling approach for the attention response. One branch performs ReLU activation and squaring on the attention response to enhance high-response regions.

[0067]

[0068] The other branch performs Softmax normalization on the attention response:

[0069]

[0070] in, This represents the normalization function.

[0071] Subsequently, learnable weight parameters are introduced to modulate the results of the two branches, and sparse attention weights are generated through element-wise multiplication:

[0072]

[0073] Where w is a learnable weight parameter, and ⊙ represents element-wise multiplication, used for feature weighting or gated modulation.

[0074] Sparse attention weights are applied to value features to complete feature aggregation:

[0075]

[0076] Then, the channel dimension is restored through linear mapping to obtain attention-enhanced features. .

[0077] Finally, the attention-enhanced features are added to the original input features element-wise using residual summation to obtain the output of the ASSA submodule:

[0078]

[0079] In the AST module, the output features of the ASSA submodule are then fed into the FRFN submodule for further feature refinement. For example... Figure 6 As shown, the FRFN module first performs layer normalization on the input features:

[0080]

[0081] The normalized features are then sequentially processed through pointwise convolution and linear transformation to enhance the channel's expressive power.

[0082]

[0083] in, This indicates a pointwise convolution operation, used for information exchange along the channel dimension; This represents a linear transformation operation.

[0084] Subsequently, the features are divided into two parts along the channel dimension:

[0085]

[0086] in, This indicates that the feature is divided into multiple sub-features along the channel dimension.

[0087] Some of the features After shape rearrangement, the input is a depthwise convolution to introduce spatial structure information, and then restored to a feature form consistent with the other branch through a flattening operation.

[0088]

[0089] in, This indicates a feature shape rearrangement operation; This represents depthwise convolution, used to introduce spatial information while reducing computational cost; This indicates a feature flattening operation.

[0090] The two features are then fused using matrix multiplication:

[0091]

[0092] The fused features are used to generate refined feature outputs through linear mapping, and then the residuals are added to the FRFN input features to obtain the final output of the AST module:

[0093]

[0094]

[0095] The features output by the AST module are then residually fused with the input features x of Transformer_ARSBlock, and then processed by layer normalization to obtain intermediate feature representations:

[0096]

[0097] intermediate features The input features x from Transformer_ARSBlock are fed together into the Spatial Augmentation Feedforward Network (SEFN) for further processing. Figure 7 As shown, the SEFN module includes a local feature enhancement branch and a spatial guidance branch. The local feature enhancement branch performs convolution and depthwise convolution operations on the input features:

[0098]

[0099] The obtained features are then divided into two parts along the channel direction:

[0100]

[0101] The spatial guidance branch performs average pooling, convolutional mapping, and upsampling operations on spatial features:

[0102]

[0103] in, This indicates an average pooling operation; This indicates an upsampling operation used to restore spatial resolution.

[0104] Subsequently, the local branch features are concatenated with the spatial guidance features:

[0105]

[0106] in, This indicates a feature splicing operation.

[0107] The concatenated features are spatially augmented using pointwise convolution, depthwise convolution, and the GELU activation function.

[0108]

[0109] in, This represents the activation function, used to enhance nonlinear expressive power.

[0110] Enhanced features The features from another local branch are spatially modulated through element-wise multiplication, and then the output features of the SEFN module are generated through pointwise convolution.

[0111]

[0112] Finally, the output features of the SEFN module are compared with the intermediate features. Residual fusion is performed, and layer normalization is applied to obtain the final output features of the Transformer_ARSBlock module:

[0113]

[0114] Step 4: Introduce the MBRepConvC3 module in the encoder stage of RT-DETR, see [link / reference]. Figure 2The original RepC3 module was replaced. To address the issues of blurred traffic sign edges and weakened directional structural features in foggy scenarios, a multi-branch reparameterizable convolutional structure was used during the training phase to enhance local features in multiple directions. Furthermore, the sub-module underwent structural reparameterization during the inference phase. This improved the discriminative power of local structural features without significantly increasing inference computational complexity, thereby enhancing the model's robustness and computational efficiency in foggy traffic sign detection tasks.

[0115] like Figure 8 As shown, in the MBRepConvC3 module, let the input feature tensor be... B represents the batch size. Let H be the number of input channels, and H and W be the height and width of the feature map, respectively. The output is... ,in This represents the number of output channels.

[0116] The module processes the input features in two parallel branches: one is the feature extraction branch, where the input features first undergo 1×1 convolution, which can be represented as:

[0117]

[0118] in, This represents a 1×1 convolution of the feature extraction branch.

[0119] Then it enters the feature extraction unit consisting of n MBRepConv modules cascaded together, such as Figure 8 As shown, MBRepConv is the core subunit of the MBRepConvC3 module. This module first feeds the input features into four different convolution operators to extract multi-scale and multi-directional information. The calculation form of the standard 3×3 convolution branch can be expressed as:

[0120]

[0121] The output of a 1×1 convolution branch can be represented as

[0122]

[0123] The enhanced 3×1 convolution branch in the vertical direction can be represented as:

[0124]

[0125] The enhanced 1×3 convolution branch in the horizontal direction can be represented as:

[0126]

[0127] Subsequently, each convolutional branch undergoes batch normalization, the calculation of which can be written as follows:

[0128]

[0129] in, For scaling parameters; For offset parameters; , These are the batch mean and batch variance, respectively. To avoid dividing by zero for small constants.

[0130] This results in four convolutional outputs and their corresponding four normalized outputs, for a total of eight feature paths.

[0131] The above eight features are cascaded along the channel dimension to obtain the spliced ​​features.

[0132]

[0133] The concatenated features are fused and compressed using a 1×1 convolution, and then the final output features of the MBRepConv module are obtained through the SiLU activation function.

[0134]

[0135] in, This represents the activation function.

[0136] Meanwhile, the input features of another branch are directly processed by a 1×1 convolution, serving as a direct-connect fusion branch, which can be represented as:

[0137]

[0138] The multi-branch extraction results of branch 1 and the direct connection results of branch 2 are added element by element to achieve feature fusion, which takes into account the rich information of multi-branch extraction and the original information of direct connection branch, and alleviates the gradient vanishing problem.

[0139] Finally, the fused features are subjected to 1×1 convolution to complete channel matching, ultimately yielding the module's output features:

[0140]

[0141] During the model deployment phase, each MBRepConv submodule within the MBRepConvC3 module independently performs structural reparameterization.

[0142] The MBRepConv module folds multi-branch convolutions during training into a single, equivalent 3×3 convolution through structural reparameterization.

[0143] Specifically, for each branch, its convolution is first fused with the corresponding BN parameters to obtain an equivalent convolution kernel and bias. The fusion rule can be expressed as:

[0144]

[0145]

[0146] in, , , , , , These are the convolution weights, biases, and BN parameters for the corresponding branches.

[0147] In addition, the equivalent convolutional kernels of 1×1, 3×1, and 1×3 are expanded to a 3×3 size through zero padding to achieve shape consistency. Then, according to the linear combination rule of the 1×1 weights at the end of the fused network, the eight-way fused convolutional kernels and biases are merged into the final equivalent parameters based on the channel compression relationship. Therefore, the equivalent calculation in the reasoning stage can be performed using the formula:

[0148]

[0149] In this way, the module provides multi-directional enhanced feature representation during the training phase, while achieving structural compression and computational acceleration during the inference phase, thereby improving the overall robustness of the model in the fog traffic sign detection task.

[0150] Step 5: Using the improved RT-DETR model completed in Steps 2 to 4, train it on a dedicated dataset for foggy traffic signs; evaluate and iteratively optimize the model's performance based on the training output; and finally, deploy the model to achieve high-precision detection of foggy traffic signs.

[0151] The original RT-DETR network and the improved RT-DETR network were used for model training. The experimental environment was configured as follows: Python 3.8, PyTorch 2.0.0, CUDA 11.8, and NVIDIA GeForce RTX 3090 graphics card.

[0152] The foggy traffic sign dataset was iteratively trained using rtdetr-r18.pt weights, the RT-DETR network, and the improved network of this invention. The training and validation loss and performance index change curves of the model, along with the visualization of the detection results, are shown in the figure below. Figure 9 , 10 As shown.

[0153] Table 1 Experimental Results

[0154]

[0155] As shown in Table 1, the original RT-DETR model, after 150 iterations of training, achieved an mAP of 0.50 of 87.5%, with 20,085,596 parameters, 58.3 GFLOPs, and a model size of 77.0 MB. The large number of parameters, complex model training, long training time, and low detection accuracy are significant limitations in foggy traffic sign detection tasks.

[0156] After 150 iterations of training, the improved RT-DETR network has a model size of 78.8 MB, 20,151,120 parameters, and 59.7 GFLOPs. Compared to the original RT-DETR network, the model's mAP0.50 has increased from 87.5% to 90.9%. Furthermore, the model size, number of parameters, and GFLOPs have only seen minor increases.

[0157] The improved RT-DETR detection model of the present invention was subjected to ablation experiments, and the specific results are shown in Table 2.

[0158] Table 2 Ablation Experiment Results

[0159]

[0160] Table 2 shows that in the experimental results, P (%) represents precision, R (%) represents recall, mAP0.50 (%) represents the average detection precision at an intersection-union ratio (IU) of 0.5, and mAP0.50:0.95 (%) represents the average detection precision from IU to IU of 0.5 to 0.95. Experimental results on the foggy traffic sign detection dataset demonstrate that the proposed structural improvement can significantly enhance the detection performance of RT-DETR in foggy conditions.

[0161] Compared with the original model, the introduction of C2f_DCAM improved the model's P by 1.1%, R by 0.8%, mAP0.50 by 1.3%, and mAP0.50:0.95 by 2.2%. All indicators showed significant improvement, indicating that the module can enhance the ability to preserve shallow texture features and multi-scale information, thereby providing high-quality basic feature support for small target traffic signs under foggy conditions.

[0162] Further integration of the Transformer_ARSBlock module resulted in a significant improvement in detection performance, with a 2.0% increase in P, a 1.7% increase in R, a 2.2% increase in mAP0.50, and a 3.8% increase in mAP0.50:0.95. This significantly surpasses the performance of the original model and the standalone C2f_DCAM, demonstrating that the adaptive sparse attention and spatially enhanced feedforward structure can strengthen global feature association and key region discrimination capabilities in low-contrast environments, effectively improving the accuracy of global semantic modeling.

[0163] When using the MBRepConvC3 module alone, the model's accuracy also increased slightly, mainly due to the enhanced modeling ability of the multi-branch reparameterizable convolutional structure for local structure and directional features, enabling the model to more accurately identify traffic signs with weakened or occluded local structures.

[0164] In the combined experiment, the model's detection performance for near, medium and far-range targets was further improved by integrating C2f_DCAM and Transformer_ARSBlock simultaneously. The P-value was improved by 2.4%, the R-value by 2.4%, the mAP0.50 by 2.8%, and the mAP0.50:0.95 by 4.7%, showing the synergistic gain of multi-layer convolutional representation and global attention modeling, which effectively alleviates the feature degradation problem under low contrast conditions in foggy weather.

[0165] When all three modules are applied to the model simultaneously, the detection performance reaches its optimal level, with P improving by 3.0%, R by 3.6%, and mAP0.50 by 3.4% compared to the original model. In particular, the mAP0.50:0.95 is improved by 5.7% compared to the original model. The results show that the overall improved structure can significantly enhance RT-DETR's ability to recognize distant small targets and traffic signs with weakened local structures in foggy and low visibility scenarios, thereby improving the robustness and accuracy of the model.

[0166] Comprehensive experimental results show that, while maintaining network lightweightness and real-time performance, the present invention significantly improves the accuracy and robustness of traffic sign detection in foggy weather, and verifies the effectiveness and synergistic enhancement of the C2f_DCAM, Transformer_ARSBlock, and MBRepConvC3 modules in foggy weather detection tasks.

[0167] Based on the RT-DETR detection framework, this invention constructs a detection model that combines feature enhancement and structural efficiency to address the problems of blurred, low-contrast, and missing outlines in traffic signs under foggy conditions.

[0168] In the feature extraction stage, this invention sets up a C2f_DCAM module to enhance the fine-grained expression of shallow textures and multi-scale basic features, and stabilizes the basic features of weak texture regions in foggy weather through multi-level convolution aggregation.

[0169] In the encoder stage, the Transformer_ARSBlock module is introduced. Through the synergistic effect of adaptive sparse attention modeling and spatial augmentation feedforward structure, the global feature association modeling ability and the discriminative representation of key regions are enhanced, thereby improving the overall performance of the model in foggy environments.

[0170] To further enhance the directional features of the local structure, this invention introduces the MBRepConvC3 module in the RT-DETR encoder stage. The local structure and directional features are enhanced through a multi-branch convolutional structure, and a highly efficient single convolutional computation structure is formed by reparameterization in the inference stage. This improves the expression effect of local features without increasing computational overhead.

[0171] Through the synergistic effect of the above structures, the model can maintain stable feature representation in degraded scenarios such as low contrast in foggy weather, occlusion, and distant small targets, improving the detection accuracy and robustness of traffic signs, while also taking into account real-time requirements, making it suitable for vehicle-mounted visual perception tasks under foggy weather conditions.

[0172] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent transformations or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for detecting traffic signs in foggy weather based on attention modeling and multi-branch aggregation, characterized in that, It includes the following steps: Step 1: Obtain the fog traffic sign dataset and preprocess it, then divide the dataset into training set, test set and validation set according to a certain ratio; Step 2: In the backbone network of RT-DETR, a C2f_DCAM module is proposed to replace the feature extraction module of the backbone network. The C2f_DCAM module enhances features by replacing the Bottleneck of the C2f module with a multi-layer DCAMBlock. Each DCAMBlock contains a convolutional mapping, a convolutional attention mechanism and an MLP feedforward network, and achieves adaptive fusion of multi-scale features through residual connection operations. Step 3: Improve the Transformer feature modeling unit in the RT-DETR model and propose the Transformer_ARSBlock feature reconstruction module; The Transformer_ARSBlock feature reconstruction module, while maintaining the stability of the original Transformer residual connection and layer normalization framework, enhances the global correlation modeling ability of traffic sign features and the discriminative expression of key regions under foggy conditions by introducing adaptive sparse attention modeling units and spatially enhanced feedforward networks. Step 4: Introduce the MBRepConvC3 module in the encoder stage of RT-DETR. Through the multi-branch reparameterizable convolutional structure in the training stage, perform multi-directional enhancement modeling of local features and perform structural reparameterization in the inference stage. Step 5: Using the improved RT-DETR model completed in Steps 2 to 4, train it on a dedicated dataset for foggy traffic signs; evaluate and iteratively optimize the model's performance based on the training output; and finally, deploy the model to achieve high-precision detection of foggy traffic signs.

2. The fog traffic sign detection method based on attention modeling and multi-branch aggregation according to claim 1, characterized in that, The C2f_DCAM module is described in detail below: The input features are first subjected to a 1×1 convolution for preliminary mapping and transformation of the channel dimensions; Then, a separation operation is performed to split the convolutional features into two sub-feature branches; A feature branch is fed into n stacked DCAMBlock modules for processing. The processed features are concatenated with the unprocessed features, and then a 1×1 convolution is used to complete the final integration of the channel dimensions, resulting in the output of the C2f_DCAM module.

3. The fog traffic sign detection method based on attention modeling and multi-branch aggregation according to claim 2, characterized in that, The specific process of DCAMBlock is as follows: First, through the channel dimension Convolution maps the input; The features are then batch normalized and fed into a convolutional attention unit for spatial correlation modeling. In the convolutional attention unit, first use The convolutional kernel extracts spatial information in the vertical direction, which is then normalized by Softmax to form a vertical attention map, and then restored to the original channel dimension through transposed convolution. Then use The convolutional kernel extracts horizontal spatial information, which is then normalized by Softmax and transposed convolution to obtain the horizontal attention map. The vertical and horizontal attention enhancement features are added together to obtain the convolutional attention output. The output is then added to the residuals of the normalized features from the synchronous batch to complete the first stage of residual fusion. The fused features are then input into the MLP module, which first performs synchronous batch normalization and then sequentially passes them through two... Convolution enables channel and spatial transformations; The MLP output is then added to the previous fused feature to complete the second stage of residual fusion, which is then used as the DCAMBlock output.

4. The fog traffic sign detection method based on attention modeling and multi-branch aggregation according to claim 1, characterized in that, The Transformer_ARSBlock module is described in detail below: First, the input feature x is input into the adaptive attention modeling unit AST to model the spatial position relationship and channel correlation of the feature. The adaptive attention modeling unit AST adopts a serial structure, which includes the sparse self-attention modeling submodule ASSA and the feature refinement feedforward submodule FRFN in sequence, and residual connection structures are set at the output of the two submodules respectively. The features processed by the adaptive attention modeling unit (AST) are residually fused with the input features, and then subjected to layer normalization to obtain the attention-enhanced features. ; Then, intermediate features Together with the input feature x, it is fed into the Spatial Augmentation Feedforward Network (SEFN) for feature reconstruction and output feature y; Output feature y and intermediate features Residual fusion is performed, and the final output feature 'out' of the Transformer_ARSBlock module is obtained through layer normalization.

5. The fog traffic sign detection method based on attention modeling and multi-branch aggregation according to claim 4, characterized in that, The spatial augmentation feedforward network (SEFN) adopts a multi-branch structure: One branch pairs of intermediate features Local spatial details are modeled by convolution and depthwise convolution operations, and the resulting features are divided into two parts x1 and x2 along the channel direction. Another branch introduces global contextual information into the input feature x through spatial pooling and feature mapping operations, and restores the feature resolution through upsampling to obtain spatially guided features; After concatenating the local branch feature x1 with the spatial guided feature, spatial enhancement is performed through pointwise convolution, depthwise convolution, and the GELU activation function to obtain enhanced features; The enhanced feature and another local branch feature x2 are spatially modulated by element-wise multiplication, and the SEFN module output feature y is generated by pointwise convolution.

6. The fog traffic sign detection method based on attention modeling and multi-branch aggregation according to claim 1, characterized in that, The MBRepConvC3 module is described in detail below: The MBRepConvC3 module is a composite reparameterizable module based on MBRepConv. After receiving the input features, the MBRepConvC3 module performs feature processing in two parallel branches: Branch 1: Directly perform a 1×1 convolution on the input features to complete the initial transformation of the channel dimensions; Branch 2: First, perform a 1×1 convolution on the input features to adjust the channels, and then connect n stacked MBRepConv sub-modules to achieve multi-round, multi-directional feature enhancement; After stacking n MBRepConv sub-modules, the output of branch 2 is added to the 1×1 convolution output of branch 1 to fuse the dual-branch feature information. The features after residual fusion are then processed by a 1×1 convolution to complete channel integration and feature compression, finally obtaining the output features of the MBRepConvC3 module.

7. The fog traffic sign detection method based on attention modeling and multi-branch aggregation according to claim 6, characterized in that, The MBRepConv is a multi-branch reparameterizable convolutional structure. After receiving the input features, the module performs multi-directional feature extraction through four different convolutional branches: 1) Standard 3×3 convolutional branches are used to capture local spatial context; 2) 1×1 convolutional branch, used for channel compression and detail enhancement; 3) 3×1 convolutional branches are used to enhance the perception of vertical features; 4) A 1×3 convolutional branch is used to enhance the horizontal feature response; Each convolutional branch is followed by a batch normalization layer. Then, the convolutional outputs of the four convolutional branches and their corresponding normalized outputs are concatenated in the channel dimension to form a total of eight feature channel combinations. The eight-way concatenated features are fused and compressed using a 1×1 convolution to generate the module's output features, which are then activated by the SiLU activation function.

8. The fog traffic sign detection method based on attention modeling and multi-branch aggregation according to claim 1, characterized in that, During the model deployment phase, each MBRepConv submodule within the MBRepConvC3 module independently performs structural reparameterization. First, the slim function is used to fold each convolutional branch and its normalization parameters into an equivalent 3×3 convolutional kernel and bias. Then, during the inference phase, the multi-branch structure is replaced with a single 3×3 convolution, achieving efficient structural simplification and computational acceleration during the inference phase.

Images