Complex scene traffic sign detection method and system based on dynamic frequency band focusing and double-domain attention screening
The traffic sign detection method based on dynamic frequency band focusing and dual-domain attention screening solves the problems of high false detection rate and false negative rate in traffic sign detection in complex environments, and achieves higher detection accuracy and recall rate, making it suitable for traffic sign detection in complex scenarios.
Patent Information
- Application Number
- CN202510938220.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-08
- Publication Date
- 2025-12-05
AI Technical Summary
Existing traffic sign detection models are prone to background interference in complex environments, resulting in high false detection and false negative rates, and their detection accuracy is low under conditions such as rain, snow, and fog.
A traffic sign detection method for complex scenes is proposed, which adopts dynamic frequency band focusing and dual-domain attention screening. The method replaces the BasicBlock module of RT-DETR with the CADPCM module, introduces the CHAttention attention mechanism, and constructs the DMPFCN neck network. It combines the CACHNet backbone network and RTDETRDecoder for feature fusion and training, and uses a traffic sign augmentation dataset to optimize the model weights.
It effectively reduces the number of model parameters, improves detection accuracy and recall, and enhances traffic sign detection performance in complex scenarios, especially under conditions such as rain, snow, and fog.
Smart Images

Figure CN121074845A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of computer vision and traffic sign detection, and particularly relates to a complex scene traffic sign detection method and system based on dynamic frequency band focusing and double-domain attention screening. BACKGROUND
[0002] In recent years, traffic sign detection has become a research hotspot as one of the key technologies of autonomous driving and advanced driver assistance systems (ADAS). Traffic signs play an important role in maintaining road traffic order and improving driving safety as an important part of road traffic management. However, in complex environments, traffic sign detection and recognition face many challenges, such as weather changes, dim light, occlusion, blurred or damaged signs, and other factors. These factors make it difficult for image processing-based methods to achieve high-precision target detection in real-world applications.
[0003] Image processing-based traffic sign detection methods mainly rely on classical computer vision techniques such as feature matching, edge detection, and template matching. Among them, feature matching often uses color and shape features to recognize signs. Color-based detection methods mainly extract pixels in a specific color range in the image to identify representative traffic signs, and common color spaces include RGB, HSI, HSV, etc. Shape-based detection methods analyze the geometric features of traffic signs, such as rectangular, circular, and triangular basic shapes, to achieve fast segmentation of signs. These methods rely on precisely designed features, but in complex environments, due to factors such as light changes and background interference, their recognition accuracy is often greatly affected, leading to decreased detection performance.
[0004] With the development of deep learning technology, convolutional neural networks have been widely applied in computer vision, and traffic sign detection technology has gradually transitioned to deep learning models. Compared with image processing-based traffic sign detection methods, deep learning-based traffic sign detection no longer relies on manual design and feature extraction, but instead uses a human-like neural network to automatically learn and adjust during training, thereby completing feature extraction and recognition. Currently, target detection algorithms based on deep learning can be roughly divided into two categories: one is the two-stage detection method represented by the R-CNN series, and the other is the one-stage detection method represented by YOLO and SSD.
[0005] RT-DETR is an end-to-end target detection framework that uses the encoder and decoder structure of the Transformer, improving detection speed while maintaining high efficiency and accuracy, and reducing the parameter amount of the model, making it applicable to more practical scenarios. The network architecture of RT-DETR mainly consists of three parts: backbone network, hybrid encoder and decoder. First, the backbone network is used to extract features from the input image; second, the hybrid encoder further processes these features; finally, the decoder selects a certain number of target queries based on the processed features and interacts with these queries through the Transformer structure to generate the final prediction results of the target.
[0006] Although existing research results have further improved the detection accuracy of traffic signs in complex scenes, the detection model still has the problems of large parameter amount and high false detection rate and high missing detection rate caused by background interference. Moreover, although the existing traffic sign detection model performs well in sunny environments, in complex conditions such as rain, snow and fog, it still has the problems of target missing and low detection accuracy due to image degradation and edge blur. SUMMARY
[0007] The purpose of the application is to solve the problems pointed out in the background art. The application proposes a complex scene traffic sign detection method and system based on dynamic frequency band focusing and dual domain attention screening, which improves the model based on RT-DETR and introduces dynamic frequency band focusing and dual domain attention screening, reduces the parameter amount, improves the detection accuracy and recall rate, and makes it better applied to complex scene traffic sign detection.
[0008] Technical scheme: The application discloses a complex scene traffic sign detection method based on dynamic frequency band focusing and dual domain attention screening, and the specific steps are as follows:
[0009] S1: Collect road traffic sign images as traffic sign data sets and perform data preprocessing, enhance part of the image data to simulate rain, fog and snow environment, and obtain traffic sign enhanced data set;
[0010] S2: Based on the RT-DETR backbone network, use the CADPCM module to replace the BasicBlock module in the RT-DETR backbone network, and increase the CHAttention attention mechanism after the CADPCM module, to construct the CACHNet backbone network;
[0011] S3: Construct a DMPFCN neck network, which introduces a hierarchical residual structure and a P2 detection layer to further fuse the shallow and deep features of the backbone network, and also introduces a WaveletPool wavelet pooling to decompose the shallow features in the frequency domain, and uses a TDFM module to effectively fuse features of different scales; the DMPFCN neck network combines the CACHNet backbone network and the RTDETRDecoder to construct a CDWN model, and uses a traffic sign enhanced dataset to train to obtain the best weight bt;
[0012] S4: Define a confidence threshold Conf, use the best model weight bt to perform traffic sign recognition and bounding box regression on the CDWN model, and output the detection accuracy and position information.
[0013] Further, the S1 is specifically implemented as follows:
[0014] S1.1 collects road traffic sign images, including images in different environments such as night, fog, snow, rain, occlusion and low light, and combines images in public datasets and part of the images collected by the network as a traffic sign dataset D1;
[0015] S1.2 adjusts the image size to the same size using mean normalization on the traffic sign dataset D1, and uses the LabelImg labeling tool to label the normalized traffic sign images, marking the position and category of each traffic sign, to obtain a traffic sign preprocessing dataset D2.
[0016] S1.3 uses the Gaussian blur, brightness reduction, noise addition, rain addition, fog addition, and snow addition in the imgaug library to perform data enhancement on part of the images in the traffic sign preprocessing dataset D2, to obtain a traffic sign enhanced dataset D3.
[0017] Further, S2 is specifically implemented as follows:
[0018] The CACHNet backbone network is composed of four layers of CADPCM modules, CHAttention attention mechanism, AIFI module and Conv in series; the input feature map is input into the four-layer CADPCM module in series after Conv processing for feature extraction, and the output feature of the uppermost CADPCM module is input into the AIFI module after CHAttention attention mechanism; the first three layers of CADPCM modules are used as P2, P3 and P4 detection layers, and the feature map processed by the AIFI module is used as the P5 detection layer output feature map.
[0019] Further, the CADPCM module is obtained by a deep convolution, a point convolution, a batch normalization BN, a ReLU activation function, a PConv convolution, a batch normalization BN, a SiLU activation function, and a CAA attention mechanism in series to obtain a first branch, the first branch is connected in parallel with a ShortCut jump connection containing an AvgPool average pooling and a 1x1 convolution to obtain a second branch, and finally the first branch and the second branch are added point by point, and the output feature map is obtained through the ReLU activation function.
[0020] Further, the CHAttention attention mechanism processing process is:
[0021] First, the input feature map C is processed in the preprocessing stage. The input feature map C is first reduced to one fourth of the original channel number through a deep convolution operation, then nonlinearly transformed through a GELU activation function, then the channel number is restored through a 1x1 convolution, and batch normalization is performed to obtain a feature map c1.
[0022] Secondly, the feature map c1 is reduced to 1 in spatial dimension through an AdaptiveAvgPool adaptive average pooling operation, then two 1x1 convolutions and a SiLU activation function are used to generate a channel attention factor, and a Sigmoid activation function is used for normalization to obtain the final channel attention factor ca.
[0023] Then, the feature map c1 is expanded through an AvgPool average pooling, then two 1x1 convolutions and two deep convolutions are connected in series to extract spatial information, and finally a Sigmoid activation function is used for normalization to obtain the final spatial attention factor sa.
[0024] Finally, the channel attention factor ca and the spatial attention factor sa are weighted and averaged, and then multiplied by the input feature map c1 to obtain the weighted output feature map c2.
[0025] Further, the DMPFCN neck network is implemented as follows:
[0026] The super-high resolution feature map extracted by the P2 detection layer, the high resolution feature map extracted by the P3 detection layer, the medium resolution feature map extracted by the P4 detection layer, and the low resolution feature map extracted by the P5 detection layer are set.
[0027] The DMPFCN neck network introduces a hierarchical residual structure and increases a P2 detection layer to deepen the network level. Through multiple convolution modules, up-sampling operations and feature fusion modules, the input feature maps are gradually processed and enhanced. First, the low-resolution feature map output by the P5 detection layer is enlarged in spatial size by DySample dynamic up-sampling to generate a feature map q1. The medium-resolution feature map output by the P4 detection layer is extracted by GSConv to obtain a feature map q2. Subsequently, the medium-resolution feature map output by the P4 detection layer, the feature map q1 and the feature map q2 are transmitted by the TDFM module to form a feature map q3. The feature map q3 is generated by GSConv to generate a feature map q4, which is further transmitted to the bottom layer. The high-resolution feature map output by the P3 detection layer is extracted by GSConv to obtain a feature map q5. The feature map q4 is up-sampled by DySample and fused with the high-resolution feature map output by the P3 detection layer and the feature map q5 by the TFDM module to obtain a feature map q6. The feature map q6 and the super-high-resolution feature map output by the P2 detection layer are dynamically sampled and interacted to obtain a feature map q7. Finally, the feature map q7 is processed by different scales, and is gradually down-sampled by WaveletPool wavelet pooling and Fusion feature fusion to further compress the spatial dimension and keep the key features. Finally, different scale output feature maps are obtained to provide multi-scale feature information for target detection.
[0028] Further, the input feature map of the TDFM module is 3, specifically a large-size input feature map F1, a medium-size input feature map F2 and a small-size input feature map F3. The specific execution process is as follows:
[0029] First, a 1x1 GSConv is used for the large-size input feature map F1 to change the output channel number of the feature map F1 to twice the input channel number. MaxPool maximum pooling and AvgPool average pooling are used in parallel and point-by-point addition is performed to adjust the feature map F1 to the same size as the medium-size input feature map F2.
[0030] A 1x1 GSConv is used for the small-size input feature map F3 to change the output channel number of the feature map F3 to half the input channel number. The nearest neighbor interpolation method is used to adjust the feature map F3 to the same size as the feature map F2.
[0031] The feature map F2 is unchanged in size and number of channels by a 1x1 convolution, and finally the three adjusted feature maps F1, F2 and F3 are transversely spliced using the splicing ConCat method to obtain a feature map f1. The feature map f1 is halved in channel number by a 1x1 convolution operation to obtain a feature map f2. The feature map f2 is processed by a 1x1 GSConv and a 3x3 GSConv to gradually extract features to obtain a feature map f3. Finally, the feature map f3 is spliced with the feature map f1 by a Concat operation and processed by a 1x1 convolution operation to obtain an output feature map f4.
[0032] Further, the CACHNet backbone network and the DMPFCN neck network and the RTDETRDecoder jointly construct a CDWN model, and a traffic sign augmented dataset is divided into a training set, a validation set and a test set in a ratio of 7:2:1, and the CDWN model is trained to obtain the best training weight bt, the validation set is validated using the bt, and the model parameters after parameter adjustment are: 100 training rounds, the optimizer is AdamW, and the learning rate is 0.0001.
[0033] Further, S4 is specifically implemented as follows:
[0034] S4.1 loads and initializes the CDWN model and uses the best model weight bt, inputs the traffic sign image into the system, performs traffic sign detection, and generates a preliminary border box set Border for each candidate region. On the basis of the candidate region, boundary box regression processing is performed to optimize the position parameters of the boundary box, and a corresponding confidence score S is generated for preliminary screening of the target. If S is greater than the confidence threshold Conf, the next step is entered, otherwise the candidate box is ignored;
[0035] S4.2 After generating the preliminary detection result, the boundary box set Border is traversed, the intersection over union IoU between each boundary box and other candidate boxes is calculated, if the IoU value of two candidate boxes is greater than the preset overlap threshold θ, the non-maximum suppression NMS algorithm is used to further screen the overlapping candidate boxes, the boundary box with the highest confidence is retained, and the remaining redundant boxes are discarded, if the IoU value is less than θ, the two boundary boxes are retained;
[0036] S4.3 outputs the final traffic sign detection result, after multi-layer screening of the IoU threshold and the confidence threshold, a final traffic sign detection dataset T is generated, which contains all reliable detection results and provides the boundary box position information and class label of each traffic sign.
[0037] The application further discloses a complex scene traffic sign detection system based on dynamic frequency band focusing and double-domain attention screening, comprising a memory, a processor and a computer program.
[0038] Beneficial effects:
[0039] 1. The CADPCM module replaces the BasicBlock module in the RT-DETR, replaces the ordinary convolution with the depth separable convolution, effectively reduces the parameter quantity, combines the PConv to suppress the occlusion and noise interference, and utilizes the CAA attention mechanism to dynamically focus on the key area, improves the feature expression capability, and finally retains the original information through the residual structure. Compared with the BasicBlock which depends on the standard convolution stacking, the CADPCM module effectively improves the robustness and detection performance of the model in complex scenes through the triple optimization of separable convolution, partial convolution and attention mechanism.
[0040] 2. The CHAttention attention mechanism is proposed, which dynamically enhances the traffic sign features through double-channel attention, focuses on the key semantics through channel attention, captures long-distance spatial relationships through spatial attention using depth convolution, and combines pre-processing feature purification and adaptive weighting. Compared with the EMA attention mechanism which easily ignores small target details, the local attention of the ELA attention mechanism which has limitations in rain and fog scenes, and the MLCA attention mechanism which ignores spatial directionality, the CHAttention has good detection effect in complex scenes such as rain and fog occlusion, strong light glare, etc. through complementary channel and spatial attention.
[0041] 3. The DMPFCN neck network designed by the application enhances the detection capability in complex scenes through multi-level feature fusion and adaptive sampling, uses DySample to retain small sign details, WaveletPool to maintain high-frequency features, combines GSConv light-weight convolution to reduce computational complexity, and uses TDFM to realize cross-scale feature interaction, and finally constructs a strong semantic multi-scale feature pyramid. Compared with the traditional PAN or FPN network with a single structure from top to bottom or from bottom to top, the DMPFCN neck network can better capture the multi-scale characteristics of traffic signs in complex environments by introducing hierarchical residuals and P2 detection layers, so as to more accurately detect traffic signs of different sizes.
[0042] 4, The experimental results prove that the CADPCM module is used, the average accuracy in complex scene traffic sign detection is improved by 1.8%; the CHAttention attention mechanism is used, the average accuracy in complex scene traffic sign detection is improved by 0.7%; the DMPFCN neck network is used, the average accuracy in complex scene traffic sign detection is improved by 2.7%; the method effectively enhances the extraction ability of traffic sign features in complex scenes, compared with the RT-DETR model, the parameter amount is reduced by 34.1%, and the precision and average precision value are improved by 1.5% and 3.3% respectively. BRIEF DESCRIPTION OF DRAWINGS
[0043] Figure 1 is a method flowchart of an embodiment of the application;
[0044] Figure 2 is a whole network structure diagram of an embodiment of the application;
[0045] Figure 3 is a part of data set schematic diagram in the application, wherein the (a) line is a shooting picture, the (b) line is a network collection picture, the (c) line is a part of picture in data set CCTSD, and the (d) line is a picture after data enhancement;
[0046] Figure 4 is a CADPCM module schematic diagram in an embodiment of the application;
[0047] Figure 5 is a structure diagram of CHAttention in an embodiment of the application;
[0048] Figure 6 is a structure diagram of DMPFCN in an embodiment of the application;
[0049] Figure 7 is a structure diagram of TDFM module in an embodiment of the application;
[0050] Figure 8 is an evaluation index visualization comparison diagram of CDWN and RT-DETR in an embodiment of the application;
[0051] Figure 9 is a loss visualization comparison diagram of CDWN and RT-DETR in an embodiment of the application;
[0052] Figure 10 is a detection effect diagram of the RT-DETR algorithm in an embodiment of the application in a live road;
[0053] Figure 11 is a detection effect diagram of the algorithm in a live road. DETAILED DESCRIPTION
[0054] The present invention will be further described below with reference to the embodiments. It should be noted that the following embodiments are only used to further illustrate the present invention and should not be construed as limiting the scope of protection of the present invention. Some non-essential improvements and adjustments made by those skilled in the art based on the above-described invention are still within the scope of protection of the present invention.
[0055] This invention proposes a traffic sign detection method for complex scenes based on dynamic frequency band focusing and dual-domain attention screening. The method includes acquiring the image or video stream to be identified, and inputting the image or video stream into the CDWN model. The specific steps include:
[0056] S1: Collect road traffic sign images as a traffic sign dataset and perform data preprocessing. Enhance some image data to simulate rain, fog, and snow environments to obtain an enhanced traffic sign dataset. The specific method is as follows:
[0057] S1.1 as Figure 3 As shown, different types of traffic signs were photographed in Huai'an City, Jiangsu Province, as research materials. Using the iPhone 14 Pro camera, traffic signs were photographed from different distances, locations, environments, and angles to ensure data diversity. A total of 2466 images were taken, with a pixel size of 4032×3024. Furthermore, to enhance the quality of the dataset and the stability of model training, traffic sign images under different lighting conditions, traffic scenarios, and environments were collected from Baidu, Sogou, Bing, and other online resources, combined with some images from the publicly available dataset CCTSDB, resulting in a total of 3534 images. These photographed images, online collected images, and images from CCTSDB, totaling 6000 images, were used as the traffic sign dataset D1.
[0058] S1.2 The images in the traffic sign dataset D1 are uniformly adjusted to 640×640 pixels using mean normalization to eliminate the size differences of the original dataset. Then, the LabelImg annotation tool is used to annotate these normalized images, classifying traffic signs into three categories: mandatory, prohibitory, and warning. After the annotation information is saved, the label dataset Label is obtained. Each label data contains five elements: category number, center point x-axis coordinate, center point y-axis coordinate, width, and height. Finally, the traffic sign preprocessing dataset D2 is obtained.
[0059] S1.3 Data augmentation is performed on part of the images in the traffic sign preprocessed dataset D2 using the Gaussian blur, brightness reduction, noise addition, rain addition, fog addition, snow addition, etc. methods in the imgaug library to simulate the complex environment that may be experienced during driving, such as blur, dim, foggy, rainy, snowy, etc. to enhance the robustness of the model in complex environments, and obtain the traffic sign augmented dataset D3.
[0060] S2: The CADPCM module is used to replace the BasicBlock module in the RT-DETR backbone network, and the CHAttention attention mechanism is added after it to construct the CACHNet backbone network, and the specific method is:
[0061] S2.1 The CACHNet backbone network is composed of four CADPCM modules, CHAttention attention mechanism, AIFI module and Conv in series, which can effectively focus on key features and suppress noise interference.
[0062] S2.2 As shown in Figure 4 , the CADPCM module is defined by a deep convolution, point convolution, batch normalization BN, ReLU activation function, PConv convolution, batch normalization BN, SiLU activation function and CAA attention mechanism in series to obtain the first branch. The first branch is connected in parallel with a ShortCut jump connection containing an AvgPool average pooling and a 1x1 convolution to obtain the second branch. Finally, the first branch and the second branch are added point by point, and the output feature map is obtained through the ReLU activation function.
[0063] S2.3 As shown in Figure 5 , the CHAttention attention mechanism first processes the input feature map C in the preprocessing stage. The input feature map C is first reduced to one fourth of the original channel number through a deep convolution operation, then nonlinearly transformed through a GELU activation function, then the channel number is restored through a 1x1 convolution and batch normalization to obtain the to-be-screened feature c1, and its formula is: c1 = BatchNorm(Conv(GELU(Conv(c))).
[0064] S2.4Secondly, the feature c1 in S2.3 is reduced to 1 in spatial dimension by an AdaptivaAvgPool adaptive average pooling operation, and then evaluated by two 1x1 convolution and SiLU activation function to obtain the importance weight of each channel, and finally the channel screen ca is obtained by Sigmoid function, forming a channel domain screening mechanism, and its formula is: ca=Sigmoid(Conv(SiLU(Conv(AdaptiveAvgPool(c1))))) which directly closes the redundant channels irrelevant to the semantic of traffic signs, such as actively filtering non-red channel responses when detecting prohibition signs.
[0065] S2.5Next, the feature c1 in S2.3 is captured by AvgPool average pooling to expand the receptive field and capture spatial context information, then two 1x1 convolution and two depth convolution are concatenated to extract spatial structure features, and finally the spatial screen sa is generated by Sigmoid activation function for normalization, and its formula is: sa=Sigmoid(Conv(DWConv(DWConv((Conv(AvgPool(c1))))))) which directly zeros the background interference area pixels in a hard filtering manner, effectively cleaning low confidence noise and retaining high probability sign regions in complex scenes such as rain and fog.
[0066] S2.6Finally, the channel screen ca and the spatial screen sa are fused by weighted average to form a joint screen, and the joint screen is multiplied by the feature c1 to realize feature purification, and its formula is: c2=c1xmean(ca+sa), which establishes a cross-verification mechanism in channel and spatial domains, and only when the feature passes through both channel semantic confirmation and spatial position verification can it be retained, forming a double-protected traffic sign feature purification system, which significantly improves the detection robustness in complex scenes.
[0067] The relevant steps of step S2 are introduced with specific examples:
[0068] The BasicBlock modules of the backbone of RT-DETR from the 4th layer to the 7th layer from the input end to the output end are replaced by CADPCM modules, and CHAttention attention mechanism is added after them to construct the CACHNet backbone network.
[0069] In the environment of snow, rain, fog and night, the traffic signs will be partially or completely blocked by snow, and the light will be scattered by water droplets in the rain and fog, resulting in unclear traffic sign outline and color recognition difficulty of traffic signs in the absence of natural light at night. BasicBlock is difficult to extract target information and prone to false detection by stacking standard convolution, while CADPCM module increases feature selection capability by combining partial convolution with CAA attention mechanism, effectively focusing on the key features of the target when it is blocked, and using depth separable convolution to reduce the number of parameters, with high environmental adaptability and detection effect. CHAttention attention mechanism strengthens color features through channel screen and captures geometric features through spatial screen, achieving good detection effect in complex environment.
[0070] S3: Constructing a DMPFCN neck network, combining a CACHNet backbone network and an RTDETRDecoder to construct a CDWN model, and using a traffic sign enhanced dataset to train to obtain the best weight bt, the specific method is:
[0071] Set P2 detection layer to extract the super high resolution feature map, P3 detection layer to extract the high resolution feature map, P4 detection layer to extract the medium resolution feature map, and P5 detection layer to extract the low resolution feature map.
[0072] S3.1 as shown in Figure 7 , define the TDFM module, first, use a 1x1 GSConv to change the output channel number of the feature map F1 to twice the input channel number, use MaxPool maximum pooling and AvgPool average pooling in parallel and add point by point to adjust the feature map F1 to the same size as the medium size input feature map F2, use a 1x1 GSConv to change the output channel number of the feature map F3 to half of the input channel number, use the nearest neighbor interpolation method to adjust the feature map F3 to the same size as the feature map F2, the feature map F2 is unchanged in size and channel number through a 1x1 convolution, finally, use the splicing ConCat method to horizontally splice the three feature maps F1, F2 and F3 after adjustment, get the feature map f1, the feature map f1 is halved in channel number through a 1x1 convolution operation, get the feature map f2, the feature map f2 is processed through a 1x1 GSConv and a 3x3 GSConv, and the features are extracted step by step to get the feature map f3, finally, the feature map f3 is spliced with the feature map f1 through the Concat operation and through a 1x1 convolution operation, the output feature map f4 is obtained.
[0073] S3.2 as shown in Figure 6As shown, the DMPFCN neck network is defined, a hierarchical residual structure is introduced, and the P2 detection layer is added to deepen the network level. Through multiple convolution modules, up-sampling operations and feature fusion modules, the input feature maps are gradually processed and enhanced. First, the low-resolution feature map output by the P5 detection layer is enlarged in spatial size by DySample dynamic up-sampling to generate a feature map q1. The medium-resolution feature map output by the P4 detection layer is extracted by GSConv to obtain a feature map q2. Then, the medium-resolution feature map output by the P4 detection layer, the feature map q1 and the feature map q2 are transmitted through the TDFM module to form a feature map q3. After GSConv, the feature map q3 generates a feature map q4, which continues to be transmitted to the bottom layer. The high-resolution feature map output by the P3 detection layer is extracted by GSConv to obtain a feature map q5. After DySample up-sampling, the feature map q4 is fused with the high-resolution feature map output by the P3 detection layer and the feature map q5 through the TFDM module to obtain a feature map q6. The feature map q6 and the super-high-resolution feature map output by the P2 detection layer are dynamically sampled and interacted to obtain a feature map q7. Finally, after processing of different scales, the feature map q7 is gradually processed through WaveletPool wavelet pooling to realize dynamic frequency band focusing processing. The input features are decomposed into low-frequency approximation components (LL), horizontal high-frequency details (LH), vertical high-frequency details (HL) and diagonal high-frequency details (HH) through two-dimensional discrete wavelet transform. Based on the traffic sign feature distribution, high-frequency enhancement weights and low-frequency suppression coefficients are dynamically calculated. After inverse wavelet transform reconstruction, spatial dimension compression and key feature preservation are realized. Finally, through Fusion feature fusion, the spatial dimension is further compressed and the key features are preserved to obtain output feature maps of different scales, providing multi-scale feature information for target detection.
[0074] S3.3 The CACHNet backbone network and the DMPFCN neck network are used to construct the CDWN model, and the traffic sign enhancement dataset D3 is divided into a training set, a validation set and a test set in a ratio of 7:2:1. The CDWN model is trained to obtain the best training weight bt. The validation set is verified using bt, and the model parameters are adjusted. The adjusted model parameters are: 100 training rounds, AdamW optimizer and 0.0001 learning rate.
[0075] The network parameters are shown in Table 1.
[0076] Table 1: Hierarchical architecture of CDWN model
[0077]
[0078]
[0079]
[0080] The relevant steps of step S3 are introduced with specific examples:
[0081] In the RT-DETR neck network part, the hierarchical residual structure and the P2 detection layer are introduced. The UpSample upsampling of the 12th layer, the 15th layer and the 18th layer of the input end to the output end of the neck network is replaced by DySample upsampling. The ordinary convolution of the 13th layer, the 16th layer and the 19th layer is replaced by GSConv. The DownSample downsampling of the 21st layer, the 23rd layer and the 25th layer is replaced by WaveletPool downsampling. The TDFM module is used in the 14th layer, the 17th layer and the 20th layer to splice and fuse the information of large, medium and small scales.
[0082] Although the PANet path aggregation network structure adds a bottom-up path aggregation network on the basis of the FPN structure, which to some extent improves the problem that the FPN easily loses bottom layer information, but when dealing with complex scenes, it will appear the problem of over-emphasizing high-frequency noise in shallow features. The DMPFCN neck network ensures the lossless transmission of deep features to shallow layers through the hierarchical residual structure, reduces the degradation of details, and uses the P2 small target detection layer to retain the detail information of the edge texture of the traffic sign. The DMPFCN neck network attenuates high-frequency noise information and enhances the preservation of low-frequency shape and color information by introducing WaveletPool wavelet pooling to decompose the shallow features in the frequency domain, effectively alleviating the problem that the traditional pooling loses a large amount of spatial information when reducing the size of the feature map, leading to inaccurate position and boundary information of the traffic sign. The introduction of GSConv reduces the amount of calculation while improving the detection speed, and better adapts to traffic signs of different scales. In order to solve the problem that traditional upsampling will cause image blurring and loss of detail information, the DMPFCN neck network uses Dysample dynamic upsampling to dynamically sample according to the image content, better preserves the key features and details of the traffic sign, and reduces information loss. The DMPFCN neck network uses the TDFM module to effectively fuse features of different scales, extracting more rich and discriminative features. This helps to more accurately depict the features of traffic signs in complex backgrounds, occlusions and other situations, and distinguish them from the background and other interference. In summary, DMPFCN improves the traditional static path aggregation to a noise-aware dynamic feature flow, through the triple optimization mechanism of frequency domain filtering, adaptive sampling and high-resolution perception, while retaining the positioning advantage of PANet, effectively solving the problem of amplification of shallow noise.
[0083] S4 defines the confidence threshold Conf, uses the best model weight bt to perform traffic sign recognition and bounding box regression on the CDWN model, outputs the detection accuracy and position information, and the specific method is:
[0084] S4.1 The operating system used in the experiment training is Ubuntu 22.04, the GPU is NVIDIA RTX 4090D, the memory is 24GB, the CPU is AMD EPYC 9754, the deep learning framework is PyTorch 2.3.0, the software programming environment is Cuda version 12.1, and the Python version is 3.12.
[0085] S4.2 In the experimental environment, load and initialize the CDWN model and use the best model weight bt, input the traffic sign image into the system, perform traffic sign detection, and generate a preliminary bounding box set Border for each candidate region. On the basis of the candidate region, perform bounding box regression processing to optimize the position parameters of the bounding box, generate the corresponding confidence score S, which is used for preliminary screening of the target. If S is greater than the confidence threshold Conf, proceed to the next step, otherwise ignore the candidate box.
[0086] S4.3 After generating the preliminary detection result, traverse the bounding box set Border, calculate the intersection over union IoU between each bounding box and other candidate boxes. If the IoU value of two candidate boxes is greater than the preset overlap threshold θ, further filter the overlapping candidate boxes through the non-maximum suppression NMS algorithm, retain the bounding box with the highest confidence, and discard the remaining redundant boxes. If the IoU value is less than θ, retain both bounding boxes.
[0087] S4.4 Output the final traffic sign detection result. After multiple filtering of the IoU threshold and the confidence threshold, a final traffic sign detection dataset T is generated, which contains all reliable detection results and provides the bounding box position information and class label of each traffic sign.
[0088] In order to better illustrate the application effect, the model in the example is evaluated. P, R, mAP@50, mAP@50-95, and Params are used to evaluate the performance of the model. P represents the precision; R represents the recall; mAP represents the mean average precision; mAP@50 refers to the average precision at an IoU threshold of 0.5; mAP@50-95 refers to the average precision at an IoU threshold between 0.5 and 0.95; and Params represents the parameter quantity of the model. Specifically:
[0089] Precision P:
[0090]
[0091] Recall R:
[0092]
[0093] Average precision mAP:
[0094]
[0095] Wherein, FP is the number of false positive samples, TP is the number of correctly predicted positive samples, FN is the number of false negative samples, AP is the area under the P-R curve of the current category, and n is the number of data categories.
[0096] As shown in Table 2, Table 2 is the experimental results of CHAttention attention mechanism, and the experiments show that compared with EMA, ELA and MLCA attention mechanisms, although the parameter amount of CHAttention is increased by 2.5% compared with RT-DETR, EMA and MLCA, other indicators are optimal.
[0097] Table 2 CHAttention attention mechanism experimental results
[0098] Params P R mAP@50 mAP@50-95 RT-DETR 19.9M 79.8% 66.5% 71.9% 44.0% RT-DETR+EMA 19.9M 78.4% 66.3% 71.6% 41.4% RT-DETR+ELA 21.7M 77.8% 64.0% 70.1% 40.8% RT-DETR+MLCA 19.9M 78.7% 65.2% 72.4% 42.7% RT-DETR+CHAttention 20.4M 83.8% 66.9% 72.6% 44.1%
[0099] In order to further evaluate the performance of CDWN, a comparative experiment is carried out, and the experimental results are shown in Table 3. From the table, it can be seen that the model of the application, compared with the RT-DETR model, the average precision is improved by 3.3%, the parameter amount is reduced by 34.1%, the precision is improved by 1.5%, and the recall rate is improved by 4.6%.
[0100] Table 3 Performance comparison of different models
[0101]
[0102]
[0103] It can be seen that the CDWN model is better than RT-DETR in precision, recall, mAP@50 and mAP@50-95. Figure 8 Figure 9 This paper demonstrates the trends of three key losses for the CDWN and RT-DETR models during training and validation rounds. Both models show a stable decrease in the three losses on the training set as training progresses, indicating that they are effectively learning from the training data. However, in terms of validation set loss, which represents the model's generalization ability, the CDWN model is significantly lower than the RT-DETR model in both val / giou_loss and val / cls_loss, and its loss curve decreases more steadily. CDWN also exhibits a lower val / l1_loss value. This clearly shows that CDWN is not only more accurate in predicting target location, target category, and other related tasks, but its learned patterns also have better generalization performance, with a consistently low validation loss and a significant advantage.
[0104] Depend on Figure 10 and Figure 11 It is known that when faced with complex environments such as snow, fog, and night, RT-DETR has difficulty accurately locating and detecting traffic signs in low visibility conditions, resulting in missed detections and false detections. The improved traffic sign detection CDWN model can effectively reduce the number of parameters while improving accuracy, and can provide high-precision traffic sign detection support for in-vehicle intelligent assisted driving systems.
[0105] Explanation of relevant terms or variables:
[0106]
[0107]
[0108] The above description merely illustrates preferred embodiments of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make modifications or alterations to the above-disclosed content to create equivalent embodiments. However, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention, without departing from the scope of the present invention, shall still fall within the protection scope of the present invention.
Claims
1. A method for traffic sign detection in complex scenes based on dynamic band focusing and dual-domain attention screening, characterized in that, The specific steps are as follows: S1: Collecting road traffic sign images as a traffic sign dataset and performing data preprocessing, enhancing part of the image data to simulate rain, fog and snow environment, and obtaining a traffic sign enhanced dataset; S2: Based on the RT-DETR backbone network, using the CADPCM module to replace the BasicBlock module in the RT-DETR backbone network, and adding the CHAttention attention mechanism after the CADPCM module, to construct the CACHNet backbone network; S3: Constructing a DMPFCN neck network, which introduces a hierarchical residual structure and a P2 detection layer to further fuse the shallow and deep features of the backbone network, and also introduces a WaveletPool wavelet pooling to perform frequency domain decomposition on the shallow features, and uses a TDFM module to effectively fuse features of different scales; The DMPFCN neck network combines the CACHNet backbone network and the RTDETRDecoder to construct a CDWN model, and uses the traffic sign enhanced dataset to train the best weight bt; S4: Define the confidence threshold Conf, use the best model weight bt to perform traffic sign recognition and bounding box regression on the CDWN model, and output the detection accuracy and position information.
2. The method for complex scene traffic sign detection based on dynamic band focusing and dual-domain attention screening according to claim 1, characterized in that, The S1 is specifically implemented as follows: S1.1 Collect road traffic sign images, including images in different environments such as night, fog, snow, rain, occlusion and low light, and combine the images in the public dataset and part of the images collected by the network as a traffic sign dataset D1; S1.2 Adjust the image size to the same size using mean normalization on the traffic sign dataset D1, and use the LabelImg labeling tool to label the normalized traffic sign images, mark the position and category of each traffic sign, and obtain a traffic sign preprocessing dataset D2; S1.3 Use the Gaussian blur, brightness reduction, noise addition, rain addition, fog addition, and snow addition in the imgaug library to enhance part of the images in the traffic sign preprocessing dataset D2, and obtain a traffic sign enhanced dataset D3.
3. The method for complex scene traffic sign detection based on dynamic band focusing and dual-domain attention screening according to claim 1, characterized in that, S2 is specifically implemented as follows: The CACHNet backbone network is composed of four layers of CADPCM modules, CHAttention attention mechanism, AIFI module and Conv in series; The input feature map is input into the four-layer CADPCM module in series after Conv processing for feature extraction, and the output feature of the uppermost CADPCM module is input into the AIFI module after CHAttention attention mechanism; The first three layers of CADPCM modules are used as P2, P3 and P4 detection layers respectively, and the feature map processed by the AIFI module is used as the output feature map of the P5 detection layer.
4. The method for complex scene traffic sign detection based on dynamic band focusing and dual-domain attention screening according to claim 3, characterized in that, The CADPCM module is obtained by a deep convolution, a point convolution, a batch normalization BN, a ReLU activation function, a PConv convolution, a batch normalization BN, a SiLU activation function, and a CAA attention mechanism in series to obtain a first branch, the first branch is connected in parallel with a ShortCut jump connection containing an AvgPool average pooling and a 1x1 convolution to obtain a second branch, and finally the first branch and the second branch are added point by point, and the output feature map is obtained through a ReLU activation function.
5. The method for complex scene traffic sign detection based on dynamic band focusing and dual-domain attention screening according to claim 3, characterized in that, The CHAttention attention mechanism processing process is: First, the input feature map C is processed in the preprocessing stage. The input feature map C is first reduced to one fourth of the original channel number through a deep convolution operation, then nonlinearly transformed through a GELU activation function, then the channel number is restored through a 1x1 convolution, and batch normalization is performed to obtain a feature map c1; Secondly, the feature map c1 is reduced to 1 in spatial dimension through an AdaptiveAvgPool adaptive average pooling operation, then two 1x1 convolutions and a SiLU activation function are used to generate a channel attention factor, and a Sigmoid activation function is used for normalization to obtain the final channel attention factor ca; Then, the feature map c1 is expanded through an AvgPool average pooling, then two 1x1 convolutions and two deep convolutions are connected in series to extract spatial information, and finally a Sigmoid activation function is used for normalization to obtain the final spatial attention factor sa; Finally, the channel attention factor ca and the spatial attention factor sa are weighted and averaged, and then multiplied by the input feature map c1 to obtain the weighted output feature map c2.
6. The method for complex scene traffic sign detection based on dynamic band focusing and dual-domain attention screening according to claim 3, characterized in that, The DMPFCN neck network is implemented as follows: Set the ultra-high resolution feature map extracted by the P2 detection layer, the high resolution feature map extracted by the P3 detection layer, the medium resolution feature map extracted by the P4 detection layer, and the low resolution feature map extracted by the P5 detection layer; The DMPFCN neck network introduces a hierarchical residual structure and increases a P2 detection layer to deepen the network level. Through multiple convolution modules, up-sampling operations and feature fusion modules, the input feature maps are gradually processed and enhanced. First, the low-resolution feature map output by the P5 detection layer is enlarged in spatial size by DySample dynamic up-sampling to generate a feature map q1. The medium-resolution feature map output by the P4 detection layer is extracted by GSConv to obtain a feature map q2. Then, the medium-resolution feature map output by the P4 detection layer, the feature map q1 and the feature map q2 are transmitted by the TDFM module to form a feature map q3. After the GSConv operation, the feature map q3 generates a feature map q4, which is further transmitted to the bottom layer. The high-resolution feature map output by the P3 detection layer is extracted by GSConv to obtain a feature map q5. After the DySample up-sampling operation, the feature map q4 is fused with the high-resolution feature map output by the P3 detection layer and the feature map q5 by the TFDM module to obtain a feature map q6. The feature map q6 and the super-high-resolution feature map output by the P2 detection layer are dynamically sampled and interacted to obtain a feature map q7. Finally, after the processing of different scales, the feature map q7 is gradually down-sampled by the WaveletPool wavelet pooling and the Fusion feature fusion to further compress the spatial dimension and keep the key features. Finally, different scale output feature maps are obtained to provide multi-scale feature information for target detection.
7. The method for complex scene traffic sign detection based on dynamic band focusing and dual-domain attention screening according to claim 6, characterized in that, The input feature maps of the TDFM module are three, specifically a large-size input feature map F1, a medium-size input feature map F2 and a small-size input feature map F3. The specific execution process is as follows: First, a 1x1 GSConv is used on the large-size input feature map F1 to double the output channel number of the feature map F1, and MaxPool and AvgPool are used in parallel to adjust the feature map F1 to the same size as the medium-size input feature map F2 by point-by-point addition. A 1x1 GSConv is used on the small-size input feature map F3 to halve the output channel number of the feature map F3, and a nearest neighbor interpolation method is used to adjust the feature map F3 to the same size as the feature map F2. The feature map F2 is processed by a 1x1 convolution to keep its size and channel number unchanged, and finally the three adjusted feature maps F1, F2 and F3 are horizontally spliced by the ConCat method to obtain a feature map f1. The feature map f1 is processed by a 1x1 convolution operation to halve the channel number to obtain a feature map f2. The feature map f2 is processed by a 1x1 GSConv and a 3x3 GSConv to gradually extract features to obtain a feature map f3. Finally, the feature map f3 is spliced with the feature map f1 by the Concat operation and processed by a 1x1 convolution operation to obtain an output feature map f4.
8. The method for complex scene traffic sign detection based on dynamic band focusing and dual-domain attention filtering according to claim 1, characterized in that, The CACHNet backbone network and the DMPFCN neck network and the RTDETRDecoder jointly construct a CDWN model, and a traffic sign augmented dataset is divided into a training set, a validation set and a test set in a ratio of 7:2:1, and the CDWN model is trained to obtain the best training weight bt, the divided validation set is verified using the bt, and the model parameters are adjusted again, and the adjusted model parameters are: 100 training rounds, the optimizer is AdamW, and the learning rate is 0.0001.
9. The method for complex scene traffic sign detection based on dynamic band focusing and dual-domain attention filtering according to claim 1, characterized in that, S4 is specifically implemented as follows: S4.1 loads and initializes the CDWN model and uses the best model weight bt, inputs the traffic sign image into the system, performs traffic sign detection, and generates a preliminary border box set Border for each candidate region, performs border box regression processing on the basis of the candidate region, optimizes the position parameters of the border box, and generates a corresponding confidence score S for preliminary screening of the target, if S is greater than a confidence threshold value Conf, the next step is entered, otherwise the candidate box is ignored; S4.2 after generating the preliminary detection result, the border box set Border is traversed, the intersection over union IoU between each border box and other candidate boxes is calculated, if the IoU value of two candidate boxes is greater than a preset overlap threshold value θ, the non-maximum suppression NMS algorithm is used to further screen the overlapping candidate boxes, the border box with the highest confidence is retained, and the remaining redundant boxes are discarded, if the IoU value is less than θ, the two border boxes are retained; S4.3 outputs the final traffic sign detection result, after multi-layer screening of the IoU threshold and the confidence threshold, a final traffic sign detection dataset T is generated, which contains all reliable detection results and provides the border box position information and the category label of each traffic sign.
10. A complex scene traffic sign detection system based on dynamic band focusing and dual domain attention screening, comprising a memory, a processor and a computer program, characterized in that, The computer program loaded into the processor and run can realize the complex scene traffic sign detection method based on dynamic frequency band focusing and double-domain attention screening of any one of claims 1-9.
Citation Information
Cited By
Target detection method and device in wide-area complex scene, equipment and storage medium
CN121582734A
Target detection methods, devices, equipment and storage media in wide-area complex scenarios
CN121582734B
Traffic scene target detection method and device based on improved RT-DETR and storage medium
CN121982677A
Traffic scene target detection method and device based on improved RT-DETR and storage medium
CN121982677B
OCT image adaptive contrast enhancement method
CN121998886A
