Traffic sign detection method based on spatial completion and multi-scale focusing

Abstract

This invention discloses a traffic sign detection method based on spatial completion and multi-scale focusing, comprising: performing category balancing enhancement on the traffic sign dataset to construct a balanced dataset; embedding a feature complementarity mapping module in the Q4 layer of the YOLOv12 backbone network, fusing color and spatial information through channel segmentation and bidirectional attention mechanism to complete the features of small targets; embedding multi-scale convolutional attention modules in the P3, P4 and P5 layers of the detection head respectively, using differentiated strip convolution kernels and color attention mechanism to achieve feature focusing of traffic signs at multiple scales; using a composite loss function combining SIoU loss and shape matching loss for network training optimization; performing color enhancement, multi-scale feature extraction and non-maximum suppression algorithm post-processing in the inference stage to output the traffic sign category, bounding box and confidence score; this invention significantly improves the accuracy of small target detection and multi-scale adaptability, meeting the real-time detection needs in complex traffic scenarios.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent transportation technology, specifically a traffic sign detection method based on spatial completion and multi-scale focusing. Background Technology

[0002] Traffic signs serve as the core environmental perception input for intelligent driving and autonomous driving perception systems. Their detection accuracy and real-time performance directly determine the reliability of critical decisions such as vehicle path planning, speed limit control, and hazard warning. With the rapid development of vehicle vision technology and deep learning, object detection algorithms have evolved from traditional methods based on shape matching and color threshold segmentation to end-to-end detection solutions based on convolutional neural networks. Among these, the YOLO series models have become the mainstream technology choice for traffic sign detection due to their high accuracy and high real-time performance.

[0003] While existing traffic sign detection methods based on the YOLO series perform well in general object detection tasks, traffic sign detection is unique in that it involves large differences in target scale, specific characteristics of color and shape, complex and varied application scenarios, and limited computing power for engineering deployment. This leads to several unresolved core shortcomings in existing technologies: First, the spatial location information of small-scale traffic signs is irreversibly lost during backbone network downsampling, resulting in low feature dimensionality and susceptibility to background noise; multi-scale fusion methods lack targeted enhancement, leading to high false negative rates and large positioning errors for small targets. Second, the specific features of traffic signs are not fully utilized; the color constancy and regular shape priors of traffic signs are not fully exploited: RGB space processing causes color features to fail under extreme lighting conditions, and shape information is not integrated into the feature extraction and regression process; the color-shape feature fusion method is simple, resulting in low feature utilization and high false positive rates in complex backgrounds. Third, the adaptability and anti-interference capabilities in complex scenes are insufficient; they struggle in backlighting and rainy nights. The system suffers from several challenges: 1) In extreme nighttime conditions, there is a lack of dedicated feature enhancement mechanisms; 2) In occluded scenes, there is no effective feature recovery strategy, and the false detection rate increases significantly when the occlusion rate is greater than 30%; 3) It is difficult to suppress interference from similar backgrounds such as billboards and light boxes; 4) Insufficient training generalization ability; The traffic sign dataset exhibits a long-tailed distribution of categories, and rare categories are prone to overfitting; The pre-trained weights in the general dataset have poor adaptability, resulting in slow convergence of transfer learning; 5) Lack of scene adaptation mechanisms, leading to a significant decrease in transfer accuracy between vehicle-mounted and aerial photography scenes; 6) Difficulty in balancing engineering deployment; Complex improvement modules lead to a surge in the number of parameters and computational load, resulting in detection speeds lower than the real-time requirements of vehicle-mounted systems; Lightweight solutions are prone to losing small target features, leading to an increased false detection rate; 7) Lack of optimization for embedded device hardware architecture, resulting in high inference latency and difficulty in implementation; 8) Poor adaptability of bounding box regression; Failure to utilize the regular shape prior of traffic signs leads to large deviations between axis-aligned bounding box regression and the actual contour; Traditional loss functions are insufficient for optimizing shape consistency, resulting in low accuracy in small target bounding box regression.

[0004] In summary, existing methods fail to fully meet the specific needs of traffic sign detection, exhibiting multiple shortcomings in areas such as small target recognition, interference resistance in complex scenes, training generalization, and engineering deployment, necessitating targeted improvement solutions. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a traffic sign detection method based on spatial completion and multi-scale focusing, which aims to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a traffic sign detection method based on spatial completion and multi-scale focusing, comprising the following steps:

[0007] Step S1: Obtain the original traffic sign dataset, perform statistical analysis and augmentation on the original traffic sign dataset to obtain a class-balanced augmented dataset;

[0008] Step S2: Construct an improved YOLOv12 detection network. The improved YOLOv12 detection network includes a backbone network, a feature fusion network, and a detection head. A feature complementation mapping module is embedded in the Q4 feature layer of the backbone network to complete the color and spatial information of small targets on traffic signs. Multi-scale convolutional attention modules are embedded in the P3, P4, and P5 feature layers of the detection head to perform multi-scale feature focusing on traffic signs of different scales.

[0009] Step S3: Train the improved YOLOv12 detection network using a class-balanced augmented dataset, and optimize the improved YOLOv12 detection network using a composite loss function;

[0010] Step S4: After preprocessing the image to be detected, input it into the trained and optimized improved YOLOv12 detection network. The network is processed through the backbone network, feature complement mapping module, feature fusion network and multi-scale convolutional attention module to output the detection results of traffic signs.

[0011] Furthermore, step S1 specifically includes:

[0012] Step S11: Obtain the original traffic sign dataset and annotation file from the public dataset. The original traffic sign dataset contains images and corresponding traffic sign category labels and bounding box information.

[0013] Step S12: Read the annotation files in the original traffic sign dataset, count the number of annotation boxes for each type of traffic sign, and identify the categories with an annotation count lower than a preset threshold as the categories to be enhanced.

[0014] Step S13: Set the target number of labels for each type of traffic sign to ensure that the number of samples for each category in the original traffic sign dataset reaches the preset balance condition; set the enhancement operation parameters, including the scaling ratio range, rotation angle range, blur intensity, and random range of pasting position.

[0015] Step S14: Enhance the category to be enhanced to obtain the enhanced annotation information data;

[0016] Step S15: Merge the original traffic sign dataset with the augmented label information data to form a class-balanced augmented dataset, ensuring that the total number of labels for each type of traffic sign is not less than the preset number; randomly divide the augmented dataset into training set, validation set and test set according to the preset ratio, and ensure that the class distribution in each subset remains balanced after the division.

[0017] Further, in step S14, the category to be enhanced is enhanced to obtain the enhanced annotation information data; specifically:

[0018] Step S141: Extract traffic sign regions from the original image of the category to be enhanced;

[0019] Step S142: Randomly scale, rotate, and blur the extracted traffic sign area;

[0020] Step S143: Randomly paste the processed traffic sign area into a background image that does not contain signs of the category to be enhanced, and ensure that its spatial position is reasonable for the traffic scene;

[0021] Step S144: Generate new annotation information data, including updated bounding box coordinates and category labels.

[0022] Furthermore, in step S2, a feature complementation mapping module is embedded in the Q4 feature layer of the backbone network to complete the color and spatial information of the small targets of traffic signs; specifically:

[0023] Step S211: Extract the first feature map output from the Q4 feature layer of the backbone network. Where C=512, H=80, W=80, R represents the set of real numbers in mathematics, and the first characteristic graph The value at each position in the graph is a real number, and C represents the first feature map. The channel dimension, H represents the first feature map. The spatial height, W represents the first feature map. Spatial width;

[0024] Step S212, transfer the first feature map Scaled along the channel dimension The code is divided into two parts, resulting in a spatial-color branch and a semantic branch, calculated using the following formula:

[0025] ;

[0026] in, The representation space-color branch is used to preserve the pixel coordinates, edge contours, and color distribution of traffic signs. This indicates a semantic branch, used to extract the shape semantic information of traffic signs. This indicates a splitting operation. This represents the scaling factor for channel segmentation in the complementary feature mapping module;

[0027] Step S213: Perform feature transformations on the space-color branch and the semantic branch respectively.

[0028] Applying a 1×1 convolution to the spatial-color branch adjusts the channel dimensions while preserving the spatial structure, yielding spatial-color features. The calculation formula is as follows:

[0029] ;

[0030] in, Represents spatial-color characteristics, Represents a 1×1 convolution;

[0031] The semantic branch is processed using a standard 3×3 convolution to extract shape semantic features. The calculation formula is as follows:

[0032] ;

[0033] in, Represents semantic features, This represents a 3×3 standard convolution. This indicates that the convolution kernel is in the first feature map. The step size of the upward slide. This indicates that before the convolution operation, the first feature map is given... Fill the edges with 0-value pixels;

[0034] Step S214: Achieve bidirectional feature complementarity through an attention mechanism:

[0035] Semantic features are processed using depthwise convolution to remove redundant channel associations. Global average pooling is then used to capture global information of typical color channels in traffic signs. Finally, channel attention weights are generated by activating the sigmoid function. The calculation formula is as follows:

[0036] ; ;

[0037] in, This represents the intermediate feature map obtained after semantic features are processed by a 3×3 depthwise convolution. Represents depthwise convolution. Indicates channel attention weights. This represents the Sigmoid function. This represents the pixel position index in the vertical direction of the intermediate feature map, with a value range of 0 ≤ i. <H, This represents the pixel position index in the horizontal direction of the intermediate feature map, with a value range of 0 ≤ j. <W;

[0038] The spatial-color features are processed using 1×1 convolution with batch normalization and a sigmoid function to focus on the spatial region where the traffic sign is located, generating spatial attention weights. The calculation formula is as follows:

[0039] ;

[0040] in, Represents spatial attention weights. This indicates a batch normalization operation;

[0041] The channel attention weights are weighted onto the spatial-color features to obtain the spatial-location feature map. The spatial attention weights are then weighted onto the semantic features to obtain the semantic feature map. The calculation formula is as follows:

[0042] ; ;

[0043] in, This represents the spatial-location feature map after channel attention weighting. This represents element-wise multiplication. This represents the semantic feature map after spatial attention weighting;

[0044] Step S215: Concatenate the supplemented spatial-location feature map and semantic feature map along the channel dimension, and then combine them with the first feature map. Perform residual connections to obtain the enhanced second feature map. The calculation formula is:

[0045] ;

[0046] in, This represents the enhanced second feature map. This indicates channel splicing, integrating bidirectional complementary features. This indicates a residual connection.

[0047] Furthermore, in step S2, multi-scale convolutional attention modules are embedded in the P3, P4, and P5 feature layers of the detection head to perform multi-scale feature focusing on traffic signs of different scales; specifically:

[0048] Step S221: Obtain the P3, P4, and P5 feature layers from the pre-processing or feature fusion network, respectively, which correspond to traffic sign feature representations at different spatial resolutions, and then process the second feature map. As input to the multi-scale convolutional attention module;

[0049] Step S222, for the second feature map Local information aggregation is performed using 3×3 depthwise convolution to extract local edge and corner features, resulting in a local information aggregation feature map. The calculation formula is as follows:

[0050] ;

[0051] in, This represents a feature map representing the aggregation of local information.

[0052] Step S223: Based on the typical scale of traffic signs corresponding to the P3, P4, and P5 feature layers of the detection head, parallel feature extraction is performed on the local information aggregation feature map using different combinations of strip convolution kernels:

[0053] For small-scale markers, the P3 feature layer uses a 3×1+1×3+5×1+1×5 convolution to capture the digital contours of the markers, resulting in a small-scale feature map. The calculation formula is as follows:

[0054] ;

[0055] in, This represents a small-scale feature map of the P3 feature layer. This represents a 3×1 convolution. This represents a 1×3 convolution. This represents a 5×1 convolution. This represents a 1×5 convolution;

[0056] For the mesoscale feature layer P4, a 7×1+1×7+11×1+1×11 convolutional kernel is used to adapt to triangular or circular shapes, resulting in a mesoscale feature map. The calculation formula is as follows:

[0057] ;

[0058] in, This represents the mesoscale feature map of the P4 feature layer. This represents a 7×1 convolution. This represents a 1×7 convolution. This represents an 11×1 convolution. This represents a 1×11 convolution;

[0059] For the large-scale marker P5 feature layer, a convolution kernel of 11×1+1×11+21×1+1×21 is used to cover the global features of the rectangular marker, resulting in a large-scale feature map. The calculation formula is as follows:

[0060] ;

[0061] in, This represents the large-scale feature map of the P5 feature layer. This represents a 21×1 convolution. Represents a 1×21 convolution;

[0062] Step S224: Process the small-scale feature maps of each layer. Mesoscale feature map Large-scale feature maps A 1×1 convolution is used to compress the channels to obtain a multi-scale fused feature map. The calculation formula is as follows:

[0063] ;

[0064] in, This represents the multi-scale fused feature map of the multi-scale convolutional attention module. Representing small-scale feature maps of each layer Mesoscale feature map Large-scale feature maps n=3, 4, 5;

[0065] Extracting multi-scale fused feature maps The high-dimensional aggregated feature map of the channels is used to generate color weights, and the calculation formula is as follows:

[0066] ;

[0067] in, Indicates color weight, A high-dimensional aggregated feature map representing the channels of a multi-scale convolutional attention module;

[0068] Fusion of color weights with multi-scale feature maps Element-wise multiplication yields the color-enhanced attention features, calculated using the following formula:

[0069] ;

[0070] in, This indicates the attentional features after color enhancement;

[0071] Step S225: Aggregate the color-enhanced attention features with the local information feature map. The feature map is fused and focused to obtain the final output enhanced feature map. The calculation formula is as follows:

[0072] ;

[0073] in, This represents the final output augmented feature map of the multi-scale convolutional attention module.

[0074] Furthermore, step S3 specifically includes:

[0075] Step S31: Load the pre-trained weights of the improved YOLOv12 backbone network, and initialize the network parameters of the embedded feature complementarity mapping module and multi-scale convolutional attention module using the uniform distribution method Xavier. The network parameter initialization formula is as follows:

[0076] ;

[0077] in, This represents the set of learnable parameters for the feature complementarity mapping module. Indicates uniform distribution. Indicates the number of input channels. Indicates the number of output channels;

[0078] Step S32: Using the stochastic gradient descent optimizer and the cosine annealing learning rate adjustment method, the dynamic learning rate for the current iteration step is obtained, as shown in the following formula:

[0079] ;

[0080] in, This represents the dynamic learning rate at the current iteration step. This represents the initial learning rate. Let T represent the cosine trigonometric function, t represent the current training iteration number, and T=160000 represent the total number of iterations. Represents pi;

[0081] Step S33: The class-balanced augmented dataset is input into the improved YOLOv12 detection network in batches. The improved YOLOv12 detection network performs forward inference on each batch of input data and calculates the composite loss function, which includes:

[0082] The SIoU loss function is used to optimize the position and shape regression of the bounding box, and its formula is:

[0083] ;

[0084] in, Represents the SIoU loss function. This represents the intersection-over-union ratio (IoU), which measures the degree of overlap between the predicted bounding box and the ground truth bounding box. Indicates distance loss. , This represents the normalized distance between the predicted bounding box and the ground truth bounding box. Indicates the center coordinates in the horizontal direction. Indicates the center coordinates in the vertical direction. Indicates shape loss, , =4, Indicates the horizontal width of the bounding box. Indicates the vertical height of the bounding box. The penalty coefficient representing shape loss;

[0085] A shape matching loss function based on image moments (Hu) is used to enhance the perception of regular shapes of traffic signs. The formula is as follows:

[0086] ;

[0087] in, This represents the shape matching loss function. This represents the natural exponential function. This represents the adjusted loss weight, k=5. This represents the image moment distance (Hu) between the predicted bounding box and the traffic sign shape template.

[0088] The composite loss function is a weighted sum of the SIoU loss function and the shape matching loss function, and the formula is:

[0089] ;

[0090] in, This represents the composite loss function.

[0091] Furthermore, step S4 specifically includes:

[0092] Step S41: Obtain the traffic scene image to be detected, and perform RGB to HSV space conversion, channel high-dimensional aggregation feature map filtering, contrast-limited adaptive histogram equalization CLAHE brightness enhancement, and 640×640 scaling operation on the traffic scene image to be detected to obtain the preprocessed image.

[0093] Step S42: Input the preprocessed image into the trained and optimized improved YOLOv12 detection network, and process it through the backbone network, feature complement mapping module, feature fusion network and multi-scale convolutional attention module to obtain enhanced multi-scale features;

[0094] Step S43: Input the enhanced multi-scale features into the detection head for processing to obtain the output features of the detection head, and map and decode the output features of the detection head into preliminary detection results;

[0095] Step S44: The preliminary detection results are processed using an improved non-maximum suppression method to suppress overlapping bounding boxes. This is combined with color-weighted correction of predicted boxes with confidence scores below 0.5, yielding the corrected target confidence score. The formula is as follows:

[0096] ;

[0097] in, This represents the corrected target confidence level. Indicates the confidence level of the original target;

[0098] Step S45: Perform final filtering based on the corrected target confidence level and the preset threshold, and output the final detection results. The final detection results include the following for each traffic sign: category name or number, precise bounding box coordinates, and corrected confidence level.

[0099] Compared with the prior art, the present invention has the following beneficial effects:

[0100] This invention addresses the problem of low detection and recognition rates in traffic sign datasets trained on current datasets due to a lack of certain sign numbers. It employs data augmentation methods and implements dataset label balancing through code to compensate for the insufficient labeling of certain signs.

[0101] This invention effectively solves the problem of spatial information loss in downsampling of small-scale traffic signs by embedding a complementary feature mapping module in the Q4 feature layer of the YOLOv12 backbone network. The complementary feature mapping module, through a four-step process of channel segmentation, orientation transformation, complementary mapping, and feature aggregation, can better preserve the color and spatial features of traffic signs, especially providing a significant improvement in detection accuracy for traffic signs with a diameter of less than 30 pixels.

[0102] The multi-scale convolutional attention module of this invention enhances the feature response of traffic signs at different scales through multi-branch convolution and color attention mechanisms. In particular, for traffic signs of different sizes, such as small speed limit signs, intersection warning signs and directional signs at a distance of 50 meters, the multi-scale convolutional attention module solves the problems of weak feature response and severe background interference of multi-scale signs by designing differentiated convolution kernel combinations, thus significantly improving detection accuracy.

[0103] This invention improves the composite loss function of YOLOv12 by combining SIoU loss and shape matching loss to enhance the regression accuracy of traffic sign bounding boxes. In particular, it introduces image moment Hu distance as a shape matching loss, enabling YOLOv12 to more accurately fit the regular shape of traffic signs, avoiding detection errors caused by shape misjudgment, thereby improving detection accuracy in complex scenes.

[0104] By loading pre-trained weights specific to traffic signs and combining Xavier initialization and stochastic gradient descent optimizer SGD with a cosine annealing learning rate strategy, this invention significantly improves training efficiency and inference speed. Simultaneously, it employs improved non-maximum suppression Soft-NMS post-processing and color weight correction for low-confidence predictions, ensuring detection accuracy while meeting real-time requirements. In vehicle applications, the system maintains a detection speed above 35 FPS, suitable for vehicle-mounted and low-altitude aerial traffic sign detection needs. Attached Figure Description

[0105] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0106] Referring to the following description and accompanying drawings, some specific embodiments of the present invention are disclosed to illustrate some ways of implementing the principles of the embodiments of the present invention. However, it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0107] like Figure 1 As shown, this invention proposes the following technical solution: a traffic sign detection method based on spatial completion and multi-scale focusing, comprising the following steps:

[0108] Step S1: Obtain the original traffic sign dataset, perform statistical analysis and augmentation on the original traffic sign dataset to obtain a class-balanced augmented dataset;

[0109] Step S2: Construct an improved YOLOv12 detection network. The improved YOLOv12 detection network includes a backbone network, a feature fusion network, and a detection head. A feature complementation mapping module is embedded in the Q4 feature layer of the backbone network to complete the color and spatial information of small targets on traffic signs. Multi-scale convolutional attention modules are embedded in the P3, P4, and P5 feature layers of the detection head to perform multi-scale feature focusing on traffic signs of different scales.

[0110] Step S3: Train the improved YOLOv12 detection network using a class-balanced augmented dataset, and optimize the improved YOLOv12 detection network using a composite loss function;

[0111] Step S4: After preprocessing the image to be detected, input it into the trained and optimized improved YOLOv12 detection network. The network is processed through the backbone network, feature complement mapping module, feature fusion network and multi-scale convolutional attention module to output the detection results of traffic signs.

[0112] Furthermore, step S1 specifically includes:

[0113] Step S11: Obtain the original traffic sign dataset and annotation file from the public dataset. The original traffic sign dataset contains images and corresponding traffic sign category labels and bounding box information.

[0114] Step S12: Read the annotation files in the original traffic sign dataset, count the number of annotation boxes for each type of traffic sign, and identify the categories with an annotation count lower than a preset threshold as the categories to be enhanced.

[0115] Step S13: Set the target number of labels for each type of traffic sign to ensure that the number of samples for each category in the original traffic sign dataset reaches the preset balance condition; set the enhancement operation parameters, including the scaling ratio range, rotation angle range, blur intensity, and random range of pasting position.

[0116] Step S14: Enhance the category to be enhanced to obtain the enhanced annotation information data;

[0117] Step S15: Merge the original traffic sign dataset with the augmented label information data to form a class-balanced augmented dataset, ensuring that the total number of labels for each type of traffic sign is not less than the preset number; randomly divide the augmented dataset into training set, validation set and test set according to the preset ratio, and ensure that the class distribution in each subset remains balanced after the division.

[0118] Further, in step S14, the category to be enhanced is enhanced to obtain the enhanced annotation information data; specifically:

[0119] Step S141: Extract traffic sign regions from the original image of the category to be enhanced;

[0120] Step S142: Randomly scale, rotate, and blur the extracted traffic sign area;

[0121] Step S143: Randomly paste the processed traffic sign area into a background image that does not contain signs of the category to be enhanced, and ensure that its spatial position is reasonable for the traffic scene;

[0122] Step S144: Generate new annotation information data, including updated bounding box coordinates and category labels.

[0123] Furthermore, in step S2, a feature complementation mapping module is embedded in the Q4 feature layer of the backbone network to complete the color and spatial information of the small targets of traffic signs; specifically:

[0124] Step S211: Extract the first feature map output from the Q4 feature layer of the backbone network. Where C=512, H=80, W=80, R represents the set of real numbers in mathematics, and the first characteristic graph The value at each position in the graph is a real number, and C represents the first feature map. The channel dimension, H represents the first feature map. The spatial height, W represents the first feature map. Spatial width;

[0125] Step S212, transfer the first feature map Scaled along the channel dimension The code is divided into two parts, resulting in a spatial-color branch and a semantic branch, calculated using the following formula:

[0126] ;

[0127] in, The representation space-color branch is used to preserve the pixel coordinates, edge contours, and color distribution of traffic signs. This indicates a semantic branch, used to extract the shape semantic information of traffic signs. This indicates a splitting operation. This represents the scaling factor for channel segmentation in the complementary feature mapping module;

[0128] Step S213: Perform feature transformations on the space-color branch and the semantic branch respectively.

[0129] Applying a 1×1 convolution to the spatial-color branch adjusts the channel dimensions while preserving the spatial structure, yielding spatial-color features. The calculation formula is as follows:

[0130] ;

[0131] in, Represents spatial-color characteristics, Represents a 1×1 convolution;

[0132] The semantic branch is processed using a standard 3×3 convolution to extract shape semantic features. The calculation formula is as follows:

[0133] ;

[0134] in, Represents semantic features, This represents a 3×3 standard convolution. This indicates that the convolution kernel is in the first feature map. The step size of the upward slide. This indicates that before the convolution operation, the first feature map is given... Fill the edges with 0-value pixels;

[0135] Step S214: Achieve bidirectional feature complementarity through an attention mechanism:

[0136] Semantic features are processed using depthwise convolution to remove redundant channel associations. Global average pooling is then used to capture global information of typical color channels in traffic signs. Finally, channel attention weights are generated by activating the sigmoid function. The calculation formula is as follows:

[0137] ; ;

[0138] in, This represents the intermediate feature map obtained after semantic features are processed by a 3×3 depthwise convolution. Represents depthwise convolution. Indicates channel attention weights. This represents the Sigmoid function. This represents the pixel position index in the vertical direction of the intermediate feature map, with a value range of 0 ≤ i. <H, This represents the pixel position index in the horizontal direction of the intermediate feature map, with a value range of 0 ≤ j. <W;

[0139] The spatial-color features are processed using 1×1 convolution with batch normalization and a sigmoid function to focus on the spatial region where the traffic sign is located, generating spatial attention weights. The calculation formula is as follows:

[0140] ;

[0141] in, Represents spatial attention weights. This indicates a batch normalization operation;

[0142] The channel attention weights are weighted onto the spatial-color features to obtain the spatial-location feature map. The spatial attention weights are then weighted onto the semantic features to obtain the semantic feature map. The calculation formula is as follows:

[0143] ; ;

[0144] in, This represents the spatial-location feature map after channel attention weighting. This represents element-wise multiplication. This represents the semantic feature map after spatial attention weighting;

[0145] Step S215: Concatenate the supplemented spatial-location feature map and semantic feature map along the channel dimension, and then combine them with the first feature map. Perform residual connections to obtain the enhanced second feature map. The calculation formula is:

[0146] ;

[0147] in, This represents the enhanced second feature map. This indicates channel splicing, integrating bidirectional complementary features. This represents a residual connection, preserving the original characteristics of the backbone network and avoiding the loss of traffic sign details.

[0148] Furthermore, in step S2, multi-scale convolutional attention modules are embedded in the P3, P4, and P5 feature layers of the detection head to perform multi-scale feature focusing on traffic signs of different scales; specifically:

[0149] Step S221: Obtain the P3, P4, and P5 feature layers from the pre-processing or feature fusion network, respectively, which correspond to traffic sign feature representations at different spatial resolutions, and then process the second feature map. As input to the multi-scale convolutional attention module;

[0150] Step S222, for the second feature map Local information aggregation is performed using 3×3 depthwise convolution to extract local edge and corner features, resulting in a local information aggregation feature map. The calculation formula is as follows:

[0151] ;

[0152] in, This represents a feature map representing the aggregation of local information.

[0153] Step S223: Based on the typical scale of traffic signs corresponding to the P3, P4, and P5 feature layers of the detection head, parallel feature extraction is performed on the local information aggregation feature map using different combinations of strip convolution kernels:

[0154] For small-scale markers, the P3 feature layer uses a 3×1+1×3+5×1+1×5 convolution to capture the digital contours of the markers, resulting in a small-scale feature map. The calculation formula is as follows:

[0155] ;

[0156] in, This represents a small-scale feature map of the P3 feature layer. This represents a 3×1 convolution. This represents a 1×3 convolution. This represents a 5×1 convolution. This represents a 1×5 convolution;

[0157] For the mesoscale feature layer P4, a 7×1+1×7+11×1+1×11 convolutional kernel is used to adapt to triangular or circular shapes, resulting in a mesoscale feature map. The calculation formula is as follows:

[0158] ;

[0159] in, This represents the mesoscale feature map of the P4 feature layer. This represents a 7×1 convolution. This represents a 1×7 convolution. This represents an 11×1 convolution. This represents a 1×11 convolution;

[0160] For the large-scale marker P5 feature layer, a convolution kernel of 11×1+1×11+21×1+1×21 is used to cover the global features of the rectangular marker, resulting in a large-scale feature map. The calculation formula is as follows:

[0161] ;

[0162] in, This represents the large-scale feature map of the P5 feature layer. This represents a 21×1 convolution. Represents a 1×21 convolution;

[0163] Step S224: Process the small-scale feature maps of each layer. Mesoscale feature map Large-scale feature maps A 1×1 convolution is used to compress the channels to obtain a multi-scale fused feature map. The calculation formula is as follows:

[0164] ;

[0165] in, This represents the multi-scale fused feature map of the multi-scale convolutional attention module. Representing small-scale feature maps of each layer Mesoscale feature map Large-scale feature maps n=3, 4, 5;

[0166] Extracting multi-scale fused feature maps The high-dimensional aggregated feature map of the channels is used to generate color weights, and the calculation formula is as follows:

[0167] ;

[0168] in, Indicates color weight, A high-dimensional aggregated feature map representing the channels of a multi-scale convolutional attention module;

[0169] Fusion of color weights with multi-scale feature maps Element-wise multiplication yields the color-enhanced attention features, calculated using the following formula:

[0170] ;

[0171] in, This indicates the attentional features after color enhancement;

[0172] Step S225: Aggregate the color-enhanced attention features with the local information feature map. The feature map is fused and focused to obtain the final output enhanced feature map. The calculation formula is as follows:

[0173] ;

[0174] in, This represents the final output augmented feature map of the multi-scale convolutional attention module.

[0175] Furthermore, step S3 specifically includes:

[0176] Step S31: Load the pre-trained weights of the improved YOLOv12 backbone network, and initialize the network parameters of the embedded feature complementarity mapping module and multi-scale convolutional attention module using the uniform distribution method Xavier. The network parameter initialization formula is as follows:

[0177] ;

[0178] in, This represents the set of learnable parameters for the feature complementarity mapping module. Indicates uniform distribution. Indicates the number of input channels. Indicates the number of output channels;

[0179] Step S32: Using a stochastic gradient descent optimizer, combined with a momentum of 0.937 and a weight decay coefficient of 0.0005, and employing a cosine annealing learning rate adjustment method, the dynamic learning rate for the current iteration step is obtained, as shown in the following formula:

[0180] ;

[0181] in, This represents the dynamic learning rate at the current iteration step. This represents the initial learning rate. Let T represent the cosine trigonometric function, t represent the current training iteration number, and T=160000 represent the total number of iterations. Represents pi;

[0182] Step S33: The class-balanced augmented dataset is input into the improved YOLOv12 detection network in batches. The improved YOLOv12 detection network performs forward inference on each batch of input data and calculates the composite loss function, which includes:

[0183] The SIoU loss function is used to optimize the position and shape regression of the bounding box, and its formula is:

[0184] ;

[0185] in, Represents the SIoU loss function. This represents the intersection-over-union ratio (IoU), which measures the degree of overlap between the predicted bounding box and the ground truth bounding box. Indicates distance loss. , This represents the normalized distance between the predicted bounding box and the ground truth bounding box. Indicates the center coordinates in the horizontal direction. Indicates the center coordinates in the vertical direction. This represents the negative exponential form of the natural exponential function. Indicates shape loss, , =4, Indicates the horizontal width of the bounding box. Indicates the vertical height of the bounding box. The penalty coefficient representing shape loss;

[0186] A shape matching loss function based on image moments (Hu) is used to enhance the perception of regular shapes of traffic signs. The formula is as follows:

[0187] ;

[0188] in, This represents the shape matching loss function. This represents the natural exponential function. This represents the adjusted loss weight, k=5. This represents the image moment distance (Hu) between the predicted bounding box and the traffic sign shape template.

[0189] The composite loss function is a weighted sum of the SIoU loss function and the shape matching loss function, and the formula is:

[0190] ;

[0191] in, This represents the composite loss function.

[0192] Furthermore, step S4 specifically includes:

[0193] Step S41: Obtain the traffic scene image to be detected, and perform RGB to HSV space conversion, channel high-dimensional aggregation feature map filtering, contrast-limited adaptive histogram equalization CLAHE brightness enhancement, and 640×640 scaling operation on the traffic scene image to be detected to obtain the preprocessed image.

[0194] Step S42: Input the preprocessed image into the trained and optimized improved YOLOv12 detection network, and process it through the backbone network, feature complement mapping module, feature fusion network and multi-scale convolutional attention module to obtain enhanced multi-scale features;

[0195] Step S43: Input the enhanced multi-scale features into the detection head for processing to obtain the output features of the detection head, and map and decode the output features of the detection head into preliminary detection results;

[0196] Step S44: The preliminary detection results are processed using an improved non-maximum suppression method to suppress overlapping bounding boxes. This is combined with color-weighted correction of predicted boxes with confidence scores below 0.5, yielding the corrected target confidence score. The formula is as follows:

[0197] ;

[0198] in, This represents the corrected target confidence level. Indicates the confidence level of the original target;

[0199] Step S45: Perform final filtering based on the corrected target confidence level and the preset threshold, and output the final detection results. The final detection results include the following for each traffic sign: category name or number, precise bounding box coordinates, and corrected confidence level.

[0200] Example 1:

[0201] Dataset Augmentation and Preprocessing: Due to the large variety of traffic signs in China, the number of labels for some traffic sign categories in publicly available datasets is insufficient, resulting in poor detection performance. This invention employs data augmentation methods, which involve randomly pasting sign images and performing scaling, rotation, and blurring techniques to balance the number of labels for each category. Ultimately, an augmented dataset containing 45 categories of traffic signs is formed, ensuring that the number of samples for each label is greater than 1200.

[0202] Integration and optimization of the complementary feature mapping module: The complementary feature mapping module is embedded in the Q4 layer of the YOLOv12 backbone network. The complementary feature mapping module completes the spatial information of small targets through a four-step process of channel segmentation, orientation transformation, complementary mapping and feature aggregation. Especially in the detection of small targets in traffic signs, the spatial information loss of small targets during downsampling is alleviated by aligning spatial features and semantic features.

[0203] Application of the multi-scale convolutional attention module: The multi-scale convolutional attention module is integrated into the P3, P4, and P5 layers of the YOLOv12 detection head. Differentiated multi-branch convolutional kernel combinations are designed for traffic signs of different scales, and color attention branches are fused. The multi-scale convolutional attention module can enhance the feature response of signs of different scales according to their different scales and color characteristics, thereby improving detection accuracy.

[0204] Training strategy and loss function: This invention adopts a composite loss function that combines SIoU loss and shape matching loss to optimize the bounding box regression and shape perception capabilities of traffic signs, especially for the detection of traffic signs with regular shapes, thereby further improving the accuracy and robustness of the model in complex scenarios.

[0205] Reasoning process:

[0206] Input preprocessing: The image to be detected is preprocessed through steps such as RGB→HSV conversion, H channel filtering, CLAHE brightness enhancement, and 640×640 scaling to generate input features.

[0207] Feature enhancement: The image is input into the backbone network, and after feature complementation mapping module and PANet feature fusion, the feature map is enhanced by multi-scale convolutional attention module, and the enhanced features are output.

[0208] Target prediction and post-processing: The detection head performs target detection and classification on each anchor box, and outputs the predicted category, bounding box and confidence score; redundant boxes are filtered by the Soft-NMS algorithm, and low-confidence predictions are corrected by combining color features, and finally the category, accurate bounding box and confidence score of the traffic sign are output.

[0209] Example 2: The specific process is as follows:

[0210] We augmented the commonly used tt100k traffic sign dataset in China to increase the number of traffic sign images that were originally underlabeled.

[0211] The traffic sign dataset is input into the backbone feature extraction layer to obtain the feature vectors of all traffic signs in the image and text. The feature encoder can be composed of a pre-trained YOLOv12 model.

[0212] The obtained flags are embedded in feature vectors and then input into the feature fusion module to fuse feature maps from different stages of backbone feature extraction, thereby enhancing representation capabilities.

[0213] The enhanced feature maps eventually reach the detection head, which is responsible for object detection and classification tasks. The detection head contains a series of convolutional and deconvolutional layers to generate detection results. These layers are responsible for predicting the bounding box regression value of each anchor box and the confidence level of the object's presence. The classification head uses global average pooling to classify each feature map. By reducing the dimensionality of the feature maps, it outputs the probability distribution of each class, enabling the model to effectively handle multi-class classification tasks.

[0214] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A traffic sign detection method based on spatial completion and multi-scale focusing, characterized in that, Includes the following steps: Step S1: Obtain the original traffic sign dataset, perform statistical analysis and augmentation on the original traffic sign dataset to obtain a class-balanced augmented dataset; Step S2: Construct an improved YOLOv12 detection network. The improved YOLOv12 detection network includes a backbone network, a feature fusion network, and a detection head. A feature complementation mapping module is embedded in the Q4 feature layer of the backbone network to complete the color and spatial information of small targets on traffic signs. Multi-scale convolutional attention modules are embedded in the P3, P4, and P5 feature layers of the detection head to perform multi-scale feature focusing on traffic signs of different scales. Step S3: Train the improved YOLOv12 detection network using a class-balanced augmented dataset, and optimize the improved YOLOv12 detection network using a composite loss function; Step S4: After preprocessing the image to be detected, input it into the trained and optimized improved YOLOv12 detection network. The network is processed through the backbone network, feature complement mapping module, feature fusion network and multi-scale convolutional attention module to output the detection result of traffic signs. In step S2, a feature complement mapping module is embedded in the Q4 feature layer of the backbone network, specifically as follows: Step S211: Extract the first feature map output from the Q4 feature layer of the backbone network. ; Step S212, transfer the first feature map Scaled along the channel dimension The data is divided into two parts, resulting in a spatial-color branch and a semantic branch. Step S213: Perform feature transformations on the space-color branch and the semantic branch respectively. Step S214: Achieve bidirectional feature complementarity through an attention mechanism: The semantic features are processed by deep convolution to cut off redundant channel associations, global average pooling is used to capture global information of typical color channels of traffic signs, and then channel attention weights are generated by activating the Sigmoid function. The spatial-color features are processed by 1×1 convolution with batch normalization and Sigmoid function to focus on the spatial region where traffic signs are located and generate spatial attention weights. The channel attention weights are weighted to the spatial-color features to obtain the spatial-location feature map, and the spatial attention weights are weighted to the semantic features to obtain the semantic feature map. Step S215: Concatenate the supplemented spatial-location feature map and semantic feature map along the channel dimension, and then combine them with the first feature map. Perform residual connections to obtain the enhanced second feature map. ; In step S2, multi-scale convolutional attention modules are embedded in the P3, P4, and P5 feature layers of the detection head, specifically as follows: Step S221: Obtain the P3, P4, and P5 feature layers from the pre-processing or feature fusion network, respectively, which correspond to traffic sign feature representations at different spatial resolutions, and then process the second feature map. As input to the multi-scale convolutional attention module; Step S222, for the second feature map Local information aggregation is performed using 3×3 depthwise convolution to extract local edge and corner features, resulting in a local information aggregation feature map. Step S223: Based on the typical scale of traffic signs corresponding to the P3, P4, and P5 feature layers of the detection head, parallel feature extraction is performed on the local information aggregation feature map using different combinations of strip convolution kernels to obtain small-scale feature maps, medium-scale feature maps, and large-scale feature maps respectively. Step S224: Compress the channels of each layer’s small-scale feature map, medium-scale feature map, and large-scale feature map using 1×1 convolution to obtain a multi-scale fused feature map. Extracting multi-scale fused feature maps High-dimensional aggregated feature maps of channels are used to generate color weights; Fusion of color weights with multi-scale feature maps Element-wise multiplication yields the color-enhanced attention features. Step S225: Aggregate the color-enhanced attention features with the local information feature map. The features are then fused to achieve feature focusing, resulting in the final output enhanced feature map.

2. The traffic sign detection method based on spatial completion and multi-scale focusing according to claim 1, characterized in that: Step S1 is as follows: Step S11: Obtain the original traffic sign dataset and annotation file from the public dataset. The original traffic sign dataset contains images and corresponding traffic sign category labels and bounding box information. Step S12: Read the annotation files in the original traffic sign dataset, count the number of annotation boxes for each type of traffic sign, and identify the categories with an annotation count lower than a preset threshold as the categories to be enhanced. Step S13: Set the target number of labels for each type of traffic sign to ensure that the number of samples in each category in the original traffic sign dataset reaches the preset balance condition; Set enhanced operation parameters, including scaling range, rotation angle range, blur intensity, and random range for paste position; Step S14: Enhance the category to be enhanced to obtain the enhanced annotation information data; Step S15: Merge the original traffic sign dataset with the augmented label information data to form a class-balanced augmented dataset, ensuring that the total number of labels for each type of traffic sign is not less than the preset number; randomly divide the augmented dataset into training set, validation set and test set according to the preset ratio, and ensure that the class distribution in each subset remains balanced after the division.

3. The traffic sign detection method based on spatial completion and multi-scale focusing according to claim 2, characterized in that: In step S14, the category to be enhanced is enhanced to obtain the enhanced annotation information data; Specifically: Step S141: Extract traffic sign regions from the original image of the category to be enhanced; Step S142: Randomly scale, rotate, and blur the extracted traffic sign area; Step S143: Randomly paste the processed traffic sign area into a background image that does not contain signs of the category to be enhanced, and ensure that its spatial position is reasonable for the traffic scene; Step S144: Generate new annotation information data, including updated bounding box coordinates and category labels.

4. The traffic sign detection method based on spatial completion and multi-scale focusing according to claim 1, characterized in that: In step S2, a feature complementation mapping module is embedded in the Q4 feature layer of the backbone network to complete the color and spatial information of small targets on traffic signs; specifically: Step S211, the first feature map is output. , Where C=512, H=80, W=80, R represents the set of real numbers in mathematics, and the first characteristic graph The value at each position in the graph is a real number, and C represents the first feature map. The channel dimension, H represents the first feature map. The spatial height, W represents the first feature map. Spatial width; Step S212 yields the space-color branch and the semantic branch, calculated using the following formula: ； in, The representation space-color branch is used to preserve the pixel coordinates, edge contours, and color distribution of traffic signs. This indicates a semantic branch, used to extract the shape semantic information of traffic signs. This indicates a splitting operation. This represents the scaling factor for channel segmentation in the complementary feature mapping module; Step S213, spatial-color features, the calculation formula is: ； in, Represents spatial-color characteristics, Represents a 1×1 convolution; The semantic branch is processed using a standard 3×3 convolution to extract shape semantic features. The calculation formula is as follows: ； in, Represents semantic features, This represents a 3×3 standard convolution. This indicates that the convolution kernel is in the first feature map. The step size of the upward slide. This indicates that before the convolution operation, the first feature map is given... Fill the edges with 0-value pixels; Step S214: Generate channel attention weights, calculated using the following formula: ； ； in, This represents the intermediate feature map obtained after semantic features are processed by a 3×3 depthwise convolution. Represents depthwise convolution. Indicates channel attention weights. This represents the Sigmoid function. This represents the pixel position index in the vertical direction of the intermediate feature map, with a value range of 0 ≤ i. <H, This represents the pixel position index in the horizontal direction of the intermediate feature map, with a value range of 0 ≤ j. <W; Spatial attention weights are calculated using the following formula: ； in, Represents spatial attention weights. This indicates a batch normalization operation; The semantic feature map is calculated using the following formula: ； ； in, This represents the spatial-location feature map after channel attention weighting. This represents element-wise multiplication. This represents the semantic feature map after spatial attention weighting; Step S215, Enhanced second feature map The calculation formula is: ； in, This represents the enhanced second feature map. This indicates channel splicing, integrating bidirectional complementary features. This indicates a residual connection.

5. The traffic sign detection method based on spatial completion and multi-scale focusing according to claim 4, characterized in that: In step S2, multi-scale convolutional attention modules are embedded in the P3, P4 and P5 feature layers of the detection head to focus on the multi-scale features of traffic signs at different scales. Specifically: Step S222, local information aggregation feature map, the calculation formula is: ； in, This represents a feature map representing the aggregation of local information. Step S223: Parallel feature extraction is performed using different combinations of strip convolution kernels to obtain small-scale feature maps, medium-scale feature maps, and large-scale feature maps respectively. For small-scale markers, the P3 feature layer uses a 3×1+1×3+5×1+1×5 convolution to capture the digital contours of the markers, resulting in a small-scale feature map. The calculation formula is as follows: ； in, This represents a small-scale feature map of the P3 feature layer. This represents a 3×1 convolution. This represents a 1×3 convolution. This represents a 5×1 convolution. This represents a 1×5 convolution; For the mesoscale feature layer P4, a 7×1+1×7+11×1+1×11 convolutional kernel is used to adapt to triangular or circular shapes, resulting in a mesoscale feature map. The calculation formula is as follows: ； in, This represents the mesoscale feature map of the P4 feature layer. This represents a 7×1 convolution. This represents a 1×7 convolution. This represents an 11×1 convolution. This represents a 1×11 convolution; For the large-scale marker P5 feature layer, a convolution kernel of 11×1+1×11+21×1+1×21 is used to cover the global features of the rectangular marker, resulting in a large-scale feature map. The calculation formula is as follows: ； in, This represents the large-scale feature map of the P5 feature layer. This represents a 21×1 convolution. Represents a 1×21 convolution; Step S224, multi-scale feature map fusion, the calculation formula is: ； in, This represents the multi-scale fused feature map of the multi-scale convolutional attention module. Representing small-scale feature maps of each layer Mesoscale feature map Large-scale feature maps n=3, 4, 5; Color weight, calculated using the following formula: ； in, Indicates color weight, A high-dimensional aggregated feature map representing the channels of a multi-scale convolutional attention module; The formula for calculating the attention features after color enhancement is as follows: ； in, This indicates the attentional features after color enhancement; Step S225, the final output is the enhanced feature map, calculated using the following formula: ； in, This represents the final output augmented feature map of the multi-scale convolutional attention module.

6. The traffic sign detection method based on spatial completion and multi-scale focusing according to claim 1, characterized in that: Step S3 is as follows: Step S31: Load the pre-trained weights of the improved YOLOv12 backbone network, and initialize the network parameters of the embedded feature complementarity mapping module and multi-scale convolutional attention module using the uniform distribution method Xavier. The network parameter initialization formula is as follows: ； in, This represents the set of learnable parameters for the feature complementarity mapping module. Indicates uniform distribution. Indicates the number of input channels. Indicates the number of output channels; Step S32: Using the stochastic gradient descent optimizer and the cosine annealing learning rate adjustment method, the dynamic learning rate for the current iteration step is obtained, as shown in the following formula: ； in, This represents the dynamic learning rate at the current iteration step. This represents the initial learning rate. Let T represent the cosine trigonometric function, t represent the current training iteration number, and T=160000 represent the total number of iterations. Represents pi; Step S33: The class-balanced augmented dataset is input into the improved YOLOv12 detection network in batches. The improved YOLOv12 detection network performs forward inference on each batch of input data and calculates the composite loss function, which includes: The SIoU loss function is used to optimize the position and shape regression of the bounding box, and its formula is: ； in, Represents the SIoU loss function. This represents the intersection-over-union ratio (IoU), which measures the degree of overlap between the predicted bounding box and the ground truth bounding box. Indicates distance loss. , This represents the normalized distance between the predicted bounding box and the ground truth bounding box. Indicates the center coordinates in the horizontal direction. Indicates the center coordinates in the vertical direction. Indicates shape loss, , =4, Indicates the horizontal width of the bounding box. Indicates the vertical height of the bounding box. The penalty coefficient representing shape loss; A shape matching loss function based on image moments (Hu) is used to enhance the perception of regular shapes of traffic signs. The formula is as follows: ； in, This represents the shape matching loss function. This represents the natural exponential function. This represents the adjusted loss weight, k=5. This represents the image moment distance (Hu) between the predicted bounding box and the traffic sign shape template. The composite loss function is a weighted sum of the SIoU loss function and the shape matching loss function, and the formula is: ； in, This represents the composite loss function.

7. The traffic sign detection method based on spatial completion and multi-scale focusing according to claim 1, characterized in that: Step S4 is as follows: Step S41: Obtain the traffic scene image to be detected, and perform RGB to HSV space conversion, channel high-dimensional aggregation feature map filtering, contrast-limited adaptive histogram equalization CLAHE brightness enhancement, and 640×640 scaling operation on the traffic scene image to be detected to obtain the preprocessed image. Step S42: Input the preprocessed image into the trained and optimized improved YOLOv12 detection network, and process it through the backbone network, feature complement mapping module, feature fusion network and multi-scale convolutional attention module to obtain enhanced multi-scale features; Step S43: Input the enhanced multi-scale features into the detection head for processing to obtain the output features of the detection head, and map and decode the output features of the detection head into preliminary detection results; Step S44: The preliminary detection results are processed using an improved non-maximum suppression method to suppress overlapping bounding boxes. This is combined with color-weighted correction of predicted boxes with confidence scores below 0.5, yielding the corrected target confidence score. The formula is as follows: ； in, This represents the corrected target confidence level. Indicates the confidence level of the original target; Step S45: Perform final filtering based on the corrected target confidence level and the preset threshold, and output the final detection results. The final detection results include the following for each traffic sign: category name or number, precise bounding box coordinates, and corrected confidence level.

Images