Traffic sign detection method based on glare environment

Abstract

The invention discloses a traffic sign detection method based on a glare environment, and the method specifically comprises the steps: employing a Mosaic data enhancement technology, splicing a plurality of images to generate a new image sample, and dividing the new image sample into a training set and a test set; designing a traffic sign detection model YOLO-SFM, and inputting the training set images and the corresponding picture labels into the traffic sign detection model YOLO-SFM for training until convergence; and inputting the test set image into the model, and outputting a detection result. A traffic sign detection model YOLO-SFM is utilized, an SPD-Conv module, a FasterNet module and an MLCA module are combined, efficient extraction and fusion of features are achieved, calculation redundancy and information loss are reduced, and the detection precision and efficiency of traffic signs under the glare condition are improved. According to the method, the traffic sign in the glare environment can be rapidly and accurately detected, and the method has the characteristics of light weight, high real-time performance, wide applicability and the like.

Description

Technical Field

[0001] The invention belongs to the technical field of target detection, and in particular relates to a traffic sign detection method based on a glare environment. Background Art

[0002] Strong sunlight is an important environmental factor affecting traffic safety, especially the interference with the driver's vision during daily driving, which has a significant negative impact on road safety. When the sun is at a low angle, the glare of the sun penetrates the car visor, which may cause the driver's visual sensors (including eyes or cameras) to fail temporarily, thereby affecting the ability to recognize traffic signs and other road information, increasing the risk of traffic accidents.

[0003] In recent years, with the development of autonomous driving technology, the application of object detection algorithms in the field of transportation has received widespread attention. Autonomous driving systems can effectively improve driving safety by identifying objects on the road, such as traffic signs, in real time. However, complex glare environments can reduce image quality, limit the performance of object detection algorithms, and increase the technical difficulty of traffic sign detection. How to improve the detection accuracy of traffic signs under glare conditions is an important challenge in the current research of autonomous driving technology.

[0004] Traffic sign detection methods can generally be divided into three categories: traditional image processing methods, machine learning-based methods, and deep learning-based methods. Traditional image processing methods include: Hough transform (used to detect shapes such as circles and lines), color threshold method (HSV, YCrCb space segmentation of traffic sign colors), contour detection and matching. It has low computational complexity, strong real-time performance, and is suitable for low-computing devices. However, it is sensitive to lighting changes, occlusions, and complex backgrounds, and is easily affected by weather, lighting, and noise. Machine learning-based methods use traditional machine learning models for classification after extracting features. For example, HOG (Histogram of Oriented Gradients) + SVM (Support Vector Machine) has high detection accuracy in simple environments. However, feature extraction requires a lot of manual intervention and is not flexible enough. Deep learning-based methods use convolutional neural networks (CNN) to automatically learn features to achieve end-to-end traffic sign detection and recognition. For example, the YOLO series (YOLOv4, YOLOv5, YOLOv8), Faster R-CNN, SSD (SingleShot MultiBox Detector), and RetinaNet can automatically learn deep features from data without manually designing features. After training on large-scale data sets, they have excellent performance, especially for multi-category and multi-scale targets. However, the models are complex, and training and reasoning require high computing power, which limits their application, especially in embedded systems. Summary of the invention

[0005] The purpose of the present invention is to provide a method for detecting traffic signs in a glare environment, which can realize real-time detection of traffic signs in glare by using lightweight and high-precision detection.

[0006] The technical solution adopted by the present invention is a method for detecting traffic signs in a glare environment, which is specifically implemented according to the following steps:

[0007] Step 1, data preprocessing: Mosaic data enhancement technology is used to splice multiple images in the data set to generate new image samples, and the data set is divided into a training set and a test set;

[0008] Step 2: Design the traffic sign detection model YOLO-SFM, which includes the Backbone module and the Head module;

[0009] Step 3: Input the training set images and corresponding image labels into the established traffic sign detection model YOLO-SFM for training until convergence;

[0010] Step 4: Input the test set images into the trained traffic sign detection model YOLO-SFM and output the detection results, which include the category label, bounding box coordinates, and confidence score of the traffic sign.

[0011] The present invention is also characterized in that:

[0012] In step 1, specifically:

[0013] Step 1.1, randomly select four images from the data set, denoted as I1, I2, I3, I4, where the size of each image is W×H; then scale the images: scale the selected images to new sizes according to the ratios α and β; then perform image stitching and place the four images on a blank image I Mosaic In the four quadrants of;

[0014] At the same time, adjust the annotation box: For each original image, the annotation box B = (X min ,Y min ,X max ,Y max ), according to the image scaling ratio and splicing position, adjust to the new annotation frame B′=(X ′ min ,Y ′ min ,X ′ max ,Y ′ max ), the adjustment formula is as follows;

[0015] x ′ =αx+x offset ,y′ =βx+y offset

[0016] where x offset ,y offset is the offset of the stitched image;

[0017] Step 1.2: The stitched image I Mosaic Normalization is performed to ensure that the pixel values ​​are within the range required for model training, and all annotation boxes are recorded in a new annotation file; the final generated Mosaic image is divided into a training set and a test set.

[0018] In step 2, specifically:

[0019] Step a, use Backbone module to process the image;

[0020] In step b, the Head module is used to fuse the features of different scales extracted by the Backbone. These features are fused through further convolution operations to meet the detection requirements of targets of different sizes.

[0021] In step a, specifically:

[0022] Step a1, the training set Mosaic image in step 1 passes through a standard convolution layer, the convolution layer uses a convolution kernel of size 3 for downsampling, increases the number of channels to keep the feature map size unchanged, the convolution layer stride is 1, and the output channel number is 64; then use SPDConv to process the feature map; pass through the convolution layer again, the convolution kernel size is 3, the stride is 1, and the output channel number is still 128; finally, use SPDConv to process the feature map again, and the final output feature map size is B×128×160×160;

[0023] The specific process of SPDConv processing feature maps is as follows: the input feature map size is B×64×640×640, where B is the batch size, 64 is the number of input channels, and 640×640 is the spatial scale. Each 2×2 pixel block is rearranged into a channel dimension through SPD rearrangement, and the number of channels increases from 64 to 64×4=256, and the spatial size is halved; the rearranged feature map X′ is subjected to standard convolution, the convolution kernel size is 1×1, the stride s=1, the convolution output is batch normalized, and then the SiLU activation function is applied for processing, and the final output feature map size is B×128×320×320; at this point, the SPD convolution processing is completed;

[0024] Step a2, process the feature map through three FasterBlocks;

[0025] Specifically: the input feature map is divided into two 64-channel blocks according to the channel dimension, and each block will be processed by FasterBlock, that is, for each block, its shape remains B×64×160×160, and the output feature map shape is still B×64×160×160. The two processed blocks will be spliced ​​in the channel dimension, that is, the two parts of the feature map will be spliced ​​into a feature map of B×128×160×160; finally, through a 1×1 convolution, the final output feature map is B×128×160×160.

[0026] Step a3, processing the feature map obtained in step a2 through the MLCA module;

[0027] Specifically, the feature map is subjected to adaptive average pooling to obtain a local feature map, the local feature map is further pooled to obtain a global feature map, the local features are flattened and transformed through 1D convolution, and the convolution layer is used for feature extraction; after flattening the global features, 1D convolution is performed and the convolution layer is used for feature extraction, and the local and global features are subjected to sigmoid activation function to generate local attention maps and global attention maps; a weighted fusion strategy is used to combine the local and global attention maps; the input feature map is element-wise multiplied with the fused attention map, and the final feature map is output, and the output feature map is still B×128×160×160;

[0028] Step a4, the feature map obtained in step a3 is processed by a convolution layer, the stride of the convolution layer is 1, the number of output channels is 256, and the spatial size remains unchanged;

[0029] In step a5, the feature map is processed using the SPDConv layer. The SPDConv layer halves the spatial size through space-to-depth conversion and increases the number of channels by 4 times. The convolution is then used to reduce the dimension back to 256 channels. The size of the output feature map is B×256×80×80.

[0030] Step a6, use FasterBlock 6 times to process the feature map in step a5. The FasterBlock module extracts and enhances features, keeping the shape of the feature map unchanged;

[0031] Step a7, using the MLCA module to optimize the feature map in step a6, and the shape of the final output feature map is B×256×80×80;

[0032] Step a8: pass the feature map obtained in step a7 through the convolution layer, increase the number of channels to 512, and the stride of the convolution layer is 1;

[0033] Step a9, use SPDConv to process the feature map of 512 channels, and the final output feature map size is B×512×40×40;

[0034] Step a10, use FasterBlock 6 times to process the feature map in step a9, keep the number of channels as 512 each time, and concatenate them to finally output the feature map B×512×40×40;

[0035] Step a11, use the MLCA module to optimize the feature map in step a10, generate and fuse local and global attention maps, and output a feature map B×512×40×40;

[0036] Step a12, passing the feature map obtained in step a11 through a convolutional layer, the stride of the convolutional layer is 1, and the number of output channels is increased to 1024;

[0037] Step a13, use SPDConv to process the feature map;

[0038] Step a14, use FasterBlock 3 times to process the feature map in step a13, keeping the number of output channels at 1024;

[0039] Step a15, using the MLCA module to optimize the feature map in step a14;

[0040] Step a16, extract multi-scale features from the feature map in step a15 through a maximum pooling operation, and the final output feature map size is maintained at B×1024×20×20.

[0041] In step b, specifically:

[0042] Step b1, upsampling, doubling the spatial size to obtain a B×1024×40×40 feature map; concatenating the feature map with the B×512×40×40 feature map output by step a11 in the channel dimension to obtain a B×1536×40×40 feature map;

[0043] Step b2: use FasterBlock 3 times for processing, the number of output channels is 512, the spatial size remains unchanged, and the output feature map is B×512×40×40;

[0044] Step b3: Use the MLCA module to perform local and global attention fusion, keep the number of output channels at 512, and keep the spatial size at 40×40;

[0045] Step b4, upsampling to double the spatial size and output a feature map B×512×80×80; the feature map and the B×256×80×80 feature map output in step a7 are concatenated in the channel dimension to obtain a B×768×80×80 feature map;

[0046] Step b5, use FasterBlock 3 times to further process the feature map, the number of output channels is 256, and the spatial size is kept at 80×80;

[0047] Step b6: Use the MLCA module to perform local and global attention fusion, keep the number of output channels at 256, and keep the spatial size at 80×80;

[0048] Step b7, upsampling to double the spatial size to obtain a feature map B×256×160×160; the feature map and the feature map B×128×160×160 output in step a3 are concatenated in the channel dimension to obtain a feature map B×384×160×160;

[0049] Step b8, further processed by FasterBlock 3 times, the number of output channels is 128, and the spatial size remains at 160×160;

[0050] Step b9: Use the MLCA module to perform local and global attention fusion on the feature map, keep the number of output channels at 128, and keep the spatial size at 160×160;

[0051] Step b10, use convolution operation with stride 1 and convolution kernel size 3×3, the spatial size of the feature map remains unchanged, and the number of channels is 128;

[0052] Step b11, after SPDConv processing, the number of channels of the feature map increases to 256, and the spatial size is halved; output a B×256×80×80 feature map; the feature map and the B×256×80×80 feature map output in step b6 are spliced ​​in the channel dimension to obtain a B×512×80×80 feature map;

[0053] Step b12, after three times of FasterBlock further processing, the number of output channels is 256 and the spatial size remains 80×80;

[0054] Step b13, use the MLCA module to perform local and global attention fusion on the feature map, keep the number of output channels at 256, and keep the spatial size at 80×80;

[0055] Step b14, using convolution operation, the spatial size of the feature map remains 80×80 and the number of channels is 256;

[0056] Step b15: After SPDConv processing, the number of channels is increased from 256 to 512, and the spatial size is reduced to output a B×512×40×40 feature map; the feature map is concatenated with the B×512×40×40 feature map outputted in step b3 in the channel dimension to obtain a B×1024×40×40 feature map;

[0057] Step b16, after three times of FasterBlock further processing, the number of output channels is 512, and the spatial size remains at 40×40;

[0058] Step b17, use the MLCA module to perform local and global attention fusion on the feature map, keep the number of output channels at 512, and keep the spatial size at 40×40;

[0059] Step b18, using convolution operation, the spatial size of the feature map remains 40×40 and the number of channels is 512;

[0060] Step b19, after SPDConv processing, the number of channels is increased from 512 to 1024, and the spatial size is reduced, and a B×1024×20×20 feature map is output; the feature map and the B×1024×20×20 feature map output by step a16 are spliced ​​in the channel dimension to obtain a B×2048×20×20 feature map;

[0061] Step b20, after three times of FasterBlock further processing, the number of output channels is 1024, and the spatial size remains at 20×20;

[0062] In step b21, the MLCA module is used to perform local and global attention fusion on the feature map, and the number of output channels is kept at 1024 and the spatial size is kept at 20×20.

[0063] In step 3, specifically:

[0064] Set the network hyperparameters of the traffic sign detection model YOLO-SFM: optimizer optimizer is SGD, initial learning rate 1r0 is 0.01, momentum momentum is 0.937, weight decay weight_decay is 0.0005, batch size batchsize is 16;

[0065] The training set images and the corresponding image labels are input into the traffic sign detection model YOLO-SFM, the loss function is calculated and the gradient is back-propagated, and the optimizer updates the parameters until the training set loss function and the validation set function no longer decrease, and the output evaluation index no longer increases. At this time, the model converges and the model parameters are the final trained model parameters.

[0066] In step 4, specifically: input the test set image into the trained traffic sign detection model YOLO-SFM to generate the predicted bounding box B pred =(X min ,Y min ,X max ,Y max ) and the corresponding class probability P pred; According to the set confidence threshold τ, filter out the prediction boxes with high confidence:

[0067] B final ={B|P pred >τ}

[0068] The detection results include the category label, bounding box coordinates, and confidence score of the traffic sign, which are output to the terminal or stored in a file.

[0069] The beneficial effects of the present invention are as follows: the present invention is based on a lightweight target detection method for traffic signs in a glare environment, uses Mosaic data enhancement technology to optimize training samples, generates diversified data sets by splicing multiple images, and improves the model's adaptability to complex scenes; uses the traffic sign detection model YOLO-SFM, which combines the SPD-Conv module, the FasterNet module, and the MLCA module to achieve efficient feature extraction and fusion, reduce computational redundancy and information loss; and improves the detection accuracy and efficiency of traffic signs under glare conditions through a complete process from data preprocessing to model training and then to target detection. The method of the present invention can quickly and accurately detect traffic signs in a glare environment, and has the characteristics of lightweight, strong real-time performance, and wide applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 It is a structural diagram of the YOLO-SFM model in the traffic sign detection method under glare environment of the present invention;

[0071] Figure 2 This is a flow chart of convolutional layer downsampling in the YOLO-SFM model in the traffic sign detection method under glare environment of the present invention;

[0072] Figure 3 It is a structural diagram of the lightweight module FasterNetBlock in the traffic sign detection method under glare environment of the present invention;

[0073] Figure 4 It is a flow chart of the attention mechanism MLCA in the traffic sign detection method under glare environment of the present invention;

[0074] Figure 5 This is the ground truth traffic sign detection result map;

[0075] Figure 6 It is a schematic diagram of the detection results of the YOLO-SFM model in the traffic sign detection method under glare environment of the present invention;

[0076] Figure 7 This is a schematic diagram of the results of the original YOLOv8n detecting traffic signs under glare. DETAILED DESCRIPTION

[0077] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0078] Example 1

[0079] The present invention is based on a traffic sign detection method in a glare environment, specifically:

[0080] Step 1, data preprocessing: Mosaic data enhancement technology is used to splice multiple images in the data set to generate new image samples, and the data set is divided into a training set and a test set;

[0081] Step 2: Design the traffic sign detection model YOLO-SFM, which includes the Backbone module and the Head module;

[0082] Step 3: Input the training set images and corresponding image labels into the established traffic sign detection model YOLO-SFM for training until convergence;

[0083] Step 4: Input the test set images into the trained traffic sign detection model YOLO-SFM and output the detection results, which include the category label, bounding box coordinates, and confidence score of the traffic sign.

[0084] Example 2

[0085] The present invention is based on a traffic sign detection method in a glare environment, which is specifically implemented according to the following steps:

[0086] Step 1, data preprocessing: Mosaic data enhancement technology is used to stitch multiple images together to generate new image samples, thereby constructing a more diverse data set to improve the model's adaptability to different scenarios; and the data set is divided into a training set and a test set;

[0087] In traffic sign detection under glare environment, different images may contain different degrees of glare (such as sunlight at different angles, different degrees of cloud cover, etc.), different types of traffic signs, and different background scenes (such as urban roads, rural roads, highways, etc.). Mosaic data enhancement generates new samples by randomly splicing four different images. Splicing these images with different characteristics together can create more diverse training scenarios, allowing the model to learn the characteristics of traffic signs in various complex situations, so as to better cope with various glare environments and scene combinations that may be encountered in actual driving. Specifically:

[0088] Step 1.1, randomly select four images from the data set, denoted as I1, I2, I3, I4, where the size of each image is W×H; then scale the images: scale the selected images to new sizes (αW)×(βH) according to the ratios α, β (0<α, β<1), where α, β are randomly generated scale factors; then perform image stitching and place the four images on a blank image I Mosaic In the four quadrants, determine the position and size of each quadrant according to the following formula;

[0089] First quadrant: I1→[0,αW)×[0,βH)

[0090] Second quadrant: I2→[αW,W)×[0,βH)

[0091] The third quadrant: I3→[0,αW)×[βH,H)

[0092] The fourth quadrant: I4→[αW,W)×[βH,H)

[0093] At the same time, adjust the annotation box: For each original image, the annotation box B = (X min ,Y min ,X max ,Y max ), according to the image scaling ratio and splicing position, adjust to the new annotation frame B′=(X ′ min ,Y ′ min ,X ′ max ,Y ′ max ), the adjustment formula is shown below;

[0094] x ′ =αx+x offset ,y ′ =βx+y offset

[0095] where x offset ,y offset is the offset of the stitched image.

[0096] Step 1.2: The stitched image I Mosaic Perform normalization to ensure that pixel values ​​are within the range required for model training, and record all annotation boxes in a new annotation file; and divide the resulting Mosaic image into a training set and a test set;

[0097] The final generated Mosaic images and annotated data are used for model training and testing, which significantly improves the diversity of the dataset and the robustness of the model.

[0098] Step 2: Design the traffic sign detection model YOLO-SFM, specifically:

[0099] The designed traffic sign detection model YOLO-SFM is used to train traffic signs in glare environments to optimize feature extraction capabilities and detection accuracy.

[0100] like Figure 1 As shown, the traffic sign detection model YOLO-SFM specifically includes a Backbone module and a Head module;

[0101] Step a: Use the Backbone module to process the image, specifically:

[0102] Step a1: The training set Mosaic image (size 640×640) in step 1 is passed through a standard convolution layer (Conv). The convolution layer uses a convolution kernel of size 3 for downsampling, increases the number of channels to keep the feature map size unchanged, and outputs a feature map with a channel number of 64. Then, SPD convolution (SPDConv) is used to process the feature map. SPDConv can adaptively perform spatial convolution and enhance feature expression capabilities.

[0103] like Figure 2 As shown in the figure, the specific process of SPD convolution (SPDConv) processing feature maps is as follows: the input feature map size is B×64×640×640, where B is the batch size, 64 is the number of input channels, and 640×640 is the spatial scale. Each 2×2 pixel block is rearranged into a channel dimension through SPD rearrangement, the number of channels increases from 64 to 64×4=256, and the spatial size is halved;

[0104] The SPD rearrangement formula is as follows:

[0105] X'=concat(X[:,:,::2,::2], X[:,:,1::2,::2],

[0106] Where X is the input feature map, with a shape of (B, 64, 320, 320); ::2 means sampling every two pixels along the spatial dimension; concat() means concatenating four sampling results along the channel dimension;

[0107] The rearranged feature map X′ is subjected to standard convolution with a convolution kernel size of 1×1 and a stride of s=1. The convolution output is batch normalized and then processed with the SiLU activation function. The size of the final output feature map is B×128×320×320. At this point, the SPD convolution is completed.

[0108] Among them, the first block: sampling even rows and even columns:

[0109]

[0110] The second block: sampling odd rows and even columns:

[0111]

[0112] The third block: sampling even rows and odd columns:

[0113]

[0114] Block 4: Sampling odd rows and columns:

[0115]

[0116] After passing through the convolution layer again, the convolution kernel size is 3 and the stride is 1. The convolution layer keeps the spatial size unchanged, but extracts more feature information, and the number of output channels is still 128; finally, SPDConv is used again to process the feature map, and the size of the final output feature map is B×128×160×160;

[0117] Step a2, process the feature map through three FasterBlocks;

[0118] like Figure 3 As shown in Figure 1, FasterBlock is an enhanced convolution module that uses partial convolution (PConv) and multi-layer perceptron (MLP) to extract richer features.

[0119] Specifically: the input feature map is divided into two 64-channel blocks according to the channel dimension, and each block will be processed by FasterBlock, that is, for each block, its shape remains B×64×160×160, and the output feature map shape is still B×64×160×160, the features will be enhanced, and the processed feature maps will be combined together; the two processed blocks will be spliced ​​in the channel dimension, that is, the two parts of the feature map will be spliced ​​into a feature map of B×128×160×160; finally, through a 1×1 convolution, the final output feature map is B × 128×160×160.

[0120] Step a3, processing the feature map obtained in step a2 through the MLCA module;

[0121] like Figure 4 As shown in Figure 2, MLCA uses a multi-level contextual attention mechanism to optimize the fusion of global and local information in feature maps.

[0122] Specifically, the feature map is adaptively averaged pooled to obtain a local feature map, and the output shape is:

[0123]

[0124] The local feature map is further pooled to obtain the global feature map, and the output shape is:

[0125]

[0126] Flatten the local features and transform them through 1D convolution:

[0127] X local(flat ) = reshape(X local , B×1×(128×25))

[0128] Use convolutional layer for feature extraction, output:

[0129] Y local =Conv1d(X local(flat ))

[0130] After flattening the global features, pass 1D convolution:

[0131] X global(flat) = reshape(X global , B×1×128)

[0132] Use convolutional layer for feature extraction, output:

[0133] Y global =Conv1d(X global(flat) )

[0134] The sigmoid activation function is used for local features and global features to generate local attention maps and global attention maps:

[0135] Att local =σ(Y local )Att global =σ(Y global )

[0136] Use a weighted fusion strategy to combine local and global attention maps:

[0137] Att all =AdaptiveAvgPool2d(Att local ×local_weight+Att global ×(1-local_weight))

[0138] Multiply the input feature map and the fused attention map element by element, and output the final feature map. The output feature map is still B×128×160×160;

[0139] X output =X⊙Att all

[0140] Step a4, the feature map obtained in step a3 is processed by a convolution layer, the stride of the convolution layer is 1, the number of output channels is 256, and the spatial size remains unchanged;

[0141] In step a5, the feature map is processed using the SPDConv layer. The SPDConv layer halves the spatial size through space-to-depth conversion and increases the number of channels by 4 times. The convolution is then used to reduce the dimension back to 256 channels. The size of the output feature map is B×256×80×80.

[0142] Step a6, use FasterBlock 6 times to process the feature map in step a5. The FasterBlock module extracts and enhances features, keeping the shape of the feature map unchanged;

[0143] Step a7, use the MLCA module to optimize the feature map in step a6. The MLCA module generates weighted features through local and global attention mechanisms to improve the expressiveness of features. The shape of the final output feature map is B×256×80×80;

[0144] Step a8: pass the feature map obtained in step a7 through the convolution layer, increase the number of channels to 512, and the stride of the convolution layer is 1;

[0145] Step a9, using SPDCon v The feature map of 512 channels is processed, and the final output feature map size is B×512×40×40;

[0146] Step a10, use FasterBlock 6 times to process the feature map in step a9, keep the number of channels as 512 each time, and concatenate them to finally output the feature map B×512×40×40;

[0147] Step a11, use the MLCA module to optimize the feature map in step a10, generate and fuse local and global attention maps, and output a feature map B×512×40×40;

[0148] Step a12, passing the feature map obtained in step a11 through a convolutional layer, the stride of the convolutional layer is 1, and the number of output channels is increased to 1024;

[0149] Step a13, use SPDConv to process the feature map;

[0150] Step a14, use FasterBlock 3 times to process the feature map in step a13, keeping the number of output channels at 1024;

[0151] Step a15, using the MLCA module to optimize the feature map in step a14;

[0152] Step a16, extract multi-scale features from the feature map in step a15 through the maximum pooling operation, and the final output feature map size is maintained at B×1024×20×20. This module is an improvement of spatial pyramid pooling (SPP), which can improve the model's ability to perceive targets at different scales, and the number of output channels is 1024;

[0153] Step b: Use the Head module to fuse the different scale features extracted by the Backbone. These features are fused through further convolution operations to meet the detection requirements of objects of different sizes. Specifically:

[0154] Step b1, upsampling, doubling the spatial size to obtain a B×1024×40×40 feature map; concatenating the feature map with the B×512×40×40 feature map output by step a11 in the channel dimension to obtain a B×1536×40×40 feature map;

[0155] Step b2: use FasterBlock 3 times for processing, the number of output channels is 512, the spatial size remains unchanged, and the output feature map is B×512×40×40;

[0156] Step b3: Use the MLCA module to perform local and global attention fusion, keep the number of output channels at 512, and keep the spatial size at 40×40;

[0157] Step b4, upsampling to double the spatial size and output a feature map B×512×80×80; the feature map and the B×256×80×80 feature map output in step a7 are concatenated in the channel dimension to obtain a B×768×80×80 feature map;

[0158] Step b5, use FasterBlock 3 times to further process the feature map, the number of output channels is 256, and the spatial size is kept at 80×80;

[0159] Step b6: Use the MLCA module to perform local and global attention fusion, keep the number of output channels at 256, and keep the spatial size at 80×80;

[0160] Step b7, upsampling to double the spatial size to obtain a feature map B×256×160×160; the feature map and the feature map B×128×160×160 output in step a3 are concatenated in the channel dimension to obtain a feature map B×384×160×160;

[0161] Step b8, further processed by FasterBlock 3 times, the number of output channels is 128, and the spatial size remains at 160×160;

[0162] Step b9: Use the MLCA module to perform local and global attention fusion on the feature map, keep the number of output channels at 128, and keep the spatial size at 160×160;

[0163] Step b10, use convolution operation with stride 1 and convolution kernel size 3×3, the spatial size of the feature map remains unchanged, and the number of channels is 128;

[0164] Step b11, after SPDConv processing, the number of channels of the feature map increases to 256, and the spatial size is halved; output a B×256×80×80 feature map; the feature map and the B×256×80×80 feature map output in step b6 are spliced ​​in the channel dimension to obtain a B×512×80×80 feature map;

[0165] Step b12, after three times of FasterBlock further processing, the number of output channels is 256 and the spatial size remains 80×80;

[0166] Step b13, use the MLCA module to perform local and global attention fusion on the feature map, keep the number of output channels at 256, and keep the spatial size at 80×80;

[0167] Step b14, using convolution operation, the spatial size of the feature map remains 80×80 and the number of channels is 256;

[0168] Step b15: After SPDConv processing, the number of channels is increased from 256 to 512, and the spatial size is reduced to output a B×512×40×40 feature map; the feature map is concatenated with the B×512×40×40 feature map outputted in step b3 in the channel dimension to obtain a B×1024×40×40 feature map;

[0169] Step b16, after three times of FasterBlock further processing, the number of output channels is 512, and the spatial size remains at 40×40;

[0170] Step b17, use the MLCA module to perform local and global attention fusion on the feature map, keep the number of output channels at 512, and keep the spatial size at 40×40;

[0171] Step b18, using convolution operation, the spatial size of the feature map remains 40×40 and the number of channels is 512;

[0172] Step b19, after SPDConv processing, the number of channels is increased from 512 to 1024, and the spatial size is reduced, and a B×1024×20×20 feature map is output; the feature map and the B×1024×20×20 feature map output by step a16 are spliced ​​in the channel dimension to obtain a B×2048×20×20 feature map;

[0173] Step b20, after three times of FasterBlock further processing, the number of output channels is 1024, and the spatial size remains at 20×20;

[0174] Step b21, use the MLCA module to perform local and global attention fusion on the feature map, keep the number of output channels at 1024, and keep the spatial size at 20×20;

[0175] Step 3: Input the training set images and corresponding image labels into the established traffic sign detection model YOLO-SFM for training until convergence; specifically:

[0176] Set the network hyperparameters of the traffic sign detection model YOLO-SFM: optimizer optimizer is SGD, initial learning rate 1r0 is 0.01, momentum momentum is 0.937, weight attenuation weight_decay is 0.0005, and batch size batchsize is 16.

[0177] The training set images and the corresponding image labels are input into the traffic sign detection model YOLO-SFM, the loss function is calculated and the gradient is back-propagated, and the optimizer updates the parameters until the training set loss function and the validation set function no longer decrease, and the output evaluation index no longer increases. At this time, the model converges and the model parameters are the final trained model parameters.

[0178] Precision (P), recall (R), F1 score, mean average precision (mAP) are used. Precision is used to evaluate the accuracy of model detection, and recall represents the ratio of detected objects to original objects. mAP is the average value of AP, where AP is the area between the PR curve and the horizontal and vertical axes. F1 score reflects the balance between P and R, and can be a good measure of the detection performance of the model.

[0179] The following is the evaluation index expression:

[0180]

[0181]

[0182] Among them, P represents precision, R represents recall, AP represents average precision, mAP represents the average precision of all categories, TP represents the number of positive samples correctly identified as positive samples, FP represents the number of negative samples incorrectly identified as positive samples, and FN represents the number of positive samples incorrectly identified as negative samples; i represents the current category, AP represents the average precision of all categories, and mAP represents the average precision of all categories. i represents the average precision of the current category, and n represents the total number of categories.

[0183] Step 4: Input the test set image into the trained traffic sign detection model YOLO-SFM to generate the predicted bounding box B pred =(X min ,Y min ,X max ,Y max ) and the corresponding class probability P pred According to the set confidence threshold τ, the prediction boxes with high confidence are screened out:

[0184] B final ={B|P pred >τ}

[0185] The detection results include the category label, bounding box coordinates, and confidence score of the traffic sign, which are output to the terminal or stored in a file.

[0186] Example 3

[0187] The divided test set is input into the final traffic sign detection model YOLO-SFM, and the precision P, recall R and average precision mAP of the final model for traffic sign detection are obtained. These three evaluation indicators are compared with the evaluation indicators of the original model YOL0v8n to prove the effectiveness of YOLO-SFM.

[0188] Through the above steps, the method of the present invention realizes high-precision detection of traffic signs under glare conditions, and is suitable for actual scene applications of autonomous driving and intelligent transportation systems.

[0189] Example 4

[0190] The traffic sign image under glare is input into the trained traffic sign detection model YOLO-SFM, which outputs the location information, category and confidence information of the traffic sign. Figure 5 is the ground truth traffic sign detection result map, Figure 6 and Figure 7 This is a comparison chart of the effects of YOLO-SFM and original YOLOv8n on traffic sign detection under glare, which shows that YOLO-SFM has improved the detection accuracy of traffic signs under glare.

[0191] Example 5

[0192] As shown in Table 1, this is a data comparison table of the traffic sign detection model YOLO-SFM and the original YOLOv8n on the GTSD dataset. The improved model reduces the number of parameters by 0.3M, and the mAP@0.5 and mAP@0.5:0.95 are 13% and 10.6% higher than the original model, respectively. The precision P and recall R are also improved by 5.6% and 8.5%, respectively. It can be seen that the traffic sign detection model YOLO-SFM has achieved better results.

[0193] Model Dataset param(M) Accuracy (%) Recall rate (%) map.5(%) map.5:.95(%) yolov8n GTSD 3.0 79.3 58.2 63.0 43.0 The present invention GTSD 2.7 84.9 66.7 76.0 53.6

[0194] Example 6

[0195] The present invention proposes a lightweight model YOLO-SFM for glare scenes, which uses SPD-Conv downsampling to reduce information loss and improve feature extraction accuracy. In order to reduce redundant calculations and memory accesses and improve detection speed, YOLO-SFM introduces FasterNet. MLCA combines local and global features as well as channel and spatial information to improve detection speed. Finally, the detection layer is adjusted, and the small target detection layer is added while retaining other detection layers, which improves the generalization ability of the model. YOLO-SFM significantly improves the ability to detect traffic signs under strong light interference conditions.

Claims

1. A traffic sign detection method based on a glare environment, characterized in that: Follow the steps below to implement it: Step 1, data preprocessing: Mosaic data enhancement technology is used to splice multiple images in the data set to generate new image samples, and the data set is divided into a training set and a test set; Step 2: Design the traffic sign detection model YOLO-SFM, which includes the Backbone module and the Head module; Step 3: Input the training set images and corresponding image labels into the established traffic sign detection model YOLO-SFM for training until convergence; Step 4: Input the test set images into the trained traffic sign detection model YOLO-SFM and output the detection results, which include the category label, bounding box coordinates, and confidence score of the traffic sign.

2. The method for detecting traffic signs in a glare environment according to claim 1, characterized in that: In the step 1, specifically: Step 1.1, randomly select four images from the data set, denoted as I1, I2, I3, I4, where the size of each image is W×H; then scale the images: scale the selected images to new sizes according to the ratios α and β; then perform image stitching and place the four images on a blank image I Mosaic In the four quadrants, determine the position and size of each quadrant according to the following formula; First quadrant: I1→[0,αW)×[0,βH) Second quadrant: I2→[αW,W)×[0,βH) The third quadrant: I3→[0,αW)×[βH,H) The fourth quadrant: I4→[αW,W)×[βH,H) At the same time, adjust the annotation box: For each original image, the annotation box B = (X min ,Y min ,X max ,Y max ), according to the image scaling and stitching position, adjust to the new annotation box B′=(X′ min ,Y′ min ,X′ max ,Y′ max ), the adjustment formula is as follows; x′=αx+x offset ,y′=βx+y offset where x offset ,y offset is the offset of the stitched image; Step 1.2: The stitched image I Mosaic Normalization is performed to ensure that the pixel values ​​are within the range required for model training, and all annotation boxes are recorded in a new annotation file; the final generated Mosaic image is divided into a training set and a test set.

3. The method for detecting traffic signs in a glare environment according to claim 1, characterized in that: In the step 2, specifically: Step a, use Backbone module to process the image; In step b, the Head module is used to fuse the features of different scales extracted by the Backbone. These features are fused through further convolution operations to meet the detection requirements of targets of different sizes.

4. The method for detecting traffic signs in a glare environment according to claim 3, characterized in that: In the step a, specifically: Step a1, the training set Mosaic image in step 1 passes through a standard convolution layer, the convolution layer uses a convolution kernel of size 3 for downsampling, increases the number of channels to keep the feature map size unchanged, the convolution layer stride is 1, and the output channel number is 64; then use SPDConv to process the feature map; pass through the convolution layer again, the convolution kernel size is 3, the stride is 1, and the output channel number is still 128; finally, use SPDConv to process the feature map again, and the final output feature map size is B×128×160×160; The specific process of SPDConv processing feature maps is as follows: the input feature map size is B×64×640×640, where B is the batch size, 64 is the number of input channels, and 640×640 is the spatial scale. Each 2×2 pixel block is rearranged into a channel dimension through SPD rearrangement, and the number of channels increases from 64 to 64×4=256, and the spatial size is halved; the rearranged feature map X′ is subjected to standard convolution, the convolution kernel size is 1×1, the stride s=1, the convolution output is batch normalized, and then the SiLU activation function is applied for processing, and the final output feature map size is B×128×320×320; at this point, the SPD convolution processing is completed; Step a2, process the feature map through three FasterBlocks; Specifically, the input feature map is divided into two 64-channel blocks according to the channel dimension. Each block will be processed by FasterBlock, that is, for each block, its shape remains B×64×160×160, and the output feature map shape is still B×64×160×160. The two processed blocks will be spliced ​​in the channel dimension, that is, the two parts of the feature map will be spliced ​​into a feature map of B×128×160×160; finally, through a 1×1 convolution, the final output feature map is B×128×160×160; Step a3, processing the feature map obtained in step a2 through the MLCA module; Specifically, the feature map is subjected to adaptive average pooling to obtain a local feature map, the local feature map is further pooled to obtain a global feature map, the local features are flattened and transformed through 1D convolution, and the convolution layer is used for feature extraction; after flattening the global features, 1D convolution is performed and the convolution layer is used for feature extraction, and the local and global features are subjected to sigmoid activation function to generate local attention maps and global attention maps; a weighted fusion strategy is used to combine the local and global attention maps; the input feature map is element-wise multiplied with the fused attention map, and the final feature map is output, and the output feature map is still B×128×160×160; Step a4, the feature map obtained in step a3 is processed by a convolution layer, the stride of the convolution layer is 1, the number of output channels is 256, and the spatial size remains unchanged; Step a5, use the SPDConv layer to process the feature map. The SPDConv layer reduces the spatial size by half through space-to-depth conversion, increases the number of channels by 4 times, and then reduces the dimension back to 256 channels through convolution. The size of the output feature map is B×256×80×80; Step a6, use FasterBlock 6 times to process the feature map in step a5. The FasterBlock module extracts and enhances features, keeping the shape of the feature map unchanged; Step a7, using the MLCA module to optimize the feature map in step a6, and the shape of the final output feature map is B×256×80×80; Step a8: pass the feature map obtained in step a7 through the convolution layer, increase the number of channels to 512, and the stride of the convolution layer is 1; Step a9, use SPDConv to process the feature map of 512 channels, and the final output feature map size is B×512×40×40; Step a10, use FasterBlock 6 times to process the feature map in step a9, keep the number of channels as 512 each time, and concatenate them to finally output the feature map B×512×40×40; Step a11, use the MLCA module to optimize the feature map in step a10, generate and fuse local and global attention maps, and output a feature map B×512×40×40; Step a12, passing the feature map obtained in step a11 through a convolutional layer, the stride of the convolutional layer is 1, and the number of output channels is increased to 1024; Step a13, use SPDConv to process the feature map; Step a14, use FasterBlock 3 times to process the feature map in step a13, keeping the number of output channels at 1024; Step a15, using the MLCA module to optimize the feature map in step a14; Step a16, extract multi-scale features from the feature map in step a15 through a maximum pooling operation, and the final output feature map size is maintained at B×1024×20×20.

5. The method for detecting traffic signs in a glare environment according to claim 4, characterized in that: In the step b, specifically: Step b1, upsampling, doubling the spatial size to obtain a B×1024×40×40 feature map; concatenating the feature map with the B×512×40×40 feature map output by step a11 in the channel dimension to obtain a B×1536×40×40 feature map; Step b2: use FasterBlock 3 times for processing, the number of output channels is 512, the spatial size remains unchanged, and the output feature map is B×512×40×40; Step b3: Use the MLCA module to perform local and global attention fusion, keep the number of output channels at 512, and keep the spatial size at 40×40; Step b4, upsampling to double the spatial size and output a feature map B×512×80×80; the feature map and the B×256×80×80 feature map output in step a7 are concatenated in the channel dimension to obtain a B×768×80×80 feature map; Step b5, use FasterBlock 3 times to further process the feature map, the number of output channels is 256, and the spatial size is kept at 80×80; Step b6: Use the MLCA module to perform local and global attention fusion, keep the number of output channels at 256, and keep the spatial size at 80×80; Step b7, upsampling to double the spatial size to obtain a feature map B×256×160×160; the feature map and the feature map B×128×160×160 output in step a3 are concatenated in the channel dimension to obtain a feature map B×384×160×160; Step b8, further processed by FasterBlock 3 times, the number of output channels is 128, and the spatial size remains at 160×160; Step b9: Use the MLCA module to perform local and global attention fusion on the feature map, keep the number of output channels at 128, and keep the spatial size at 160×160; Step b10, use convolution operation with stride 1 and convolution kernel size 3×3, the spatial size of the feature map remains unchanged, and the number of channels is 128; Step b11, after SPDConv processing, the number of channels of the feature map increases to 256, and the spatial size is halved; output a B×256×80×80 feature map; the feature map and the B×256×80×80 feature map output in step b6 are spliced ​​in the channel dimension to obtain a B×512×80×80 feature map; Step b12, after three times of FasterBlock further processing, the number of output channels is 256 and the spatial size remains 80×80; Step b13, use the MLCA module to perform local and global attention fusion on the feature map, keep the number of output channels at 256, and keep the spatial size at 80×80; Step b14, using convolution operation, the spatial size of the feature map remains 80×80 and the number of channels is 256; Step b15: After SPDConv processing, the number of channels is increased from 256 to 512, and the spatial size is reduced to output a B×512×40×40 feature map; the feature map is concatenated with the B×512×40×40 feature map outputted in step b3 in the channel dimension to obtain a B×1024×40×40 feature map; Step b16, after three times of FasterBlock further processing, the number of output channels is 512, and the spatial size remains at 40×40; Step b17, use the MLCA module to perform local and global attention fusion on the feature map, keep the number of output channels at 512, and keep the spatial size at 40×40; Step b18, using convolution operation, the spatial size of the feature map remains 40×40 and the number of channels is 512; Step b19, after SPDConv processing, the number of channels is increased from 512 to 1024, and the spatial size is reduced, and a B×1024×20×20 feature map is output; the feature map and the B×1024×20×20 feature map output by step a16 are spliced ​​in the channel dimension to obtain a B×2048×20×20 feature map; Step b20, after three times of FasterBlock further processing, the number of output channels is 1024, and the spatial size remains at 20×20; In step b21, the MLCA module is used to perform local and global attention fusion on the feature map, and the number of output channels is kept at 1024 and the spatial size is kept at 20×20.

6. The method for detecting traffic signs in a glare environment according to claim 5, characterized in that: In the step 3, specifically: Set the network hyperparameters of the traffic sign detection model YOLO-SFM: optimizer optimizer is SGD, initial learning rate 1r0 is 0.01, momentum momentum is 0.937, weight decay weight_decay is 0.0005, batch size batch size is 16; The training set images and the corresponding image labels are input into the traffic sign detection model YOLO-SFM, the loss function is calculated and the gradient is back-propagated, and the optimizer updates the parameters until the training set loss function and the validation set function no longer decrease, and the output evaluation index no longer increases. At this time, the model converges and the model parameters are the final trained model parameters.

7. The method for detecting traffic signs in a glare environment according to claim 6, characterized in that: The accuracy of model detection is evaluated using precision P, recall rate R, F1 score, and mean average precision mAP. The following is the expression of the evaluation index: Among them, P represents precision, R represents recall, AP represents average precision, mAP represents the average precision of all categories, TP represents the number of positive samples correctly identified as positive samples, FP represents the number of negative samples incorrectly identified as positive samples, and FN represents the number of positive samples incorrectly identified as negative samples; i represents the current category, AP represents the average precision of all categories, and mAP represents the average precision of all categories. i represents the average precision of the current category, and n represents the total number of categories.

8. The method for detecting traffic signs in a glare environment according to claim 7, characterized in that: In the step 4, specifically: Input the test set image into the trained traffic sign detection model YOLO-SFM to generate the predicted bounding box B pred =(X min ,Y min ,X max ,Y max ) and the corresponding class probability P pred ; According to the set confidence threshold τ, filter out the prediction boxes with high confidence: B final ={B|P pred >t} The detection results include the category label, bounding box coordinates, and confidence score of the traffic sign, which are output to the terminal or stored in a file.

Images