A Traffic Facility Detection Method Based on Improved YOLOV8n

Abstract

This invention relates to a traffic facility detection method based on an improved YOLOv8n, comprising the following steps: obtaining a traffic facility dataset through raw data acquisition and processing; constructing training and testing sets; improving the YOLOv8n model by proposing a multi-scale deformable enhancement residual module C2f-MSDERM, which is applied to the backbone network; wherein the C2f-MSDERM module contains two sub-modules: one sub-module is a multi-scale deformable convolutional block MSDConvBlock; the other sub-module is a TripletAttention module; establishing a residual connection between the sequential multi-scale deformable convolutional block MSDConvBlock and the TripletAttention module, that is, adding the enhanced feature map Z obtained by the TripletAttention module to the feature map X1 of the input MSDConvBlock to retain the original feature information and promote gradient propagation; the output of the C2f-MSDERM module is processed by the SPPF module to obtain the feature map extracted by the backbone network.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a traffic facility detection method based on an improved YOLOv8n. Background Technology

[0002] Traffic facilities are a crucial part of road administration and an indispensable component of ensuring traffic safety. Traffic safety facilities play a vital role in driving safety and reducing potential accidents. For example, placing crash barriers at U-turns can prevent scratches; installing guardrails on double solid lines can prevent vehicles from traveling in the opposite direction; installing crash barriers on highway curves provides protection and control when vehicles are exiting the road; and protective nets ensure that vehicles are not disturbed by road debris at high speeds. Maintaining traffic facilities is essential for ensuring our traffic safety. Common traffic facilities include anti-glare panels, reflective cones, and anti-throw nets. Road administration departments need to expend considerable manpower and resources to maintain their normal operation. Identifying abnormal traffic facilities through inspections is the first step in maintenance work. With the development of technology in recent years, automatic inspection has been put into practical application in scenarios such as drone power line inspection. However, traffic facilities are still mainly inspected manually. Due to their variety and complexity and wide geographical distribution, manual inspection requires a lot of time and manpower to compare databases and maps, resulting in low inspection efficiency. Relevant departments urgently need a vehicle-mounted device that can automatically detect traffic facilities in real-time road conditions and realize functions such as automatic database comparison and automatic inspection. However, datasets and algorithms are the two major challenges for vehicle-mounted devices to achieve real-time traffic facility inspection.

[0003] In terms of datasets, with the development of object detection and autonomous driving, a large number of traffic sign and marking datasets have emerged. For example, the US dataset BDD100K covers different times, weather conditions (including sunny, cloudy, and rainy days, as well as different times of day and night), and driving scenarios. The dataset is geographically distributed in New York, Berkeley, San Francisco, and other locations. In this dataset, road object detection involves annotating 2D bounding boxes on 100,000 images of buses, traffic lights, traffic signs, people, bicycles, trucks, motorcycles, cars, trains, and passengers. The German traffic sign dataset GTSDB contains 900 images of 1360×800 pixels, divided into four categories (prohibited, dangerous, mandatory, and other). The Chinese traffic sign dataset CCTSDB replaces many of the original, easily detectable images with more challenging samples to adapt to complex and changing detection environments. The commonly used domestic public dataset TT100K has relatively rich data resources, subdividing over one hundred traffic sign types, but lacks annotations for traffic facilities.

[0004] In terms of algorithms, with the development of deep learning and the progress of the autonomous driving industry, traffic sign detection based on deep learning, similar to traffic facility detection, has also made great strides. Liu et al. (Liu Y, Peng J, Xue JH, et al. TSingNet: Scale-aware and context-rich feature learning for traffic sign detection and recognition in the wild[J]. Neurocomputing, 2021, 447: 10-22.) proposed a new traffic sign detection network, TSingNet. By constructing an attention-driven bidirectional feature pyramid network, it transmits foreground semantic information bidirectionally at multiple scales through scale self-attention learning. At the same time, it introduces an adaptive receptive field fusion block with a variable expansion rate, which effectively improves the detection accuracy of occluded targets. Yamamoto M et al. (Yamamoto M, Sultana R, Ohashi G. Nighttime Traffic Sign and Pedestrian Detection Using RefineDet with Time-Series Information[J]. IEEJ Transactions on Electrical and Electronic Engineering, 2023, 18(3): 408-417.) proposed a target detection model that combines the RefineDet small object detection model with the TSSD video detection model, which can improve the detection accuracy in nighttime driving scenarios. Han et al. (Han C, Gao G, Zhang Y. Real-time small traffic sign detection with revised faster-RCNN[J]. Multimedia Tools and Applications, 2019, 78: 13263-13278.) proposed a real-time small traffic sign detection method based on an improved Faster-RCNN. Considering that the stride of the generator is too large, some layers were removed, and dilated convolutions were introduced to improve the detection accuracy of small traffic signs.

[0005] While significant progress has been made in traffic sign detection research, research and applications in traffic facility detection remain insufficient. The challenges in this field lie not only in the lack of relevant datasets and the diverse types and forms of traffic facilities, but also in their complexity in real-world scenarios. Factors such as occlusion, changes in lighting, and changes in viewing angle can severely impact detection accuracy. Therefore, strengthening research and application in traffic facility detection is of great significance for promoting the further maturity and commercialization of autonomous driving technology. Summary of the Invention

[0006] To address the above problems, this invention proposes a traffic facility detection method based on an improved YOLOv8n. The technical solution is as follows:

[0007] A traffic facility detection method based on an improved YOLOv8n includes the following steps:

[0008] Step 1: After collecting and processing the raw data, a traffic facility dataset is obtained;

[0009] Step 2: Divide the dataset obtained in Step 1 to construct the training set and the test set;

[0010] Step 3: Improve the YOLOv8n model by proposing a multi-scale deformable augmentation residual module C2f-MSDERM, which is applied to the backbone network. The C2f-MSDERM module comprises two sub-modules: one is a multi-scale deformable convolutional block MSDConvBlock, which dynamically extracts features of targets of different scales and shapes from the input feature map X1, resulting in a multi-scale feature map Y; the other is a TripletAttention module, which captures cross-dimensional attention weights through a three-branch structure to augment the features extracted by MSDConvBlock. The enhanced feature map Z is obtained. A residual connection is established between the serial multi-scale deformable convolutional block MSDConvBlock and the TripletAttention module. That is, the enhanced feature map Z obtained by the TripletAttention module is added to the feature map X1 of the input MSDConvBlock to preserve the original feature information and promote gradient propagation. The output of the C2f-MSDERM module is processed by the SPPF module to obtain the feature map extracted by the backbone network. The feature map extracted by the backbone network is input into the Neck network for multi-scale feature fusion, and then into the Head network to complete target detection and classification.

[0011] Step 4: The CIoU loss function is used to introduce the aspect ratio between the predicted bounding box and the ground truth bounding box; this allows the predicted box to more accurately match the shape and size of the ground truth box, improving the accuracy of target localization.

[0012] Step 5: Train and test the model to obtain an optimized network model.

[0013] Furthermore, in step 1, the processing of the raw data is divided into three steps: video cropping and frame extraction, manual initial screening, and annotation. In the annotation step, traffic facility targets in the images are annotated. The dataset includes 10 types of traffic facility targets, such as city road signs, highway signs, large warning signs, small directional signs, mileage markers, linear guidance signs, reflective barrels, anti-glare panels, urban metal guardrails, and bridge sound insulation or anti-throw nets. The urban metal guardrails and bridge sound insulation or anti-throw nets are annotated using polygon annotation, while the remaining eight targets are annotated using rectangular boxes.

[0014] Furthermore, step 3 specifically includes:

[0015] Step 3.1: The input image passes through several convolutional layers of the backbone network and the C2f module to extract features, resulting in feature map X. Feature map X is then input into the C2f-MSDERM module, first passing through a 1×1 convolutional layer. Then, a split operation is performed to split the channels of the feature map to obtain feature maps X1 and X2. The part of feature map X1 output by the split operation is then fed into the subsequent multi-scale deformable convolutional block MSDConvBlock to extract features.

[0016] Step 3.2: The partial feature map X1 output from Split enters the multi-scale deformable convolutional block MSDConvBlock. A 3×3 convolutional layer is used to perform preliminary feature extraction on the input feature map X1, reducing the number of output channels to half the number of input channels to obtain feature map a. Feature map a is then passed through parallel deformable convolutional layers, with each layer simultaneously extracting features through three parallel deformable convolutional DCN module branches. Feature map a is passed through the first parallel branch, i.e., a 1×1 dynamic convolutional layer DCN, to obtain output b. Feature map a is passed through the second parallel branch, i.e., a 3×3 dynamic convolutional layer DCN, to obtain output c. Feature map a is passed through the third parallel branch, i.e., a 5×5 dynamic convolutional layer DCN, to obtain output d. The outputs {b, c, d} of the three parallel branches are then fused through a Concat operation. The fused output is e, which is passed through a 1×1 convolutional layer to output the multi-scale feature map Y. C×H×W ;

[0017] Step 3.3: The multi-scale feature map Y output by the deformable convolutional block MSDConvBlock is input to the TripletAttention module, which further enhances the features to obtain the enhanced feature map Z. The TripletAttention module captures cross-dimensional attention weights through a three-branch structure to enhance the feature map.

[0018] Step 3.4: After the enhanced feature map Z output by the TripletAttention module, establish a residual connection between the enhanced feature map Z and the input feature map X1 of MSDConvBlock; add the enhanced feature map Z and the input feature map X1 to obtain the output feature map U, so as to preserve the original feature information and promote gradient propagation;

[0019] Step 3.5: Perform a concat operation on feature map U and part of feature map X2 from the Split output, concatenating them along the channel dimension. After concatenation, pass through a 1×1 convolutional layer to obtain the final output of the C2f-MSDERM module. The output of the C2f-MSDERM module is processed by the SPPF module to obtain the feature map extracted by the backbone network.

[0020] Step 3.6: Input the feature map extracted by the backbone network into the Neck network for multi-scale feature fusion, and then input it into the Head network to complete the final target detection and classification.

[0021] Furthermore, the TripletAttention module comprises three key branches, each focusing on different dimensions of feature interaction to enhance feature representation capabilities:

[0022] a) Channel attention branch will apply multi-scale feature map Y C×H×W The C-dimensional tensor is reduced to 2-dimensional using Z-Pool, and the average pooling and max pooling features in the C-dimensional region are concatenated. Then, convolution is applied to reduce the C-dimensional tensor to 1-dimensional, followed by batch normalization layers. Finally, a sigmoid activation function is used to generate attention weights S1, which are then compared with the input feature map Y. C×H×W Multiplying them yields the output Z1 of the first critical branch;

[0023] b) The interaction branch between the channel and the spatial W dimension, adjusting the multi-scale feature map Y through the Permute operation. C×H×W The dimensional order is changed to a new order Y: height H, number of channels C, and width W. H×C×WSubsequently, Z-Pool is applied along the height H dimension to reduce the H-dimensional tensor to 2-dimensionality. The average pooling and max pooling features along the H dimension are then concatenated. Convolution is then applied to reduce the H-dimensional tensor to 1-dimensionality. A batch normalization layer is then applied, and finally, a sigmoid activation function is used to generate attention weights S2. These attention weights S2 are then compared with the feature map Y. H×C×W Multiply the results and then pass them through Permute to restore the original dimensional order, resulting in the output Z2 of the second critical branch;

[0024] c) The interaction branch between the channel and the spatial H dimension, adjusting the multi-scale feature map Y through the Permute operation. C×H×W The dimensional order is changed to a new order Y: width W, height H, and number of channels C. W×H×C Then, Z-Pool is applied along the width W dimension to reduce the W-dimensional tensor to 2-dimensionality. The average pooling features and max pooling features along the W dimension are concatenated, and then convolution is applied to reduce the W-dimensional tensor to 1-dimensionality. A batch normalization layer is then applied, and finally, a sigmoid activation function is used to generate attention weights S3. Attention weights S3 are then compared with the feature map Y. W×H×C Multiply the results and then pass them through Permute to restore the original dimensional order, resulting in the output Z3 of the third critical branch;

[0025] d) Add the output features {Z1, Z2, Z3} of the three branches element by element and take the average value to obtain the enhanced feature map Z.

[0026] Furthermore, the specific formula for the CIoU loss function is as follows:

[0027]

[0028] w and h are the width and height of the prediction box, respectively; w gt h gt These are the width and height of the actual bounding box, respectively; ρ 2 c is the square of the distance between the center point of the predicted bounding box and the center point of the ground truth bounding box. 2 α is the square of the diagonal length of the smallest rectangle that can contain both the predicted and ground truth boxes; where α is a parameter used for tradeoffs, and v is a parameter used to measure the consistency of aspect ratio; the corresponding CIoU loss function is as follows:

[0029]

[0030] This invention presents a traffic facility detection method based on an improved YOLOv8n. It proposes a C2f-MSDERM module to enhance the backbone network. By combining the advantages of multi-scale feature extraction and deformable convolution, a multi-scale deformable convolutional block MSDConvBlock is introduced to improve the model's ability to detect multi-scale and irregularly shaped targets. Simultaneously, a TripletAttention module is introduced within the module, capturing cross-dimensional attention weights through a three-branch structure to enhance the feature map with extremely low computational cost. Finally, a CIoU loss function is employed, enabling the predicted bounding box to more accurately match the shape and size of the ground truth bounding box, thereby improving the accuracy of traffic facility detection. Attached Figure Description

[0031] Figure 1 For traffic facility detection networks based on improved YOLOv8n;

[0032] Figure 2 This is a schematic diagram of the C2f-MSDERM module structure;

[0033] Figure 3 This is a schematic diagram of the structure of a multi-scale deformable convolutional block (MSDConvBlock).

[0034] Figure 4 This is a schematic diagram of the TripletAttention module structure. Detailed Implementation

[0035] The present invention will now be described in conjunction with the accompanying drawings and embodiments. The specific steps of the present invention are as follows.

[0036] Step 1: Data Acquisition and Processing. This includes: Raw data acquisition, which involved collecting a large amount of real driving video data under various weather and lighting conditions, from urban areas to mountainous regions, using IMX477 high-definition CSI cameras, industrial cameras, and Hikvision dashcams. This video data underwent cropping and frame extraction, initial manual screening, and detailed annotation (including polygon and rectangular bounding box annotation methods), ultimately forming a traffic facility dataset containing 4477 high-definition images, covering 10 key traffic facility targets.

[0037] Step 2: Divide the traffic facility dataset into training set and test set according to a certain ratio.

[0038] Step 3: Construct the network model architecture, including proposing the C2f-MSDERM module to improve the backbone network. Specifically, combining the advantages of multi-scale feature extraction and deformable convolution, a multi-scale deformable convolutional block MSDConvBlock is proposed; simultaneously, a TripletAttention module is introduced within the module.

[0039] Step 3.1: Propose and adopt the C2f-MSDERM module to improve the backbone network. Feature map X is input into the C2f-MSDERM module, first passing through a 1×1 convolutional layer; then, a split operation is used to split the channels of the feature map to obtain feature maps X1 and X2. A portion of feature map X1 output from the split operation is fed into the subsequent multi-scale deformable convolutional block MSDConvBlock to extract features, resulting in a multi-scale feature map Y. The multi-scale feature map Y then passes through a serial TripletAttention module to obtain an enhanced feature map Z. The enhanced feature map Z then establishes a residual connection with the input feature map X1 of MSDConvBlock. The enhanced feature map Z is added to the input feature map X1 to obtain the output feature map U. The output feature map U is channel-fused with the split feature map X2 through a Concat operation, and the final output of the C2f-MSDERM module is completed through a 1×1 convolutional layer.

[0040] Step 3.2: Input the feature map extracted by the backbone network into the Neck network for multi-scale feature fusion, and then input it into the Head network to complete the final target detection and classification.

[0041] Step 4: Employ the CIoU loss function. CIoU incorporates the aspect ratio between the predicted bounding box and the ground truth bounding box. This allows the predicted box to more accurately match the shape and size of the ground truth box, thereby improving the accuracy of target localization.

[0042] Step 5: Train and test the model to obtain the network model with the best performance.

[0043] The present invention will now be described in detail with reference to the accompanying drawings and embodiments, such as... Figure 1 As shown, the specific implementation steps of the present invention include:

[0044] Step 1: Raw Data Acquisition and Processing. Raw data acquisition includes: one part using an IMX477 high-definition CSI camera paired with an industrial camera to collect real-world driving video data within urban areas, with a resolution of 4032×3040; the other part using a dashcam to collect traffic facilities in multiple cities, covering various actual road conditions including urban roads, highways, mountain national roads, and side streets. The dashcam model used was a Hikvision AE-DC5313-C6, with a resolution of 1306×984. To facilitate subsequent processing, all acquired data maintained the same aspect ratio. To ensure the model can be applied to real-world driving scenarios, the acquired data included various weather and lighting conditions such as sunny daytime, foggy daytime, foggy nighttime, sunny nighttime, and rainy daytime.

[0045] The processing of the raw data consisted of three steps: video cropping and frame extraction, manual initial screening, and annotation. First, key areas were extracted from the collected raw videos, and the data was converted into image data at a rate of one frame per second. Then, manual initial screening removed duplicate and non-target images. Finally, software was used to annotate the traffic facility targets in the images. The dataset included 10 types of traffic facility targets: city road signs, highway signs, large warning signs, small directional signs, mileage markers, linear guidance signs, reflective cones, anti-glare panels, urban metal guardrails, and bridge sound barriers or anti-throw nets. To ensure the annotations closely approximate the shapes of the real targets, polygonal annotation was used for urban metal guardrails and bridge sound barriers or anti-throw nets, while rectangular bounding boxes were used for the other eight types of targets. The final result was a high-resolution dataset of 4477 images of traffic facilities.

[0046] Step 2: Divide the traffic facility dataset into two parts: a training set and a test set. The training set contains 3805 images, and the test set contains 672 images.

[0047] Step 3: As Figure 1 , Figure 2 , Figure 3 , Figure 4 As shown, the traffic facility detection method based on the improved YOLOv8n includes the following steps:

[0048] Step 3.1: The YOLOv8n network structure is mainly divided into three parts: the backbone network, the neck network, and the head network. In the backbone network, the C2f module is used to improve computational efficiency while reducing redundant parameters, achieving both lightweight design and excellent feature extraction capabilities. In the neck network, a PAN network structure is adopted to fuse feature maps from different stages of the backbone network, enhancing feature representation capabilities. However, YOLOv8n still has significant shortcomings in detecting diverse and varied traffic facility targets in complex traffic scenarios, especially under conditions of occlusion, lighting changes, and viewing angle changes.

[0049] Step 3.2: To address the above problems, this invention proposes the C2f-MSDERM module to improve the backbone network. The C2f-MSDERM module comprises two main sub-modules: one is a multi-scale deformable convolutional block MSDConvBlock, where the input feature map X1 is passed through MSDConvBlock to dynamically extract features of targets of different scales and shapes, resulting in a multi-scale feature map Y. The other sub-module is a TripletAttention module, which captures cross-dimensional attention weights through a three-branch structure, enhancing the features extracted by MSDConvBlock with extremely low computational cost, resulting in an enhanced feature map Z. Finally, a residual connection is established between the sequential multi-scale deformable convolutional block MSDConvBlock and the TripletAttention module, that is, the feature map Z enhanced by the TripletAttention module is added to the input feature map X1 of MSDConvBlock to preserve the original feature information and promote gradient propagation.

[0050] Step 3.3: The input image passes through several convolutional layers preceding the backbone network and the C2f module to extract features, resulting in feature map X. Feature map X is then input into the C2f-MSDERM module, first passing through a 1×1 convolutional layer; then, a split operation is performed to split the channels of the feature map, resulting in feature maps X1 and X2. The portion of feature map X1 output from the split operation is then fed into the subsequent multi-scale deformable convolutional block MSDConvBlock for feature extraction.

[0051] Step 3.4: The partial feature map X1 output from Split enters the multi-scale deformable convolutional block MSDConvBlock. First, a 3×3 convolutional layer is used to perform preliminary feature extraction on the input feature map X1, reducing the number of output channels to half the number of input channels to obtain feature map a. Feature map a is then processed by parallel deformable convolutional layers, with each layer simultaneously extracting features through three parallel deformable convolutional DCN module branches. Feature map a is processed through the first parallel branch, a 1×1 dynamic convolutional layer DCN, to obtain output b. It is then processed through the second parallel branch, a 3×3 dynamic convolutional layer DCN, to obtain output c. Finally, it is processed through the third parallel branch, a 5×5 dynamic convolutional layer DCN, to obtain output d. The outputs {b, c, d} of the three parallel branches are then fused using a Concat operation, resulting in the fused output e. Finally, e is processed through a 1×1 convolutional layer to output the multi-scale feature map Y.

[0052] Step 3.5: The multi-scale feature map Y output by the deformable convolutional block MSDConvBlock is input to the TripletAttention module, which further enhances the features to obtain the enhanced feature map Z.

[0053] The TripletAttention module captures cross-dimensional attention weights through a three-branch structure, enhancing feature maps with extremely low computational overhead. The TripletAttention module comprises three key branches, each focusing on a different feature interaction dimension to enhance feature representation capabilities:

[0054] a) Channel attention branch will apply multi-scale feature map Y C×H×W The C-dimensional tensor is reduced to 2-dimensional using Z-Pool, and the average pooling and max pooling features in the C-dimensional region are concatenated. Then, a 7×7 convolution is applied to reduce the C-dimensional tensor to 1-dimensional, followed by batch normalization layers. Finally, an attention weight S1 is generated using a sigmoid activation function. The attention weight S1 is then compared with the input feature map Y. C×H×W Multiplying them together yields the output Z1 of the first critical branch.

[0055] b) The interaction branch between the channel and the spatial W dimension, adjusting the multi-scale feature map Y through the Permute operation. C×H×W The dimensional order is changed to a new order Y: height H, number of channels C, and width W. H×C×W Then, Z-Pool is applied along the height H dimension to reduce the H-dimensional tensor to 2-dimensionality, concatenating the average pooling and max pooling features along the H dimension. A 7×7 convolution is then applied to reduce the H-dimensional tensor to 1-dimensionality. A batch normalization layer is then applied, and finally, a sigmoid activation function is used to generate attention weights S2. These attention weights S2 are then compared with the feature map Y. H×C×W Multiply the results and then pass them through Permute to restore the original dimensional order, resulting in the output Z2 of the second critical branch.

[0056] c) The interaction branch between the channel and the spatial H dimension, adjusting the multi-scale feature map Y through the Permute operation. C×H×W The dimensional order is changed to a new order Y: width W, height H, and number of channels C. W×H×C Then, a Z-Pool is applied along the width W dimension to reduce the W-dimensional tensor to 2-dimensionality. The average pooling features and max pooling features along the W dimension are concatenated. A 7×7 convolution is then applied to reduce the W-dimensional tensor to 1-dimensionality. A batch normalization layer is then applied, and finally, a sigmoid activation function is used to generate attention weights S3. These attention weights S3 are then compared with the feature map Y. W×H×CMultiply the results and then pass them through Permute to restore the original dimensional order, resulting in the output Z3 of the third critical branch.

[0057] d) Finally, the output features {Z1, Z2, Z3} of the three branches are summed element by element and the average value is taken to obtain the enhanced feature map Z.

[0058] Step 3.4: After obtaining the enhanced feature map Z output by the TripletAttention module, establish a residual connection between the enhanced feature map Z and the input feature map X1 of MSDConvBlock. Add the enhanced feature map Z to the input feature map X1 to obtain the output feature map U, so as to preserve the original feature information and promote gradient propagation.

[0059] Step 3.5: Perform a concat operation on feature map U and the partial feature map X2 output from Split, concatenating them along the channel dimension. After concatenation, pass the concatenated image through a 1×1 convolutional layer to obtain the final output of the C2f-MSDERM module. The output of the C2f-MSDERM module is then processed by the SPPF module to obtain the feature map extracted by the backbone network.

[0060] Step 3.6: Input the feature map extracted by the backbone network into the Neck network for multi-scale feature fusion, and then input it into the Head network to complete the final target detection and classification.

[0061] Step 4: Employ the CIoU loss function. CIoU incorporates the aspect ratio between the predicted bounding box and the ground truth bounding box. This allows the predicted box to more accurately match the shape and size of the ground truth box, thereby improving the accuracy of target localization.

[0062] Step 5: Train the improved YOLOv8n model. The training experimental platform used... The system was equipped with a Rayzen 5 3600 3.6GHz processor, 32GB of RAM, and a GeForce RTX 3090 GPU with 24GB of VRAM. The software environment consisted of Ubuntu 18.04.3 operating system and PyTorch 1.13.1 deep learning framework (Python 3.8). The SGD optimizer was used for training, with an initial learning rate of 0.01, SGD momentum of 0.937, weight decay of 0.0005, a batch size of 32, and 500 training epochs.

[0063] To verify the effectiveness of the model of this invention, while ensuring the consistency of the target detection network structure, this experiment added an improved C2f-MSDERM module to the backbone network. In order to enhance the backbone network's ability to extract traffic facility features, an ablation experiment was conducted to compare its performance with the original YOL0v8n model. The performance is shown in Table 1.

[0064] Table 1 Ablation Experiment Results

[0065]

[0066] The ablation experiment results show that the introduction of the multi-scale deformable convolutional block MSDConvBlock is beneficial to improving the model's feature extraction ability and increasing the detection accuracy of traffic facilities; the addition of the Triplet Attention mechanism further enhances the model's feature extraction ability and achieves the best detection accuracy for traffic facilities.

[0067] To verify the optimization effect of different loss functions on the improved Y0L0v8n network model, while ensuring the consistency of the object detection network structure and training hyperparameters, multiple loss functions, including GIoU, DIoU, EIoU, and SIoU, were changed to guide the model training. The model performance under different loss functions was compared, as shown in Table 2.

[0068] Table 2. Model results under different loss functions

[0069]

[0070] Based on the training results, training with the CIoU loss function only slightly lags behind DIoU in accuracy, but achieves optimal performance in recall and mAP. Therefore, it can be concluded that training with the CIoU loss function can effectively improve the model's detection capabilities.

[0071] This invention improves the YOLOv8n model by introducing the innovative C2f-MSDERM module, significantly enhancing its performance in traffic facility detection tasks. The core of this module lies in the design of the multi-scale deformable convolutional block (MSDConvBlock). This innovation not only integrates the broad perspective of multi-scale feature extraction but also combines the flexibility of deformable convolution, enabling the model to more effectively capture targets of different sizes and shapes in traffic scenes, such as traffic signs, reflective cones, and metal guardrails, which often exhibit high diversity and irregularity. Furthermore, the TripletAttention module, through a refined feature enhancement mechanism, further improves the model's ability to recognize key features, helping to accurately distinguish and locate targets in complex traffic backgrounds. The introduction of this attention mechanism makes the model more robust to challenges such as occlusion and changes in lighting. In addition, training using the CIoU loss function not only considers the overlap rate of bounding boxes but also incorporates geometric factors such as center point distance and aspect ratio, further optimizing the regression accuracy of the bounding boxes, which is particularly important for traffic facility detection requiring high-precision localization. In summary, the improved YOLOV8n model of this invention, through the synergistic effect of the C2f-MSDERM module and its internal multi-scale deformable convolutional blocks and TripletAttention module, combined with optimized training of the CIoU loss function, significantly improves the accuracy, robustness, and efficiency of the model in traffic facility detection tasks, providing strong technical support for the construction of intelligent transportation systems.

Claims

1. A traffic facility detection method based on an improved YOLOv8n, comprising the following steps: Step 1: After collecting and processing the raw data, a traffic facility dataset is obtained; Step 2: Divide the dataset obtained in Step 1 to construct the training set and the test set; Step 3: Improve the YOLOv8n model by proposing a multi-scale deformable augmentation residual module C2f-MSDERM, which is applied to the backbone network. The C2f-MSDERM module consists of two sub-modules: one is a multi-scale deformable convolutional block MSDConvBlock, where the input feature map X1 is passed through MSDConvBlock to dynamically extract features of targets of different scales and shapes, resulting in a multi-scale feature map Y; the other sub-module is a Triplet Attention module, which captures cross-dimensional attention weights through a three-branch structure to augment the features extracted by MSDConvBlock, resulting in an augmented feature map Z. A residual connection is established between the sequential multi-scale deformable convolutional block MSDConvBlock and the Triplet Attention module, meaning the feature map passing through the Triplet Attention module... The enhanced feature map Z obtained by the Attention module is added to the feature map X1 of the input MSDConvBlock to preserve the original feature information and promote gradient propagation; the output of the C2f-MSDERM module is processed by the SPPF module to obtain the feature map extracted by the backbone network; the feature map extracted by the backbone network is input into the Neck network for multi-scale feature fusion, and then into the Head network to complete target detection and classification. The method is as follows: Step 3.1: The input image passes through several convolutional layers of the backbone network and the C2f module to extract features, resulting in feature map X. Feature map X is then input into the C2f-MSDERM module, first passing through a 1×1 convolutional layer. Then, a split operation is performed to split the channels of the feature map to obtain feature maps X1 and X2. The part of feature map X1 output by the split operation is then fed into the subsequent multi-scale deformable convolutional block MSDConvBlock to extract features. Step 3.2: The partial feature map X1 output from Split enters the multi-scale deformable convolutional block MSDConvBlock. A 3×3 convolutional layer is used to perform preliminary feature extraction on the input feature map X1, reducing the number of output channels to half the number of input channels to obtain feature map a. Feature map a is then passed through parallel deformable convolutional layers, with each layer simultaneously extracting features through three parallel deformable convolutional DCN module branches. Feature map a is passed through the first parallel branch, i.e., a 1×1 dynamic convolutional layer DCN, to obtain output b. Feature map a is passed through the second parallel branch, i.e., a 3×3 dynamic convolutional layer DCN, to obtain output c. Feature map a is passed through the third parallel branch, i.e., a 5×5 dynamic convolutional layer DCN, to obtain output d. The outputs {b, c, d} of the three parallel branches are then fused through a Concat operation. The fused output is e, which is passed through a 1×1 convolutional layer to output the multi-scale feature map Y. C×H×W ; Step 3.3: The multi-scale feature map Y output by the deformable convolutional block MSDConvBlock is input to the TripletAttention module, which further enhances the features to obtain the enhanced feature map Z. The TripletAttention module captures cross-dimensional attention weights through a three-branch structure to enhance the feature map. Step 3.4: After the enhanced feature map Z output by the Triplet Attention module, establish a residual connection between the enhanced feature map Z and the input feature map X1 of MSDConvBlock; add the enhanced feature map Z to the input feature map X1 to obtain the output feature map U, so as to preserve the original feature information and promote gradient propagation; Step 3.5: Perform a concat operation on feature map U and part of feature map X2 from the Split output, concatenating them along the channel dimension. After concatenation, pass through a 1×1 convolutional layer to obtain the final output of the C2f-MSDERM module. The output of the C2f-MSDERM module is processed by the SPPF module to obtain the feature map extracted by the backbone network. Step 3.6: Input the feature map extracted by the backbone network into the Neck network for multi-scale feature fusion, and then into the Head network to complete the final target detection and classification; Step 4: The CIoU loss function is used to introduce the aspect ratio between the predicted bounding box and the ground truth bounding box; this allows the predicted box to more accurately match the shape and size of the ground truth box, improving the accuracy of target localization. Step 5: Train and test the model to obtain an optimized network model.

2. The traffic facility detection method based on the improved YOLOv8n according to claim 1, characterized in that... In step 1, the processing of the raw data is divided into three steps: video cropping and frame extraction, manual initial screening, and annotation. In the annotation step, traffic facility targets in the images are annotated. The dataset includes 10 types of traffic facility targets, such as city road signs, highway signs, large warning signs, small directional signs, mileage markers, linear guidance signs, reflective barrels, anti-glare panels, urban metal guardrails, and bridge sound insulation or anti-throw nets. The urban metal guardrails and bridge sound insulation or anti-throw nets are annotated using polygon annotation, while the other eight targets are annotated using rectangular boxes.

3. The traffic facility detection method based on the improved YOLOv8n according to claim 1, characterized in that... The TripletAttention module comprises three key branches, each focusing on different dimensions of feature interaction to enhance feature representation capabilities: a) Channel attention branch will apply multi-scale feature map Y C×H×W The C-dimensional tensor is reduced to 2-dimensional using Z-Pool, and the average pooling and max pooling features in the C-dimensional region are concatenated. Then, convolution is applied to reduce the C-dimensional tensor to 1-dimensional, followed by batch normalization layers. Finally, a sigmoid activation function is used to generate attention weights S1, which are then compared with the input feature map Y. C×H×W Multiplying them yields the output Z1 of the first critical branch; b) The interaction branch between the channel and the spatial W dimension, adjusting the multi-scale feature map Y through the Permute operation. C×H×W The dimensional order is changed to a new order Y: height H, number of channels C, and width W. H×C×W Subsequently, Z-Pool is applied along the height H dimension to reduce the H-dimensional tensor to 2-dimensionality. The average pooling and max pooling features along the H dimension are then concatenated. Convolution is then applied to reduce the H-dimensional tensor to 1-dimensionality. A batch normalization layer is then applied, and finally, a sigmoid activation function is used to generate attention weights S2. These attention weights S2 are then compared with the feature map Y. H×C×W Multiply the results and then pass them through Permute to restore the original dimensional order, resulting in the output Z2 of the second critical branch; c) The interaction branch between the channel and the spatial H dimension, adjusting the multi-scale feature map Y through the Permute operation. C×H×W The dimensional order is changed to a new order Y: width W, height H, and number of channels C. W×H×C Then, Z-Pool is applied along the width W dimension to reduce the W-dimensional tensor to 2-dimensionality. The average pooling features and max pooling features along the W dimension are concatenated, and then convolution is applied to reduce the W-dimensional tensor to 1-dimensionality. A batch normalization layer is then applied, and finally, a sigmoid activation function is used to generate attention weights S3. Attention weights S3 are then compared with the feature map Y. W×H×C Multiply the results and then pass them through Permute to restore the original dimensional order, resulting in the output Z3 of the third critical branch; d) Add the output features {Z1, Z2, Z3} of the three branches element by element and take the average value to obtain the enhanced feature map Z.

4. The traffic facility detection method based on the improved YOLOv8n according to claim 1, characterized in that... The specific formula for the CIoU loss function is as follows: , These are the width and height of the prediction box, respectively; , These are the width and height of the actual bounding box, respectively. The square of the distance between the center point of the predicted bounding box and the center point of the ground truth bounding box. Let be the square of the diagonal length of the smallest rectangle that can exactly contain both the predicted and ground truth bounding boxes; where... It is a parameter used for trade-offs. It is a parameter used to measure the consistency of aspect ratio; the corresponding CIoU loss function is as follows: 。

Images