Traffic facility detection method based on improved YOLOV8n

Abstract

The invention relates to a traffic facility detection method based on improved YOLOV8n, comprising the following steps: acquiring and processing original data to obtain a traffic facility data set; constructing a training set and a test set; a YOLOV8n model is improved, and a multi-scale deformable enhanced residual module C2f-MSDERM is provided and is applied to a backbone network part; wherein the C2f-MSDERM module comprises two sub-modules: one sub-module is a multi-scale deformable convolution block MSDConvBlock, and the other sub-module is a multi-scale deformable convolution block MSDConvBlock; the other submodule is a Triplet Attention module, and the other submodule is a Triplet Attention module; the method comprises the following steps of: establishing residual connection for a serial multi-scale deformable convolution block MSDConvBlock and a Triplet Attention module, namely adding an enhanced feature map Z obtained by the Triplet Attention module and a feature map X1 input into the MSDConvBlock so as to reserve original feature information and promote gradient propagation; and the output of the C2f-MSDERM module is processed by the SPPF module to obtain a feature map extracted by the backbone network.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular relates to a traffic facility detection method based on improved YOLOV8n. Background Art

[0002] Traffic facilities are an important part of road engineering and an indispensable part of ensuring driving safety. Traffic safety facilities play an important role in driving safety and reducing potential accidents. For example, placing crash barrels at U-turns can prevent scratches, installing isolation fences on double solid lines can prevent vehicles from traveling in the opposite direction, and installing crash piers on highway curves can provide protection and control when vehicles are out. Protective nets can ensure that vehicles are not disturbed by road debris when driving at high speeds. Pay attention to the maintenance of traffic facilities, and protect them to ensure our traffic safety. Common traffic facilities include anti-glare panels, reflective barrels, anti-throwing nets, etc. Road administration departments need to spend a lot of manpower and material resources to maintain their normal operation. Finding abnormal traffic facilities through inspections is the first step in maintenance work. With the development of technology in recent years, automatic inspections have been put into practical application in scenarios such as drone power inspections, but traffic facilities are still mainly inspected manually. Due to their large variety and complexity and wide geographical distribution, manual inspections require a lot of time and manpower to compare databases with maps, resulting in low inspection efficiency. Relevant departments are in urgent need of an on-board device that can automatically detect traffic facilities in real-time road conditions and realize functions such as automatic comparison of databases and automatic inspections. However, data sets and algorithms are the two major difficulties for on-board equipment to realize real-time traffic facility inspections.

[0003] In terms of data sets, with the development of target detection and autonomous driving, a large number of data sets of traffic signs and markings have emerged, such as the BDD100K in the United States. BDD100K covers different times, different weather conditions (including sunny, cloudy and rainy days, and different times of day and night) and driving scenes. The geographical locations of the data sets are distributed in New York, Berkeley, San Francisco and other places. In the data set, road target detection is to mark 2D bounding boxes on 100,000 pictures of buses, traffic lights, traffic signs, people, bicycles, trucks, motorcycles, cars, trains and passengers. The German traffic sign data set GTSDB contains a total of 900 images of 1360×800 pixels, including 4 categories (prohibited, dangerous, mandatory, and other). And the Chinese traffic sign data set CCTSDB replaces many original easy-to-detect images with difficult samples to adapt to complex and changeable detection environments. And the common domestic public data set TT100K has relatively rich data resources and subdivides more than 100 types of traffic signs, but lacks traffic facility annotations.

[0004] In terms of algorithms, with the development of deep learning and the progress of the autonomous driving industry, traffic sign detection based on deep learning, similar to traffic facility detection, has also made great progress. Liu et al. (LiuY, Peng J, Xue JH, et al. TSingNet: Scale-aware and context-rich feature learning for traffic sign detection and recognition in the wild [J]. Neurocomputing, 2021, 447: 10-22.) proposed a new traffic sign detection network TSingNet, which constructs an attention-driven bidirectional feature pyramid network and transfers foreground semantic information bidirectionally at multiple scales through scale self-attention learning. At the same time, an adaptive receptive field fusion block with a variable expansion rate is introduced to effectively improve the detection accuracy of occluded targets. Yamamoto M et al. (Yamamoto M, Sultana R, Ohashi G. Nighttime Traffic Sign and Pedestrian Detection Using RefineDet with Time-Series Information [J]. IEEJ Transactions on Electrical and Electronic Engineering, 2023, 18 (3): 408-417.) proposed a target detection model that combines the RefineDet small object detection model with the TSSD video detection model, which can improve the detection accuracy in night driving scenes. Han et al. (Han C, Gao G, Zhang Y. Real-time small traffic sign detection with revised faster-RCNN [J]. Multimedia Tools and Applications, 2019, 78: 13263-13278.) proposed a real-time small traffic sign detection method based on improved Faster-RCNN. Considering that the step size of the generator is too large, some layers are removed and a dilated convolution is introduced to improve the detection accuracy of small traffic signs.

[0005] Although research related to traffic sign detection has made great progress, research and application work on traffic facility detection is still insufficient. The challenges in this field are not only the lack of data sets related to traffic facilities, the wide variety of traffic facilities and their different forms, but also their complexity in real scenes. Factors such as occlusion, lighting changes, and perspective changes will seriously affect the accuracy of detection. Therefore, strengthening the research and development and application of traffic facility detection is of great significance to promoting the further maturity and commercialization of autonomous driving technology. Summary of the invention

[0006] In view of the above problems, the present invention proposes a traffic facility detection method based on improved YOLOV8n. The technical solution is as follows:

[0007] A traffic facility detection method based on improved YOLOV8n includes the following steps:

[0008] Step 1: After collecting and processing the original data, the traffic facility data set is obtained;

[0009] Step 2: Divide the data set obtained in step 1 to construct training set and test set;

[0010] Step 3: Improve the YOLOV8n model, propose a multi-scale deformable enhanced residual module C2f-MSDERM, and apply it to the backbone network. The C2f-MSDERM module contains two sub-modules: one sub-module is the multi-scale deformable convolution block MSDConvBlock. The input feature map X1 passes through the multi-scale deformable convolution block MSDConvBlock to dynamically extract the features of targets of different scales and shapes to obtain a multi-scale feature map Y. The other sub-module is the TripletAttention module, which captures the cross-dimensional attention weights through a three-branch structure and enhances the features extracted by MSDConvBlock. , and obtain the enhanced feature map Z; establish a residual connection for the serial multi-scale deformable convolution block MSDConvBlock and the TripletAttention module, that is, add the enhanced feature map Z obtained by the TripletAttention module to the feature map X1 of the input MSDConvBlock to retain the original feature information and promote gradient propagation; the output of the C2f-MSDERM module is processed by the SPPF module to obtain the feature map extracted by the backbone network; the feature map extracted by the backbone network is input into the Neck network for multi-scale feature fusion, and then enters the Head network to complete target detection and classification;

[0011] Step 4: Use the CIoU loss function and introduce the aspect ratio of the predicted bounding box and the true bounding box; this enables the predicted box to more accurately match the shape and size of the true box, thereby improving the accuracy of target positioning;

[0012] Step 5: Train and test the model to obtain the optimized network model.

[0013] Furthermore, in step 1, the processing of the original data is divided into three steps: video cropping and frame extraction, manual preliminary screening, and labeling. In the labeling step, the traffic facility targets in the picture are labeled. The data set includes urban road signs, highway signs, large warning signs, small signs, mileage signs, linear induction signs, reflective barrels, anti-glare panels, urban metal guardrails, bridge sound insulation or anti-throwing nets, a total of 10 types of traffic facility targets; the polygonal labeling method is used to label urban metal guardrails and bridge sound insulation or anti-throwing nets, while the remaining eight targets are labeled using rectangular frames.

[0014] Furthermore, step 3 specifically includes:

[0015] Step 3.1: The input image passes through several convolutional layers of the backbone network and the C2f module to extract features and obtain a feature map X; the feature map X enters the C2f-MSDERM module as input, first passing through a 1×1 convolutional layer; then the channels of the feature map are split through the Split operation to obtain feature maps X1 and X2, and the partial feature map X1 output by Split enters the subsequent multi-scale deformable convolution block MSDConvBlock to extract features;

[0016] Step 3.2: The partial feature map X1 output by Split enters the multi-scale deformable convolution block MSDConvBlock, and uses a 3×3 convolution layer to perform preliminary feature extraction on the input feature map X1, reducing the number of output channels to half of the number of input channels to obtain feature map a; feature map a passes through a parallel deformable convolution layer, and respectively passes through three parallel deformable convolution DCN module branches to synchronously extract features; feature map a passes through the first parallel branch, i.e., a 1×1 dynamic convolution layer DCN, to obtain output b, and the feature map a passes through the second parallel branch, i.e., a 3×3 dynamic convolution layer DCN, to obtain output c, and the feature map a passes through the third parallel branch, i.e., a 5×5 dynamic convolution layer DCN, to obtain output d; the outputs {b, c, d} of the three parallel branches are then channel-fused through the Concat operation, and the fused output is e, which passes through a 1×1 convolution layer to output the multi-scale feature map Y C×H×W ;

[0017] Step 3.3: The multi-scale feature map Y output by the deformable convolution block MSDConvBlock is input to the TripletAttention module, which further enhances the features to obtain the enhanced feature map Z; the TripletAttention module captures the cross-dimensional attention weights through a three-branch structure to enhance the feature map;

[0018] Step 3.4: After the enhanced feature map Z output by the TripletAttention module is set, a residual connection is established between the enhanced feature map Z and the input feature map X1 of the MSDConvBlock; the enhanced feature map Z is added to the input feature map X1 to obtain the output feature map U to retain the original feature information and promote gradient propagation;

[0019] Step 3.5: Concat the feature map U with the partial feature map X2 output by Split, and splice them in the channel dimension. After splicing, pass through a 1×1 convolution layer to obtain the final output of the C2f-MSDERM module; the output of the C2f-MSDERM module is processed by the SPPF module to obtain the feature map extracted by the backbone network;

[0020] Step 3.6: The feature map extracted by the backbone network is input into the Neck network for multi-scale feature fusion, and then into the Head network to complete the final target detection and classification.

[0021] Furthermore, the TripletAttention module contains three key branches, focusing on different feature interaction dimensions to enhance feature representation capabilities:

[0022] a) The channel attention branch converts the multi-scale feature map Y C×H×W The C-dimensional Tensor is reduced to 2 dimensions through Z-Pool, and the average pooling features and the maximum pooling features on the C dimension are connected; then convolution is applied to reduce the C-dimensional Tensor to 1 dimension, and then through the batch normalization layer, and finally the attention weight S1 is generated through the Sigmoid activation function. The attention weight S1 is related to the input feature map Y C×H×W Multiply to get the output Z1 of the first key branch;

[0023] b) The interactive branch between the channel and the spatial W dimension adjusts the multi-scale feature map Y through the Permute operation C×H×W The dimension order becomes a new order Y of height H, number of channels C, and width W H×C×W; Then apply Z-Pool on the height H dimension to reduce the H-dimensional Tensor to 2 dimensions, connect the average pooling features and the maximum pooling features on the H dimension, and then apply convolution to reduce the H-dimensional Tensor to 1 dimension, then pass through the batch normalization layer, and finally generate the attention weight S2 through the Sigmoid activation function. The attention weight S2 is associated with the feature map Y H×C×W Multiply and then pass Permute to restore the original dimension order to obtain the output Z2 of the second key branch;

[0024] c) The interactive branch between the channel and the spatial H dimension adjusts the multi-scale feature map Y through the Permute operation C×H×W The dimension order of the new order Y is the width W, height H and number of channels C. W×H×C ; Then apply Z-Pool on the width W dimension to reduce the W-dimensional Tensor to 2 dimensions, connect the average pooling features and the maximum pooling features on the W dimension, and then apply convolution to reduce the W-dimensional Tensor to 1 dimension, then pass through the batch normalization layer, and finally generate the attention weight S3 through the Sigmoid activation function. The attention weight S3 is associated with the feature map Y W×H×C Multiply and then pass Permute to restore the original dimension order to obtain the output Z3 of the third key branch;

[0025] d) Add the output features {Z1, Z2, Z3} of the three branches element by element and take the average to obtain the enhanced feature map Z.

[0026] Furthermore, the specific formula of CIoU loss function is as follows:

[0027]

[0028] w, h are the width and height of the prediction box respectively; w gt ,h gt are the width and height of the real frame respectively; ρ 2 is the square of the distance between the center point of the predicted box and the real box, c 2 It is the square of the diagonal length of the smallest rectangular box that can just contain the predicted box and the real box; where α is a parameter used as a trade-off, and v is a parameter used to measure the consistency of the aspect ratio; the corresponding CIoU loss function is as follows:

[0029]

[0030] The present invention proposes a traffic facility detection method based on improved YOLOV8n, proposes a C2f-MSDERM module to improve the backbone network, and proposes a multi-scale deformable convolution block MSDConvBlock by combining the advantages of multi-scale feature extraction and deformable convolution, which improves the model's detection ability for multi-scale targets and irregular-shaped targets; at the same time, the TripletAttention module is introduced inside the module, and the attention weights across dimensions are captured through a three-branch structure, and the feature map is enhanced with extremely low computational overhead. Finally, the CIoU loss function is used to enable the predicted box to more accurately match the shape and size of the real box, thereby improving the accuracy of traffic facility detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A traffic facility detection network based on improved YOLOV8n;

[0032] Figure 2 This is a schematic diagram of the C2f-MSDERM module structure;

[0033] Figure 3 Schematic diagram of the structure of the multi-scale deformable convolution block MSDConvBlock.

[0034] Figure 4 Schematic diagram of the TripletAttention module structure. DETAILED DESCRIPTION

[0035] The present invention is described below in conjunction with the accompanying drawings and embodiments. The specific steps of the present invention are as follows.

[0036] Step 1: Data collection and processing, including: The raw data collection work covers the collection of a large amount of real driving video data in various road environments from urban areas to mountainous areas under different weather and lighting conditions through IMX477 high-definition CSI cameras and industrial cameras, as well as Hikvision driving recorders. These video data are cropped, manually screened, and detailedly labeled (including polygon and rectangular box labeling methods), and finally organized into a traffic facility dataset containing 4,477 high-definition pictures, covering 10 key traffic facility targets.

[0037] Step 2: Divide the traffic facility dataset into training set and test set according to a certain ratio.

[0038] Step 3: Construct the network model architecture, including: proposing the C2f-MSDERM module to improve the backbone network. Among them, combining the advantages of multi-scale feature extraction and deformable convolution, a multi-scale deformable convolution block MSDConvBlock is proposed; at the same time, the TripletAttention module is introduced inside the module.

[0039] Step 3.1: Propose and use the C2f-MSDERM module to improve the backbone network. The feature map X enters the C2f-MSDERM module as input, and first passes through a 1×1 convolution layer; then the channels of the feature map are shunted through the Split operation to obtain feature maps X1 and X2. A part of the feature map X1 output by Split enters the subsequent multi-scale deformable convolution block MSDConvBlock to extract features to obtain a multi-scale feature map Y. The multi-scale feature map Y then passes through the serial TripletAttention module to obtain the enhanced feature map Z. The enhanced feature map Z then establishes a residual connection with the input feature map X1 of the MSDConvBlock. The enhanced feature map Z is added to the input feature map X1 to obtain the output feature map U. The output feature map U is channel-fused with the shunted feature map X2 through the Concat operation, and the final output of the C2f-MSDERM module is completed through a 1×1 convolution layer.

[0040] Step 3.2: The feature map extracted by the backbone network is input into the Neck network for multi-scale feature fusion, and then into the Head network to complete the final target detection and classification.

[0041] Step 4: Use the CIoU loss function, which introduces the aspect ratio of the predicted bounding box to the true bounding box. This enables the predicted box to more accurately match the shape and size of the true box, thereby improving the accuracy of target positioning.

[0042] Step 5: Train and test the model to obtain the best performing network model.

[0043] The present invention is described in detail below with reference to the accompanying drawings and embodiments. Figure 1 As shown, the specific implementation steps of the present invention include:

[0044] Step 1: Raw data collection and processing. The collection of raw data includes: one part of the raw data uses the IMX477 high-definition CSI camera with an industrial camera to collect real driving video data in the urban area with a resolution of 4032×3040; the other part of the data is collected using a driving recorder, which collects traffic facilities in real road conditions in multiple cities, including urban roads, highways, mountain national roads, small roads and other actual road conditions. The driving recorder model is Hikvision AE-DC5313-C6, with a resolution of 1306×984. To facilitate subsequent processing, the collected data maintains the same aspect ratio. To ensure that the model can be applied to actual driving scenarios, the collected data includes sunny day, foggy day, foggy night, clear night, rainy day and other weather and lighting conditions.

[0045] The processing of raw data is divided into three steps: video cropping and frame extraction, manual preliminary screening, and annotation. First, the key area of ​​the collected raw video is intercepted, and converted into image data in the form of one frame per second. Then, through manual preliminary screening, the repeated and targetless images are removed. Finally, the traffic facility targets in the image are annotated by software. The data set includes urban road signs, highway signs, large warning signs, small signs, mileage signs, linear induction signs, reflective barrels, anti-glare plates, urban metal guardrails, bridge sound insulation or anti-throwing nets, a total of 10 types of traffic facility targets. At the same time, in order to ensure that the annotation of the data set can be closer to the shape of the real target, the polygon annotation method is used to annotate the urban metal guardrails and bridge sound insulation or anti-throwing nets, and the remaining eight targets are annotated with rectangular frames. Finally, a total of 4477 high-definition data sets of traffic facilities were obtained.

[0046] Step 2: Divide the traffic facility dataset into two parts: training set and test set. The training set contains 3805 images and the test set contains 672 images.

[0047] Step 3: If Figure 1 , Figure 2 , Figure 3 , Figure 4 As shown, the traffic facility detection method based on the improved YOLOV8n includes the following steps:

[0048] Step 3.1: The YOLOv8n network structure is mainly divided into three parts: the backbone network, the neck network, and the head network. In the backbone network, the C2f module is used to reduce redundant parameters while improving computational efficiency, taking into account lightweight while ensuring excellent feature extraction capabilities; in the neck network, the PAN network structure is used to fuse the feature maps of different stages of the backbone network to enhance the feature representation capability. However, in complex traffic scenes, especially when dealing with occlusion, lighting changes, and perspective changes, YOLOv8n still has great deficiencies in the detection effect of traffic facility targets of various shapes and types.

[0049] Step 3.2: In response to the above problems, the present invention proposes a C2f-MSDERM module to improve the backbone network. The C2f-MSDERM module includes two main sub-modules: one sub-module is a multi-scale deformable convolution block MSDConvBlock. The input feature map X1 passes through the multi-scale deformable convolution block MSDConvBlock to dynamically extract the features of targets of different scales and shapes to obtain a multi-scale feature map Y. The other sub-module is the TripletAttention module, which captures the cross-dimensional attention weights through a three-branch structure, and enhances the features extracted by MSDConvBlock with extremely low computational overhead to obtain an enhanced feature map Z. Finally, a residual connection is established between the serial multi-scale deformable convolution block MSDConvBlock and the TripletAttention module, that is, the feature map Z enhanced by the TripletAttention module is added to the feature map X1 of the input MSDConvBlock to retain the original feature information and promote gradient propagation.

[0050] Step 3.3: The input image passes through several convolutional layers in front of the backbone network and the C2f module to extract features and obtain feature map X. Feature map X enters the C2f-MSDERM module as input, first passing through a 1×1 convolutional layer; then the channels of the feature map are split through the Split operation to obtain feature maps X1 and X2, and the partial feature map X1 output by Split enters the subsequent multi-scale deformable convolution block MSDConvBlock to extract features.

[0051] Step 3.4: The partial feature map X1 output by Split enters the multi-scale deformable convolution block MSDConvBlock. First, a 3×3 convolution layer is used to perform preliminary feature extraction on the input feature map X1, and the number of output channels is reduced to half of the number of input channels to obtain feature map a. Feature map a passes through the parallel deformable convolution layer and three parallel deformable convolution DCN module branches to extract features synchronously; feature map a passes through the first parallel branch, i.e., the 1×1 dynamic convolution layer DCN, to obtain output b, and the feature map a passes through the second parallel branch, i.e., the 3×3 dynamic convolution layer DCN, to obtain output c, and the feature map a passes through the third parallel branch, i.e., the 5×5 dynamic convolution layer DCN, to obtain output d; the outputs {b, c, d} of the three parallel branches are then channel-fused through the Concat operation, and the fused output is e, and finally e passes through a 1×1 convolution layer to output the multi-scale feature map Y.

[0052] Step 3.5: The multi-scale feature map Y output by the deformable convolution block MSDConvBlock is input to the TripletAttention module, which further enhances the features to obtain the enhanced feature map Z.

[0053] The TripletAttention module captures cross-dimensional attention weights through a three-branch structure and enhances feature maps with extremely low computational overhead. The TripletAttention module contains three key branches, each focusing on different feature interaction dimensions to enhance feature representation capabilities:

[0054] a) The channel attention branch converts the multi-scale feature map Y C×H×W The C-dimensional Tensor is reduced to 2 dimensions through Z-Pool, and the average pooling features and the maximum pooling features on the C dimension are connected. Then a 7×7 convolution is applied to reduce the C-dimensional Tensor to 1 dimension, and then a batch normalization layer is passed, and finally the attention weight S1 is generated through the Sigmoid activation function. The attention weight S1 is related to the input feature map Y C×H×W Multiply to get the output Z1 of the first key branch.

[0055] b) The interactive branch between the channel and the spatial W dimension adjusts the multi-scale feature map Y through the Permute operation C×H×W The dimension order becomes a new order Y of height H, number of channels C, and width W H×C×W Then, Z-Pool is applied to the height H dimension to reduce the H-dimensional Tensor to 2 dimensions, and the average pooling features and the maximum pooling features in the H dimension are connected. Then, a 7×7 convolution is applied to reduce the H-dimensional Tensor to 1 dimension, and then a batch normalization layer is passed. Finally, the attention weight S2 is generated by the Sigmoid activation function. The attention weight S2 is associated with the feature map Y H×C×W Multiply and then pass Permute to restore the original dimension order to obtain the output Z2 of the second key branch.

[0056] c) The interactive branch between the channel and the spatial H dimension adjusts the multi-scale feature map Y through the Permute operation C×H×W The dimension order of the new order Y is the width W, height H and number of channels C. W×H×C Then, Z-Pool is applied to the width W dimension to reduce the W-dimensional Tensor to 2 dimensions, and the average pooling features and the maximum pooling features on the W dimension are connected. Then, a 7×7 convolution is applied to reduce the W-dimensional Tensor to 1 dimension, and then a batch normalization layer is passed. Finally, the attention weight S3 is generated through the Sigmoid activation function. The attention weight S3 is associated with the feature map Y W×H×CMultiply and then pass Permute to restore the original dimension order to obtain the output Z3 of the third key branch.

[0057] d) Finally, the output features {Z1, Z2, Z3} of the three branches are added element by element and the average is taken to obtain the enhanced feature map Z.

[0058] Step 3.4: After obtaining the enhanced feature map Z output by the TripletAttention module, establish a residual connection between the enhanced feature map Z and the MSDConvBlock input feature map X1. Add the enhanced feature map Z to the input feature map X1 to obtain the output feature map U to preserve the original feature information and promote gradient propagation.

[0059] Step 3.5: Concat the feature map U with the partial feature map X2 output by Split, splice them in the channel dimension, and pass through a 1×1 convolution layer to obtain the final output of the C2f-MSDERM module. The output of the C2f-MSDERM module is processed by the SPPF module to obtain the feature map extracted by the backbone network.

[0060] Step 3.6: The feature map extracted by the backbone network is input into the Neck network for multi-scale feature fusion, and then into the Head network to complete the final target detection and classification.

[0061] Step 4: Use the CIoU loss function, which introduces the aspect ratio of the predicted bounding box to the true bounding box. This enables the predicted box to more accurately match the shape and size of the true box, thereby improving the accuracy of target positioning.

[0062] Step 5: Train the improved YOLOV8n model. The training experiment platform uses Rayzen53600 3.6GHz processor, 32GB memory, Geforce RTX3090 GPU, 24GB video memory, Ubuntu 18.04.3 operating system, Python 3.8 version Pytorch 1.13.1 deep learning framework for training. The SGD optimizer was used for training, the initial learning rate was set to 0.01, the SGD optimizer momentum was 0.937, the weight decay was 0.0005, the batch-size was set to 32, and the number of training rounds was 500.

[0063] To verify the effectiveness of the model of the present invention, while ensuring the consistency of the target detection network structure, this experiment added an improved C2f-MSDERM module to the backbone network. To enhance the backbone network's ability to extract traffic facility features, an ablation experiment was conducted to compare it with the original YOL0v8n model. The performance is shown in Table 1.

[0064] Table 1 Ablation experiment results

[0065]

[0066] From the ablation experiment results, it can be seen that the introduction of the multi-scale deformable convolution block MSDConvBlock is conducive to improving the feature extraction ability of the model and increasing the detection accuracy of traffic facilities; the addition of the Triplet Attention mechanism further enhances the model's ability to extract features and achieves the best detection accuracy for traffic facilities.

[0067] In order to verify the optimization effect of different loss functions on the improved Y0L0v8n network model, while ensuring the consistency of the target detection network structure and training hyperparameters, multiple loss functions including GIoU, DIoU, EIoU, and SIoU are replaced to guide model training, and the effects of the models under different loss functions are compared, as shown in Table 2.

[0068] Table 2 Model results under different loss functions

[0069]

[0070] From the training results, the CIoU loss function is only slightly inferior to DIoU in accuracy, but the recall rate and mAP indicators make the model reach the best level. Therefore, it can be concluded that the CIoU loss function can be used for training to improve the detection ability of the model.

[0071] The YOLOV8n model improvement of the present invention significantly enhances the performance of the model in the task of traffic facility detection by introducing the innovative C2f-MSDERM module. The core of this module lies in the design of the multi-scale deformable convolution block (MSDConvBlock). This innovation not only integrates the broad field of view of multi-scale feature extraction, but also combines the flexibility of deformable convolution, so that the model can more effectively capture targets of different sizes and shapes in traffic scenes, such as traffic signs, reflective barrels, metal guardrails, etc., which are often highly diverse and irregular. Furthermore, the TripletAttention attention module further enhances the model's ability to recognize key features through a refined feature enhancement mechanism, which helps to accurately distinguish and locate targets in complex traffic backgrounds. The introduction of this attention mechanism enables the model to show stronger robustness when dealing with challenges such as occlusion and illumination changes. In addition, the CIoU loss function is used for training, which not only considers the overlap rate of the bounding box, but also adds geometric factors such as center point distance and aspect ratio, further optimizing the regression accuracy of the bounding box, which is particularly important for traffic facility detection that requires high-precision positioning. In summary, the YOLOV8n model improvement of the present invention, through the synergy of the C2f-MSDERM module and its internal multi-scale deformable convolution block and TripletAttention attention module, combined with the optimization training of the CIoU loss function, significantly improves the accuracy, robustness and efficiency of the model in traffic facility detection tasks, and provides strong technical support for the construction of intelligent transportation systems.

Claims

1. A traffic facility detection method based on improved YOLOV8n, comprising the following steps: Step 1: After collecting and processing the original data, the traffic facility data set is obtained; Step 2: Divide the data set obtained in step 1 to construct training set and test set; Step 3: Improve the YOLOV8n model, propose a multi-scale deformable enhanced residual module C2f-MSDERM, and apply it to the backbone network. The C2f-MSDERM module contains two sub-modules: one sub-module is the multi-scale deformable convolution block MSDConvBlock. The input feature map X1 passes through the multi-scale deformable convolution block MSDConvBlock to dynamically extract the features of targets of different scales and shapes to obtain a multi-scale feature map Y. The other sub-module is the TripletAttention module, which captures the cross-dimensional attention weights through a three-branch structure and enhances the features extracted by MSDConvBlock. , and obtain the enhanced feature map Z; establish a residual connection for the serial multi-scale deformable convolution block MSDConvBlock and the TripletAttention module, that is, add the enhanced feature map Z obtained by the TripletAttention module to the feature map X1 of the input MSDConvBlock to retain the original feature information and promote gradient propagation; the output of the C2f-MSDERM module is processed by the SPPF module to obtain the feature map extracted by the backbone network; the feature map extracted by the backbone network is input into the Neck network for multi-scale feature fusion, and then enters the Head network to complete target detection and classification; Step 4: Use the CIoU loss function and introduce the aspect ratio of the predicted bounding box and the true bounding box; this enables the predicted box to more accurately match the shape and size of the true box, thereby improving the accuracy of target positioning; Step 5: Train and test the model to obtain the optimized network model.

2. The traffic facility detection method based on improved Y0L0v8n according to claim 1 is characterized in that In step 1, the processing of the original data is divided into three steps: video cropping and frame extraction, manual preliminary screening, and annotation. In the annotation step, the traffic facility targets in the picture are annotated. The data set includes urban road signs, highway signs, large warning signs, small signs, mileage signs, linear induction signs, reflective barrels, anti-glare panels, urban metal guardrails, bridge sound insulation or anti-throwing nets, a total of 10 types of traffic facility targets; the polygonal annotation method is used to annotate urban metal guardrails and bridge sound insulation or anti-throwing nets, and the remaining eight targets are annotated using rectangular frames.

3. The traffic facility detection method based on improved Y0L0v8n according to claim 1 is characterized in that ,Step 3 specifically includes: Step 3.1: The input image passes through several convolutional layers of the backbone network and the C2f module to extract features and obtain a feature map X; the feature map X enters the C2f-MSDERM module as input, first passing through a 1×1 convolutional layer; then the channels of the feature map are split through the Split operation to obtain feature maps X1 and X2, and the partial feature map X1 output by Split enters the subsequent multi-scale deformable convolution block MSDConvBlock to extract features; Step 3.2: The partial feature map X1 output by Split enters the multi-scale deformable convolution block MSDConvBlock, and uses a 3×3 convolution layer to perform preliminary feature extraction on the input feature map X1, reducing the number of output channels to half of the number of input channels to obtain feature map a; feature map a passes through a parallel deformable convolution layer, and respectively passes through three parallel deformable convolution DCN module branches to synchronously extract features; feature map a passes through the first parallel branch, i.e., a 1×1 dynamic convolution layer DCN, to obtain output b, and the feature map a passes through the second parallel branch, i.e., a 3×3 dynamic convolution layer DCN, to obtain output c, and the feature map a passes through the third parallel branch, i.e., a 5×5 dynamic convolution layer DCN, to obtain output d; the outputs {b, c, d} of the three parallel branches are then channel-fused through the Concat operation, and the fused output is e, which passes through a 1×1 convolution layer to output the multi-scale feature map Y C×H×W ; Step 3.3: The multi-scale feature map Y output by the deformable convolution block MSDConvBlock is input to the TripletAttention module, which further enhances the features to obtain the enhanced feature map Z; the TripletAttention module captures the cross-dimensional attention weights through a three-branch structure to enhance the feature map; Step 3.4: After the enhanced feature map Z output by the TripletAttention module is set, a residual connection is established between the enhanced feature map Z and the input feature map X1 of the MSDConvBlock; the enhanced feature map Z is added to the input feature map X1 to obtain the output feature map U to retain the original feature information and promote gradient propagation; Step 3.5: Concat the feature map U with the partial feature map X2 output by Split, and splice them in the channel dimension. After splicing, pass through a 1×1 convolution layer to obtain the final output of the C2f-MSDERM module; the output of the C2f-MSDERM module is processed by the SPPF module to obtain the feature map extracted by the backbone network; Step 3.6: The feature map extracted by the backbone network is input into the Neck network for multi-scale feature fusion, and then into the Head network to complete the final target detection and classification.

4. The traffic facility detection method based on improved Y0L0v8n according to claim 3 is characterized in that ,TripletAttention module contains three key branches, focusing on different feature interaction dimensions to enhance feature representation capabilities: a) The channel attention branch converts the multi-scale feature map Y C×H×W The C-dimensional Tensor is reduced to 2 dimensions through Z-Pool, and the average pooling features and the maximum pooling features on the C dimension are connected; then convolution is applied to reduce the C-dimensional Tensor to 1 dimension, and then through the batch normalization layer, and finally the attention weight S1 is generated through the Sigmoid activation function. The attention weight S1 is related to the input feature map Y C×H×W Multiply to get the output Z1 of the first key branch; b) The interactive branch between the channel and the spatial W dimension adjusts the multi-scale feature map Y through the Permute operation C×H×W The dimension order becomes a new order Y of height H, number of channels C, and width W H×C×W ; Then apply Z-Pool on the height H dimension to reduce the H-dimensional Tensor to 2 dimensions, connect the average pooling features and the maximum pooling features on the H dimension, and then apply convolution to reduce the H-dimensional Tensor to 1 dimension, then pass through the batch normalization layer, and finally generate the attention weight S2 through the Sigmoid activation function. The attention weight S2 is associated with the feature map Y H×C×W Multiply and then pass Permute to restore the original dimension order to obtain the output Z2 of the second key branch; c) The interactive branch between the channel and the spatial H dimension adjusts the multi-scale feature map Y through the Permute operation C×H×W The dimension order of the new order Y is the width W, height H and number of channels C. W×H×C ; Then apply Z-Pool on the width W dimension to reduce the W-dimensional Tensor to 2 dimensions, connect the average pooling features and the maximum pooling features on the W dimension, and then apply convolution to reduce the W-dimensional Tensor to 1 dimension, then pass through the batch normalization layer, and finally generate the attention weight S3 through the Sigmoid activation function. The attention weight S3 is associated with the feature map Y W×H×C Multiply and then pass Permute to restore the original dimension order to obtain the output Z3 of the third key branch; d) Add the output features {Z1, Z2, Z3} of the three branches element by element and take the average to obtain the enhanced feature map Z.

5. The traffic facility detection method based on improved Y0L0v8n according to claim 3 is characterized in that ,The specific formula of CIoU loss function is as follows: w, h are the width and height of the prediction box respectively; w gt ,h gt are the width and height of the real frame respectively; ρ 2 is the square of the distance between the center point of the predicted box and the real box, c 2 It is the square of the diagonal length of the smallest rectangular box that can just contain the predicted box and the real box; where α is a parameter used as a trade-off, and v is a parameter used to measure the consistency of the aspect ratio; the corresponding CIoU loss function is as follows:

Images