YOLOv5 pavement damage detection method based on attention mechanism and BiFPN improvement

Abstract

The invention provides a YOLOv5 pavement damage detection method based on an attention mechanism and BiFPN improvement, and relates to the technical field of road damage detection. The method comprises the following steps: firstly, acquiring a pavement damage image data set, and preprocessing to obtain a preprocessed pavement damage image data set; based on an existing YOLOv5 network, an ECA module, a BiFPN network, an HLDFF module and a Mish activation function are fused, and an improved YOLOv5 detection model is obtained; obtaining an original YOLOv5 network model, and replacing a part of convolution operation in the C3 module with an ECA module; a BiFPN network is introduced into a Neck network part of the YOLOv5 to replace Concat operation, and meanwhile, an HLDFF module is designed and introduced to optimize a feature fusion process; replacing a SiLU activation function with a Mish activation function to enhance the nonlinear expression ability of the model, thereby completing the construction of the improved YOLOv5 detection model; and training the improved YOLOv5 detection model, and carrying out pavement damage detection by using the trained model weight to obtain the pavement damage type.

Description

Technical Field

[0001] The present invention relates to the technical field of road damage detection, and in particular to a YOLOv5 road damage detection method based on an attention mechanism and BiFPN improvement. Background Art

[0002] The diversity and complexity of pavement damage increase the difficulty of pavement damage detection. Different types of damage, such as cracks, potholes, and undulations, have different shapes and characteristics. At the same time, pavement damage often occurs in a changing environment with complex backgrounds. Light changes, shadows, and weather factors can affect image quality. The interweaving of these factors makes it more difficult to accurately identify damage features. Traditional pavement damage detection methods rely heavily on the experience of road maintenance personnel, who use ground measurements and vehicle-mounted multi-sensor detection systems to monitor pavement conditions. These methods are often time-consuming, inefficient, and hinder traffic. They are not suitable for monitoring large areas of road surfaces. With the development of deep learning, pavement damage detection has gradually shifted from traditional machine learning to automated detection based on deep learning.

[0003] In recent years, research on improvements to the YOLO model has made significant progress in the field of object detection, especially in complex scenarios such as road damage detection and bridge crack detection. However, despite advances in lightweighting, attention mechanisms, and feature fusion modules, existing models still face challenges in detecting multi-scale and complex road damage features. This is particularly true in terms of detection accuracy when handling fine-grained damage features, occluded areas, and low-contrast environments. Furthermore, insufficient feature fusion leads to poor information transfer, which hinders the recognition of multi-scale damage features.

[0004] Therefore, there is an urgent need for an improved YOLOv5 road damage detection method based on the attention mechanism and BiFPN. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the present invention provides a road damage detection method based on the attention mechanism and BiFPN improved YOLOv5;

[0006] A road damage detection method based on the attention mechanism and BiFPN improved YOLOv5 includes the following steps:

[0007] Step 1: Obtain a road damage image dataset and perform preprocessing to obtain a preprocessed road damage image dataset;

[0008] The road surface damage image dataset includes road damage images and annotates damage point information;

[0009] Step 2: Based on the existing YOLOv5 network, the ECA module, BiFPN network, HLDFF module and Mish activation function are integrated to obtain an improved YOLOv5 detection model;

[0010] Specifically: obtain the original YOLOv5 network model and use the ECA module to replace some convolution operations in the C3 module; introduce the BiFPN network to the Neck network part of YOLOv5 to replace the Concat operation, and design and introduce the HLDFF module to optimize the feature fusion process; replace the SiLU activation function with the Mish activation function to enhance the nonlinear expression ability of the model, thereby completing the construction of the improved YOLOv5 detection model.

[0011] Step 2.1: Road damage feature extraction based on ECA attention mechanism:

[0012] Use the ECA module to replace some convolution operations in the C3 module in the YOLOv5 network, introduce the channel attention mechanism, and automatically calculate the channel weights by performing channel weighting on the input feature map;

[0013] Specifically, the ECA module is integrated into the end of the C3 module and acts on the feature map after convolution processing. It optimizes the feature representation by dynamically adjusting the weights of each channel. First, global average pooling (GAP) is performed on the feature map output by the C3 module in YOLOv5 to compress the spatial dimension information into global features of the channel dimension, obtaining the global features corresponding to the channel and retaining the global semantic information of each channel.

[0014] It is expressed by the following formula:

[0015]

[0016] Among them, F C Represents a lightweight local one-dimensional convolution kernel, which is used to introduce the interaction information between adjacent channels. C is the number of channels, H and W are the height and width of the feature map respectively, σ is the Sigmoid activation function, which is used to introduce nonlinear transformation, and Y C Represents the attention weight of the Cth channel, which is multiplied by the original feature map to form the weighted feature output, X c,i,j Represents the pixel value of the Cth channel at position (i, j) in the input feature map;

[0017] The ECA module then performs a weighted operation on the global features through a one-dimensional convolution kernel. The size of the convolution kernel k is adaptively determined according to the number of channels C. The feature weights after convolution are nonlinearly mapped through an activation function to optimize the attention allocation of the channel; it can be expressed as follows:

[0018]

[0019] Among them, ψ(C) represents the mapping function of the adaptive convolution kernel size, γ and b are control parameters, γ odd Make sure the convolution kernel size is always an odd number;

[0020] Finally, the calculated channel weight is multiplied by the original feature map channel by channel to obtain a feature map that integrates the channel weight information;

[0021] Step 2.2: Improve the Neck network part of the YOLOv5 model and introduce the BiFPN network structure to replace the Concat operation;

[0022] Specifically, when the BiFPN network generates the feature pyramid, it prunes nodes that have only a single input and no feature fusion function. In the feature fusion stage, the BiFPN network introduces a bidirectional feature transfer path to fuse high-level semantic features and low-level detail features upward and downward, ensuring that feature information at different levels can interact efficiently. When the input and output nodes are at the same scale, additional nodes and jump connections are added to strengthen the feature interaction between different scales. At the same time, the BiFPN network uses the bidirectional path as a layer of feature fusion network and performs multiple iterations.

[0023] Step 2.3: Designed and introduced the HLDFF module;

[0024] Specifically, the convolution operation is used to keep the number of channels of the medium-dimensional feature map consistent with that of the high-dimensional feature map. At the same time, the resolution of the medium-dimensional feature map is amplified through upsampling to match the resolution of the low-dimensional feature map. The adjusted medium-dimensional features are then spliced with the low-dimensional feature map in the channel dimension to fuse feature information of different scales. Further feature extraction is then performed through the convolution layer to generate a weighted fused feature map. Finally, the fused feature map and the high-dimensional feature map are combined through the BiFPN structure to complete the final feature fusion.

[0025] Step 2.4: Use the Mish activation function instead of the SiLU function:

[0026] The formula of the Mish activation function is:

[0027] Mish(x)=x·tanh(softplus(x))

[0028] softplus(x)=ln(1+e x )

[0029] Among them, x is the input feature, tanh represents the hyperbolic tangent function, and softplus(x) is the smooth approximation function of ReLU, which realizes smooth nonlinear mapping through the combination of natural logarithm and exponential function.

[0030] Step 3: Train the improved YOLOv5 detection model, use the trained model weights to detect road damage, and obtain the road damage type.

[0031] The beneficial effects of adopting the above technical solution are:

[0032] This paper provides an improved YOLOv5 road damage detection method based on the attention mechanism and BiFPN. By introducing the ECA attention mechanism in the feature extraction stage, the model's ability to perceive complex damage features is enhanced. In the feature fusion stage, the BiFPN and HLDFF modules are introduced to fully utilize multi-scale feature information, improving the model's detection capabilities for damage features of different sizes. This improved model effectively addresses the issues faced by YOLOv5 in road damage image detection, such as the difficulty in capturing highly recognizable damage features during feature extraction, and the insufficient fusion of features at different levels, which leads to poor information transfer and feature loss.

[0033] By introducing the ECA attention mechanism and BiFPN architecture, the model improves the detection accuracy of fine-grained damage features in complex environments. The HLDFF module also optimizes feature fusion, addressing the shortcomings of traditional models in fusing features at different scales. Ultimately, through training and evaluation, the optimal detection model is selected, ensuring efficient and accurate pavement damage detection in practical applications. This invention can be widely applied in road maintenance, traffic management, and intelligent transportation systems, providing strong support for road damage monitoring and repair. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a workflow diagram of the present invention;

[0035] Figure 2 This is a structural diagram of the ECA module in an embodiment of the present invention;

[0036] Figure 3 The method and location of adding the ECA module in the embodiment of the present invention;

[0037] Figure 4 This is a comparison diagram of the feature fusion network structure in an embodiment of the present invention;

[0038] Among them, (a)-original YOLOv5 network model structure, (b)-BiFPN network structure, (c)-complete BiFPN feature fusion structure;

[0039] Figure 5 This is a structural diagram of the HLDFF module in an embodiment of the present invention;

[0040] Figure 6 This is a diagram of the optimized YOLO v5 network structure in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0042] A road damage detection method based on attention mechanism and BiFPN improved YOLOv5, such as Figure 1 As shown, the following steps are included:

[0043] Step 1: Obtain a road damage image dataset and perform preprocessing to obtain a preprocessed road damage image dataset;

[0044] The road surface damage image dataset includes road damage images and annotates damage point information;

[0045] This example is based on the public dataset Road Damage Dataset 2020, which contains approximately 26,336 road damage images and annotates approximately 31,000 damage points. The dataset includes three main types of road damage: D00 (longitudinal cracks), D10 (transverse cracks), and D20 (network cracks). The dataset is divided into training, validation, and testing sub-datasets in a ratio of 7:1.5:1.5. Standardization or normalization is performed on the training, validation, and testing sub-datasets to obtain the preprocessed road damage image dataset.

[0046] Step 2: Based on the existing YOLOv5 network, the ECA module, BiFPN network, HLDFF module and Mish activation function are integrated to obtain an improved YOLOv5 detection model;

[0047] Specifically: Get the original YOLOv5 network model, such as Figure 4 In (a), the ECA module is used to replace some convolution operations in the C3 module; the BiFPN network is introduced in the Neck network part of YOLOv5 to replace the Concat operation, and the HLDFF module is designed and introduced to optimize the feature fusion process; the SiLU activation function is replaced by the Mish activation function to enhance the nonlinear expression ability of the model, thereby completing the construction of the improved YOLOv5 detection model.

[0048] Step 2.1: Road damage feature extraction based on ECA (Efficient Channel Attention) mechanism:

[0049] The ECA module is used to replace some convolution operations in the C3 module in the YOLOv5 network, and the channel attention mechanism is introduced. By performing channel weighting on the input feature map, the channel weight is automatically calculated; the expression of key channel features is enhanced, while redundant information is suppressed, and the ability to extract key features of road damage areas is enhanced, especially in complex backgrounds and multi-scale damage scenes, which reduces the loss of detailed features and improves detection accuracy and robustness. The ECA structure is as follows: Figure 2 shown.

[0050] The ECA module uses global information to calculate the weight of each channel so that the network can adaptively adjust the degree of attention to different channel features.

[0051] Specifically, the ECA module is integrated into the end of the C3 module and acts on the feature map after convolution processing. It optimizes the feature representation by dynamically adjusting the weights of each channel. First, global average pooling (GAP) is performed on the feature map output by the C3 module in YOLOv5 to compress the spatial dimension information into global features of the channel dimension, obtaining the global features corresponding to the channel and retaining the global semantic information of each channel.

[0052] It is expressed by the following formula:

[0053]

[0054] Among them, F C Represents a lightweight local one-dimensional convolution kernel, which is used to introduce the interaction information between adjacent channels. C is the number of channels, H and W are the height and width of the feature map respectively, σ is the Sigmoid activation function, which is used to introduce nonlinear transformation, and Y C Represents the attention weight of the Cth channel, which is multiplied by the original feature map to form the weighted feature output, X c,i,j Represents the pixel value of the Cth channel at position (i, j) in the input feature map;

[0055] The ECA module then performs a weighted operation on the global features through a one-dimensional convolution kernel. The size of the convolution kernel k is adaptively determined according to the number of channels C. The feature weights after convolution are nonlinearly mapped through an activation function to optimize the attention allocation of the channel; it can be expressed as follows:

[0056]

[0057] Among them, ψ(C) represents the mapping function of the adaptive convolution kernel size, γ and b are control parameters, γ odd Ensure that the size of the convolution kernel is always odd; ensure that high-dimensional channels have a wider range of interactions, while low-dimensional channels maintain local dependencies.

[0058] Finally, the calculated channel weights are multiplied by the original feature map channel by channel to obtain a feature map that integrates the channel weight information; this significantly enhances the important feature channels while suppressing irrelevant or redundant information. Figure 3 shown.

[0059] Step 2.2: Improve the Neck network part of the YOLOv5 model and introduce the Bidirectional Feature Pyramid Network (BiFPN) network structure to replace the Concat operation; through the repeated stacking of bidirectional feature fusion layers, the information transmission of multi-scale features is enhanced.

[0060] Specifically, when the BiFPN network generates the feature pyramid, for nodes with only a single input and no feature fusion function, such as nodes in the P3 and P7 layers, BiFPN will cut them to avoid redundant calculations and improve network efficiency. In the feature fusion stage, the BiFPN network introduces a bidirectional feature transfer path to fuse high-level semantic features and low-level detail features upward and downward, ensuring that feature information at different levels can interact efficiently. When the input and output nodes are at the same scale, additional nodes and jump connections are added to strengthen the feature interaction between different scales, such as Figure 4 At the same time, the BiFPN network uses the bidirectional path as a feature fusion network and performs multiple iterations to further improve the expression ability of feature fusion, such as Figure 4 As shown in (c), the bidirectional structure and skip connection ensure the efficient fusion and transmission of feature information.

[0061] Step 2.3: Design and introduce the HLDFF module (Hierarchical Low-Dimensional Feature Fusion); the module structure diagram is as follows Figure 5 As shown in Figure 2, it aims to solve the information loss problem when the scales of low-dimensional, medium-dimensional and high-dimensional features do not match.

[0062] The HLDFF module adjusts the number of channels of the medium-dimensional feature map through convolution operations to keep it consistent with the high-dimensional features, and amplifies the resolution of the medium-dimensional feature map through upsampling operations to match the resolution of the low-dimensional feature map. The adjusted medium-dimensional features and low-dimensional feature maps are spliced in the channel dimension to fuse feature information of different scales. The features are then further extracted through the convolution layer to generate a weighted fused feature map. Finally, the fused feature map and the high-dimensional feature map are finally fused through the BiFPN structure, ensuring the effective transmission and fusion of multi-level features, thereby further improving the accuracy and robustness of the YOLOv5 road damage detection model. By optimizing the feature fusion process and feature transfer path, the present invention significantly improves the interaction efficiency of multi-scale features and effectively solves the problem of information loss caused by scale mismatch, thereby improving detection performance.

[0063] Specifically: the convolution operation is performed on the medium-dimensional feature map to make it consistent with the number of high-dimensional feature channels, and the upsampling operation is performed to amplify the resolution of the medium-dimensional feature map so that it matches the resolution of the low-dimensional feature map; then the adjusted medium-dimensional features are spliced with the low-dimensional feature map in the channel dimension, and feature information of different scales is fused, and further feature extraction is performed through the convolution layer to generate a weighted fused feature map. Finally, the fused feature map and the high-dimensional feature map complete the final feature fusion through the BiFPN structure to ensure the information transmission and effective fusion of multi-level features. The HLDFF structure is as follows Figure 5 As shown in the figure, the low-dimensional feature map (C3_1_3) and the adjusted medium-dimensional feature map (C3_2_3) are jointly passed to BiFPN for multi-scale feature fusion, and the high-dimensional feature map (C3_1_6) and the medium-dimensional feature map (C3_2_3) after one convolution process are further passed to the final BiFPN fusion stage.

[0064] The beneficial effects of the above technical solution are as follows: the optimized network not only retains the original feature fusion logic of YOLOv5, but also strengthens the feature extraction capability by introducing the attention mechanism; at the same time, with the help of BiFPN and HLDFF modules, it realizes efficient interaction and deep fusion of multi-scale features, making up for the shortcomings of the original YOLOv5 in feature extraction and fusion. The adjustment of the overall structure has significantly improved the detection accuracy and robustness of the network in complex scenes. The overall optimized network structure is as follows: Figure 6 shown.

[0065] Step 2.4: Use the Mish activation function instead of the SiLU function:

[0066] In order to solve the problems of gradient vanishing and insufficient utilization of negative features caused by the SiLU activation function in the YOLOv5 network, the present invention adopts the Mish activation function as a substitute.

[0067] The Mish activation function has smoother nonlinear characteristics, can maintain non-zero output in the negative range, avoid neuron death, and effectively alleviate the gradient vanishing problem. Specifically, the formula of the Mish activation function is:

[0068] Mish(x)=x·tanh(softplus(x))

[0069] softplus(x)=ln(1+e x )

[0070] Among them, x is the input feature, tanh represents the hyperbolic tangent function, and softplus(x) is the smooth approximation function of ReLU, which realizes smooth nonlinear mapping through the combination of natural logarithm and exponential function.

[0071] The beneficial effect of the above technical solution is: by replacing the SiLU activation function in the YOLOv5 network with the Mish activation function, the network not only alleviates the gradient vanishing problem, but also enhances the integrity and expression ability of feature information, thereby improving the accuracy and robustness of the model in road damage detection tasks.

[0072] In the YOLOv5 network, this paper replaces the SiLU activation function with the Mish activation function. The Mish activation function combines the smooth nonlinear properties of the Tanh and Softplus functions, overcoming the problem of SiLU's output being too small in negative regions and effectively alleviating the vanishing gradient phenomenon. Compared to SiLU, the Mish activation function can better preserve the representation of negative features, improving the model's stability when handling complex nonlinear relationships and its ability to capture fine-grained features. This feature significantly enhances the network's performance in road damage detection tasks, improving detection accuracy and robustness.

[0073] In this example, a modified YOLOv5 model was pre-trained on the Road Damage Dataset 2020. This dataset contains approximately 26,336 images of road damage, annotated with information on approximately 31,000 damage locations. The images were collected in multiple countries using in-vehicle smartphones and cover three main types of road damage: D00 (longitudinal cracks), D10 (transverse cracks), and D20 (network cracks).

[0074] The experiment uses the PyTorch deep learning framework for development and training. The system runs on the Windows 10 operating system and uses CUDA 11.3 and Python 3.10.

[0075] The training parameter settings include input image resolution 640×640, initial learning rate 0.001, batch size 16, training iteration 300 rounds, and other hyperparameters keep the default values.

[0076] The hardware configuration is Intel i7-9700K CPU, NVIDIA RTX 2080Ti graphics card, 64GB memory and 12GB video memory, ensuring efficient computing performance and a stable operating environment.

[0077] After the training is completed, the optimal model weight is obtained, and the weight is used to detect road damage and obtain the road damage type.

[0078] The above description is merely a preferred embodiment of the present disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also encompass other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by mutually replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A road damage detection method based on attention mechanism and BiFPN improved YOLOv5, characterized by: The following steps are involved: Step 1: Obtain a road damage image dataset and perform preprocessing to obtain a preprocessed road damage image dataset; Step 2: Based on the existing YOLOv5 network, the ECA module, BiFPN network, HLDFF module and Mish activation function are integrated to obtain an improved YOLOv5 detection model; Specifically, the original YOLOv5 network model is obtained and some convolution operations in the C3 module are replaced with the ECA module. The BiFPN network is introduced to the Neck network of YOLOv5 to replace the Concat operation. The HLDFF module is designed and introduced to optimize the feature fusion process. The SiLU activation function is replaced with the Mish activation function to enhance the nonlinear expression ability of the model, thus completing the construction of the improved YOLOv5 detection model. Step 2.1: Road damage feature extraction based on ECA attention mechanism: Use the ECA module to replace some convolution operations in the C3 module in the YOLOv5 network, introduce the channel attention mechanism, and automatically calculate the channel weights by performing channel weighting on the input feature map; Step 2.2: Improve the Neck network part of the YOLOv5 model and introduce the BiFPN network structure to replace the Concat operation; Step 2.3: Designed and introduced the HLDFF module; Step 2.4: Use the Mish activation function instead of the SiLU function: Step 3: Train the improved YOLOv5 detection model, use the trained model weights to detect road damage, and obtain the road damage type.

2. The YOLOv5 road damage detection method based on attention mechanism and BiFPN improvement according to claim 1 is characterized in that: The road surface damage image dataset in step 1 includes road damage images and annotated damage point information.

3. The YOLOv5 road damage detection method based on attention mechanism and BiFPN improvement according to claim 1 is characterized in that: Specifically, the ECA module is integrated into the end of the C3 module and acts on the feature map after convolution processing. The weight of each channel is dynamically adjusted to optimize the feature representation. First, the feature map output by the C3 module in YOLOv5 is subjected to global average pooling (GAP) to compress the spatial dimension information into global features of the channel dimension, thereby obtaining the global features corresponding to the channel and retaining the global semantic information of each channel. It is expressed by the following formula: Among them, F C Represents a lightweight local one-dimensional convolution kernel, which is used to introduce the interaction information between adjacent channels. C is the number of channels, H and W are the height and width of the feature map respectively, σ is the Sigmoid activation function, which is used to introduce nonlinear transformation, and Y C Represents the attention weight of the Cth channel, which is multiplied by the original feature map to form the weighted feature output, X c,i,j Represents the pixel value of the Cth channel at position (i, j) in the input feature map; The ECA module then performs a weighted operation on the global features through a one-dimensional convolution kernel. The size of the convolution kernel k is adaptively determined according to the number of channels C. The feature weights after convolution are nonlinearly mapped through an activation function to optimize the attention allocation of the channel; it can be expressed as follows: Among them, ψ(C) represents the mapping function of the adaptive convolution kernel size, γ and b are control parameters, γ odd Make sure the convolution kernel size is always an odd number; Finally, the calculated channel weights are multiplied by the original feature map channel by channel to obtain a feature map that incorporates the channel weight information.

4. The YOLOv5 road damage detection method based on attention mechanism and BiFPN improvement according to claim 1 is characterized in that: The step 2.2 is specifically as follows: when the BiFPN network generates the feature pyramid, for nodes with only a single input and no feature fusion function, BiFPN will crop them. In the feature fusion stage, the BiFPN network introduces a bidirectional feature transfer path, and simultaneously fuses high-level semantic features and underlying detail features upward and downward to ensure that feature information at different levels can interact efficiently; when the input and output nodes are at the same scale, additional nodes and jump connections are added to strengthen the feature interaction between different scales. At the same time, the BiFPN network uses the bidirectional path as a layer of feature fusion network and performs multiple repeated iterations.

5. The YOLOv5 road damage detection method based on attention mechanism and BiFPN improvement according to claim 1 is characterized in that: The step 2.3 is specifically as follows: the medium-dimensional feature map is subjected to a convolution operation to make it consistent with the number of high-dimensional feature channels, and the resolution of the medium-dimensional feature map is amplified by an upsampling operation to match the resolution of the low-dimensional feature map; the adjusted medium-dimensional features are then spliced with the low-dimensional feature map in the channel dimension to fuse feature information of different scales, and further feature extraction is performed through the convolution layer to generate a weighted fused feature map; finally, the fused feature map and the high-dimensional feature map are combined through the BiFPN structure to complete the final feature fusion.

6. The YOLOv5 road damage detection method based on attention mechanism and BiFPN improvement according to claim 1 is characterized in that: The formula of the Mish activation function described in step 2.4 is: Mish(x)=x·tanh(softplus(x)) softplus(x)=ln(1+e x ) Among them, x is the input feature, tanh represents the hyperbolic tangent function, and softplus(x) is the smooth approximation function of ReLU, which realizes smooth nonlinear mapping through the combination of natural logarithm and exponential function.

Images