Road surface micro crack detection method based on improved multi-scale feature fusion algorithm

Abstract

The invention provides a pavement micro crack detection method based on an improved multi-scale feature fusion algorithm, and relates to the technical field of pavement crack detection. Firstly, input features are processed through four feature calibration layers, and the layers adjust the number of input channels to a specified number of output channels through 1 * 1 convolution. Then, the calibrated features are weighted through four attention mechanism modules, and an attention weight is generated by adopting a Sigmoid activation function; the weighted features are subjected to feature extraction through four convolution layers with different scales, each convolution layer is connected with a gamma normalization layer and a Dropout layer, and global average pooling is used for performing spatial dimension compression on the features of each scale to 1 * 1 so as to obtain global information of each scale. And finally, the features after global average pooling are spliced on channel dimensions to form final multi-scale feature representation, so that the algorithm can more fully extract effective features of the pavement micro-cracks, and thus pavement micro-crack detection is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of pavement crack detection, and in particular to a pavement micro-crack detection method using an improved multi-scale feature fusion algorithm. Background Art

[0002] Although microcracks in roads may appear to be minor structural defects, if left undetected and repaired, they can become channels for rainwater and other liquids. Water seeping into the road can damage the roadbed and pavement structure. Especially in cold regions, water freezes and expands, accelerating crack growth and damaging the pavement. Prompt detection and repair of microcracks in roads can prevent their further growth. Early crack repair can often be addressed with low-cost filling and sealing materials, avoiding the need for extensive repairs later. Therefore, a detection model for microcracks in roads is needed.

[0003] With the rapid development of deep learning, road crack detection technology has also made significant progress. Chen et al. proposed a new road crack / defect detection method, IlumiCrack, which integrates a Gaussian mixture model (GMM) and an object detection CNN model. The combined model achieved an accuracy and geometric mean (G-mean) of 79.1% and 77.1%, respectively. Nhung HongThi Nguyen et al. proposed a new method using a two-stage convolutional neural network (CNN) for road crack detection and segmentation in pixel-level images. Experimental results show that the two-stage CNN model outperforms existing methods, especially for noisy, low-resolution images and imbalanced datasets. Xuezhi Xiang et al. proposed a deep learning-based pavement crack detection network that applied a state-of-the-art transformer module to improve crack detection. The network was trained on pavement datasets from India, the Czech Republic, and Japan. It achieved F1 scores of 0.6739 and 0.6650 in two online test sets with relatively few network parameters. Jiahao Jiang et al. proposed a GSkYOLOv5 method for accurate crack detection in infrared images of asphalt pavements. Compared to YOLOv5, GSkYOLOv5 achieves 4.7% higher detection accuracy and 1.3% higher recall, outperforming other tested algorithms.

[0004] While these methods have achieved some success in road crack detection, they generally suffer from issues such as insufficient sensitivity to small cracks and resolution limitations that affect detection effectiveness. Most existing methods focus on overall crack characteristics and lack specialized mechanisms for handling smaller cracks. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the present invention provides a pavement micro-crack detection method with an improved multi-scale feature fusion algorithm to solve the problem of low accuracy in pavement micro-crack detection.

[0006] A pavement micro-crack detection method using an improved multi-scale feature fusion algorithm includes the following steps:

[0007] Step 1: Collect a dataset of road microcrack images and divide the dataset into training set, validation set, and test set;

[0008] Step 1.1: Acquire a road microcrack dataset; the road microcrack image dataset includes images of longitudinal cracks, transverse cracks, cracks, and potholes;

[0009] Step 1.2: Divide the dataset into training set, test set and validation set according to the set ratio;

[0010] Step 2: Construct an improved multi-scale feature fusion algorithm;

[0011] Step 2.1: Introduce the existing feature calibration module and process the input features through four feature calibration layers. Each feature calibration layer uses 1x1 convolution to adjust the number of channels.

[0012] Step 2.2: Introduce an improved attention mechanism module to weight the calibrated features and use the Sigmoid activation function to generate effective attention weights;

[0013] The details are as follows:

[0014] Input the feature map X′ after feature calibration i After 1x1 convolution layer W i A Generate effective attention weight A i Then we get the convolution output feature map Y i :

[0015] Y i =W i A *X′ i

[0016] Convolution output feature map Y i Apply the sigmoid activation function to each element y to obtain the effective attention weight A i :

[0017]

[0018] A i =σ(Y i )

[0019] Where i represents the number of output channels, X′ i is the feature map after feature calibration, W i A is a 1x1 convolution kernel used to generate attention weights, * represents the convolution operation, σ is the sigmoid activation function, and y is the feature map Y i For each element in A i is the effective attention weight.

[0020] Step 2.3: Use four convolutional layers of different scales to perform in-depth feature extraction on the attention-weighted features. Each convolutional layer is followed by a gamma normalization layer and a dropout layer.

[0021] The details are as follows:

[0022] The size of the input feature maps X1, X2, X3, and X4 is (C, H, W), and the size of the output feature map of the convolution operation is (C', H', W'), where

[0023] For a 1×1 convolution kernel:

[0024]

[0025] For a 3×3 convolution kernel:

[0026]

[0027] For a 5×5 convolution kernel:

[0028]

[0029] For a 7×7 convolution kernel:

[0030]

[0031] Among them, C, C' represents the number of input channels, H, H' represents the height of the feature map, W, W' represents the width of the feature map, i represents the convolution kernel height index, ranging from 1 to 3, j represents the convolution kernel width index, ranging from 1 to 3, c represents the channel index of the input feature map, ranging from 1 to C, c' represents the channel index of the output feature map, ranging from 1 to C', h' represents the height index of the output feature map, ranging from 1 to H', w' represents the width index of the output feature map, ranging from 1 to W', W 1,c',c 、W 2,c',c,i,j 、W 3,c',c,i,j and W 4,c',c,i,j Represents the convolution kernel weight, the convolution kernel size is 1×1 and 3×3, X 1,c,h',w' 、X 2,c,h'+i-1,w'+j-1 、X 3,c,h'+i-1,w'+j-1 and X 4,c,h'+i-1,w'+j-1represents the input feature map, b 1,c' 、b 2,c' 、b 3,c' and b 4,c' represents the bias term;

[0032] Each feature map X1, X2, X3, X4 and the corresponding attention weight att i , element-wise multiplication:

[0033]

[0034] in, Represents the feature map after attention weighting, att1, att2, att3, att4 represent the attention weights, and are multiplied element-by-element with the corresponding convolution feature map, c' represents the number of input channels, h' represents the height of the feature map, and w' represents the width of the feature map;

[0035] Step 2.4: Compress the spatial dimension through the global average pooling layer to extract the global information of each scale feature, and then splice the pooled features in the channel dimension to form the final multi-scale feature fusion representation.

[0036] The details are as follows:

[0037] Global average pooling takes the average of all elements of each feature map to obtain a C'-dimensional vector; for feature maps The mathematical expression of global average pooling is:

[0038]

[0039]

[0040] Finally, the global average pooling results of these four scales are concatenated along the channel dimension to form the final feature representation X f :

[0041]

[0042] in, represents the features after global average pooling, Represents the element value at (c', h', w') of the feature map after attention weighting of the input feature map. H'×W' represents the product of the height and width of the input feature map, that is, the total number of pixels of the feature map. H' and W' represent the height and width of the input feature map.

[0043] Step 3: Integrate the improved multi-scale feature fusion algorithm and use it to detect pavement micro cracks.

[0044] The beneficial effects of adopting the above technical solution are:

[0045] The present invention provides a pavement micro-crack detection method with an improved multi-scale feature fusion algorithm. The present invention solves the problem of low pavement micro-crack detection accuracy by enhancing the ability to extract key crack features and express multi-scale features, and by using multi-scale convolution and feature calibration layers to improve the extraction of road micro-cracks and the detection accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is an overall structural diagram of the improved multi-scale feature fusion algorithm in an embodiment of the present invention;

[0047] Figure 2 This is a structural diagram of the characteristic calibration layer of the present invention;

[0048] Figure 3 This is the structural diagram of the attention mechanism of the present invention;

[0049] Figure 4 This is a diagram of the multi-scale convolutional layer structure of the global average pooling structure of the present invention. DETAILED DESCRIPTION

[0050] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0051] A pavement micro-crack detection method using an improved multi-scale feature fusion algorithm includes the following steps:

[0052] Step 1: Collect a dataset of road microcrack images and divide the dataset into training set, validation set, and test set;

[0053] Step 1.1: Acquire a road microcrack dataset; the road microcrack image dataset includes images of longitudinal cracks, transverse cracks, cracks, and potholes;

[0054] Step 1.2: Divide the dataset into training set, test set and validation set according to the set ratio;

[0055] In this embodiment, based on the public dataset RDD2022, the crack dataset is processed and the vehicle dataset is divided into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0056] Step 2: Construct an improved multi-scale feature fusion algorithm, such as Figure 1 As shown;

[0057] Step 2.1: Introduce the existing feature calibration module and process the input features through four feature calibration layers. Each feature calibration layer uses 1x1 convolution to adjust the number of channels to achieve effective compression and adjustment of channel dimensions. Figure 2 As shown;

[0058] Step 2.2: Introduce an improved attention mechanism module to weight the calibrated features and use the Sigmoid activation function to generate effective attention weights to highlight the importance of crack features;

[0059] The details are as follows:

[0060] Input the feature map X′ after feature calibration i After 1x1 convolution layer W i A Generate effective attention weight A i Then we get the convolution output feature map Y i :

[0061] Y i =W i A *X′ i

[0062] Convolution output feature map Y i Apply the sigmoid activation function to each element y to obtain the effective attention weight A i :

[0063]

[0064] A i =σ(Y i )

[0065] Where i represents the number of output channels, X′ i is the feature map after feature calibration, W i A is a 1x1 convolution kernel used to generate attention weights, * represents the convolution operation, σ is the sigmoid activation function, and y is the feature map Y i For each element in A i is the effective attention weight.

[0066] In step 2.2, the reference of the attention mechanism enables the algorithm to dynamically adjust its attention allocation, emphasizing more important features while suppressing unimportant features. By assigning an attention weight to each position in the feature map, features of different scales may contain different information. The attention mechanism assigns weights to each feature, allowing the model to automatically focus on important features and suppress irrelevant features, thereby improving the effect of feature fusion. Figure 3shown.

[0067] Step 2.3: Use four convolutional layers of different scales to perform in-depth feature extraction on the attention-weighted features. Each convolutional layer of different scales is followed by a gamma normalization layer and a Dropout layer to improve the generalization performance and robustness of the network model. Figure 4 As shown;

[0068] The details are as follows:

[0069] The size of the input feature maps X1, X2, X3, and X4 is (C, H, W), and the size of the output feature map of the convolution operation is (C', H', W'), where

[0070] For a 1×1 convolution kernel:

[0071]

[0072] For a 3×3 convolution kernel:

[0073]

[0074] For a 5×5 convolution kernel:

[0075]

[0076] For a 7×7 convolution kernel:

[0077]

[0078] Among them, C, C' represents the number of input channels, H, H' represents the height of the feature map, W, W' represents the width of the feature map, i represents the convolution kernel height index, ranging from 1 to 3, j represents the convolution kernel width index, ranging from 1 to 3, c represents the channel index of the input feature map, ranging from 1 to C, c' represents the channel index of the output feature map, ranging from 1 to C', h' represents the height index of the output feature map, ranging from 1 to H', w' represents the width index of the output feature map, ranging from 1 to W', W 1,c',c 、W 2,c',c,i,j 、W 3,c',c,i,j and W 4,c',c,i,j Represents the convolution kernel weight, the convolution kernel size is 1×1 and 3×3, X 1,c,h',w' 、X 2,c,h'+i-1,w'+j-1 、X 3,c,h'+i-1,w'+j-1 and X 4,c,h'+i-1,w'+j-1 represents the input feature map, b 1,c' 、b 2,c' 、b 3,c' and b 4,c' represents the bias term;

[0079] Each feature map X1, X2, X3, X4 and the corresponding attention weight att i , element-wise multiplication:

[0080]

[0081] in, Represents the feature map after attention weighting, att1, att2, att3, att4 represent the attention weights, and are multiplied element-by-element with the corresponding convolution feature map, c' represents the number of input channels, h' represents the height of the feature map, and w' represents the width of the feature map;

[0082] Step 2.4: Compress the spatial dimension through the global average pooling layer to extract the global information of each scale feature, and then splice the pooled features in the channel dimension to form the final multi-scale feature fusion representation.

[0083] The details are as follows:

[0084] Global average pooling takes the average of all elements of each feature map to obtain a C'-dimensional vector; for feature maps The mathematical expression of global average pooling is:

[0085]

[0086] Finally, the global average pooling results of these four scales are concatenated along the channel dimension to form the final feature representation X f :

[0087]

[0088] in, represents the features after global average pooling, Represents the element value at (c', h', w') of the feature map after attention weighting of the input feature map. H'×W' represents the product of the height and width of the input feature map, that is, the total number of pixels of the feature map. H' and W' represent the height and width of the input feature map.

[0089] Step 3: Integrate the improved multi-scale feature fusion algorithm and use it to detect pavement micro cracks.

[0090] In this embodiment, an improved multi-scale fusion algorithm is proposed. The algorithm is mainly used for road micro-crack image processing tasks by integrating features of multiple scales. First, the input features are processed by four feature calibration layers, which adjust the number of input channels to the specified number of output channels through 1x1 convolution. Subsequently, the calibrated features are weighted by four attention mechanism modules, and the attention weights are generated using the Sigmoid activation function. The weighted features are extracted by four convolution layers of different scales, and each convolution layer is followed by a gamma normalization layer and a Dropout layer to improve the generalization ability of the model. In addition, global average pooling is used to compress the spatial dimension of the features of each scale to 1x1 to obtain global information of each scale. Finally, the features after global average pooling are spliced in the channel dimension to form the final multi-scale feature representation as the output of the algorithm, so that the algorithm can more fully extract effective features of pavement micro-cracks, thereby improving the algorithm's detection accuracy of pavement micro-cracks.

[0091] The above description is merely a preferred embodiment of the present disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also encompass other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by mutually replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A pavement micro-crack detection method based on an improved multi-scale feature fusion algorithm, characterized in that: The following steps are involved: Step 1: Collect a dataset of road microcrack images and divide the dataset into training set, validation set, and test set; Step 2: Construct an improved multi-scale feature fusion algorithm; Step 2.1: Introduce the existing feature calibration module and process the input features through four feature calibration layers. Each feature calibration layer uses 1x1 convolution to adjust the number of channels. Step 2.2: Introduce an improved attention mechanism module to weight the calibrated features and use the Sigmoid activation function to generate effective attention weights; Step 2.3: Use four convolutional layers of different scales to perform in-depth feature extraction on the attention-weighted features. Each convolutional layer is followed by a gamma normalization layer and a dropout layer. Step 2.4: Compress the spatial dimension through the global average pooling layer to extract the global information of each scale feature, and then splice the pooled features in the channel dimension to form the final multi-scale feature fusion representation; Step 3: Integrate the improved multi-scale feature fusion algorithm and use it to detect pavement micro cracks.

2. The pavement micro-crack detection method using an improved multi-scale feature fusion algorithm according to claim 1 is characterized in that: The step 1 comprises the following steps: Step 1.1: Obtain pavement microcrack dataset; The road micro crack image dataset includes images of longitudinal cracks, transverse cracks, fissures and potholes; Step 1.2: Divide the dataset into training set, test set and validation set according to the set ratio.

3. The pavement micro-crack detection method using an improved multi-scale feature fusion algorithm according to claim 1 is characterized in that: The step 2.2 is as follows: Input the feature map X′ after feature calibration i After 1x1 convolution layer Generate effective attention weight A i Then we get the convolution output feature map Y i : Convolution output feature map Y i Apply the sigmoid activation function to each element y to obtain the effective attention weight A i : A i =σ(Y i ) Where i represents the number of output channels, X′ i is the feature map after feature calibration, is a 1x1 convolution kernel used to generate attention weights, * represents the convolution operation, σ is the sigmoid activation function, and y is the feature map Y i For each element in A i is the effective attention weight.

4. The method for detecting pavement microcracks using an improved multi-scale feature fusion algorithm according to claim 1, characterized in that: The step 2.3 is as follows: The size of the input feature maps X1, X2, X3, and X4 is (C, H, W), and the size of the output feature map of the convolution operation is (C', H', W'), where For a 1×1 convolution kernel: For a 3×3 convolution kernel: For a 5×5 convolution kernel: For a 7×7 convolution kernel: Among them, C, C' represents the number of input channels, H, H' represents the height of the feature map, W, W' represents the width of the feature map, i represents the convolution kernel height index, ranging from 1 to 3, j represents the convolution kernel width index, ranging from 1 to 3, c represents the channel index of the input feature map, ranging from 1 to C, c' represents the channel index of the output feature map, ranging from 1 to C', h' represents the height index of the output feature map, ranging from 1 to H', w' represents the width index of the output feature map, ranging from 1 to W', W 1,c',c 、W 2,c',c,i,j 、W 3,c',c,i,j and W 4,c',c,i,j Represents the convolution kernel weight, the convolution kernel size is 1×1 and 3×3, X 1,c,h',w' 、X 2,c,h'+i-1,w'+j-1 、X 3,c,h'+i-1,w'+j-1 and X 4,c,h'+i-1,w'+j-1 represents the input feature map, b 1,c' 、b 2,c' 、b 3,c' and b 4,c' represents the bias term; Each feature map X1, X2, X3, X4 and the corresponding attention weight att i , element-wise multiplication: in, Represents the feature map after attention weighting, att1, att2, att3, att4 represent the attention weights, and are multiplied element-by-element with the corresponding convolutional feature map, c' represents the number of input channels, h' represents the height of the feature map, and w' represents the width of the feature map.

5. The pavement micro-crack detection method using an improved multi-scale feature fusion algorithm according to claim 1 is characterized in that: The step 2.4 is as follows: Global average pooling takes the average of all elements of each feature map to obtain a C'-dimensional vector; for feature maps The mathematical expression of global average pooling is: Finally, the global average pooling results of these four scales are concatenated along the channel dimension to form the final feature representation X f : in, Represents the features after global average pooling, Represents the element value at (c', h', w') of the feature map after attention weighting of the input feature map. H'×W' represents the product of the height and width of the input feature map, that is, the total number of pixels of the feature map. H' and W' represent the height and width of the input feature map.

Images