Crack detection method based on high-resolution and large receptive field convolutional network

Abstract

A crack detection method based on a high-resolution and large-receptive-field convolutional network comprises the following steps: S01, shooting a crack image on a traffic facility by using a camera, labeling by using a pixel-level label, and constructing a crack data set; s02, constructing an HRLRF-Net model based on a high resolution and a large receptive field; s03, training the HRLRF-Net model by using the crack data set; and S04, carrying out crack detection by using the trained HRLRF-Net model. According to the method, multi-scale feature extraction of a large receptive field is realized through an HRLRF-Net model; through a multi-resolution feature extraction and fusion method, the problems of large calculation amount and insufficient feature expression ability of a high-resolution network are solved, and the accuracy and robustness of crack detection are improved. The crack detector is especially suitable for detecting cracks of different sizes, and especially has advantages in detection of ultra-thin and long cracks, so that the crack detector is suitable for structural health detection of wall surfaces, roads and bridges.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning-related image recognition technology, and in particular to a crack detection method based on a high-resolution convolutional neural network and image segmentation. Background Art

[0002] With the rapid development of highway traffic, ensuring the safe operation of highway facilities has become a key concern of management departments. Cracks in bridges, walls and pavement are common defects in highway facilities, which may have a potential impact on driving safety and traffic efficiency. The use of scientific detection methods and the implementation of effective prevention and maintenance measures can significantly extend the service life of transportation facilities and reduce maintenance costs.

[0003] Traditional crack detection methods for transportation facilities mainly rely on manual inspections and visual observations, which have the disadvantages of large workload, low efficiency, poor safety, and large variability in results. Existing crack detection technologies using equipment such as laser scanners or radars have problems such as high equipment cost, strong environmental dependence, and the need for professional operation and maintenance. At present, automated crack detection technology based on image sensor data acquisition and computer vision has attracted widespread attention and developed rapidly.

[0004] At present, the automatic crack detection technology based on computer vision is mainly divided into traditional image processing methods and machine learning methods. Traditional image processing methods analyze the image features of cracks, such as texture, edge and contrast, and combine them with artificially designed segmentation and extraction basis to achieve good results on specific data sets. Machine learning methods are currently dominated by deep learning methods. According to the technical means of crack detection algorithms, they can be divided into the following three categories: 1) Image classification methods, which are used to determine whether there are cracks in the image; 2) Target detection methods, which use bounding boxes to locate the specific location of cracks in the image; 3) Pixel-level detection methods, which identify all pixels where cracks are located in the image. Pixel-level detection can not only identify the location of cracks, but also calculate the geometric features of cracks such as shape, direction, length and width. It is the current mainstream crack detection method.

[0005] Pixel-level crack detection usually adopts an encoder-decoder network architecture; due to its advantages in feature extraction, reconstruction and multi-scale information fusion, this architecture has been widely used in the field of pixel-level crack detection.

[0006] The encoding-decoding network expands the receptive field and extracts rich features through multi-layer convolution and pooling operations. This process not only obtains high-level semantic information, but also removes unnecessary details. However, it also loses spatial position information, thus affecting the detection accuracy of small cracks.

[0007] According to the ACI224R-01 document of the American Concrete Institute (ACI), cracks wider than 0.3 mm in concrete structures are considered unacceptable and can easily cause unsafe effects on the structure. In such images, the width of small cracks is usually only 1 to 2 pixels, and common encoding-decoding network architectures often have difficulty accurately detecting such cracks.

[0008] In the field of pixel-level crack detection, another less used network architecture is the high-resolution convolutional network without downsampling. For example, in 2020, Yue Fei et al. (Fei Y, Wang KCP, Zhang A, et al. Pixel-level cracking detection on 3D asphalt pavement images through deep-learning-based CrackNet-V. IEEE Transactions on Intelligent Transportation Systems, 2019, 21(1): 273-284. That is, pixel-level crack detection on 3D asphalt pavement images based on deep learning CrackNet-V, IEEE Transactions on Intelligent Transportation Systems, 2019, 21(1): 273-284) proposed the CrackNet-V network, which increases the depth of the network by stacking 8 3×3 convolutional layers and combines the responses of all channel information without losing resolution. However, as the authors of CrackNet-V wrote in the paper, the use of continuous 3×3 convolution kernels cannot fully represent large receptive fields, and the detection effect of long-distance or large-sized cracks is poor. In 2021, Chen H, Lin H. An effective hybrid atrous convolutional network for pixel-level crack detection [J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-12. That is, an effective hybrid atrous convolutional network for pixel-level crack detection, IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-12) proposed the HACNet model, which uses a hybrid atrous convolutional structure to increase the network's receptive field and improve the ability to extract features. HACNet has the advantages of simple structure and small number of parameters. It has certain advantages in detecting small cracks on a small-scale training set, but there are two problems: 1) Low computational efficiency, because the convolution operation is directly performed on a high-resolution image, lacking the ability to learn and fuse features at different resolutions, so it is difficult to build a deeper network structure, thereby limiting the realization of high-precision detection under large-scale data; 2) Insufficient feature expression ability, the model may treat all features equally, resulting in the inability to handle long-distance dependencies, affecting the quality of crack detection results.

[0009] Although the existing crack detection method of the encoding-decoding structure can identify cracks of different sizes, the effect of detecting small cracks, especially cracks with a width of only 1 to 2 pixels, is still not ideal. High-resolution convolutional networks without downsampling perform well in detecting small cracks, but it is difficult to balance the detection accuracy and efficiency. In addition, under the condition of limited number of convolutional layers and channels, the detection effect of large width and long distance cracks is not good. Therefore, there is still a lack of an efficient and full-size crack detection method. Summary of the invention

[0010] In order to overcome the shortcomings of the prior art, the present invention provides a crack detection method based on a high-resolution and large receptive field convolutional network. The HRLRF-Net model of the present invention realizes multi-scale feature extraction of a large receptive field by combining the hole convolution with different expansion rates with the spatial attention mechanism through hybrid connection; by designing an efficient multi-resolution feature extraction and fusion method, the problems of large computational complexity and insufficient feature expression ability of the high-resolution network are solved, thereby improving the accuracy and robustness of crack detection. The present invention is particularly suitable for detecting cracks of different sizes, especially showing advantages in the detection of ultra-thin cracks, and is therefore suitable for structural health detection of walls, roads and bridges.

[0011] In order to achieve the above object, the present invention provides the following technical solutions:

[0012] A crack detection method based on a high-resolution and large receptive field convolutional network comprises the following steps:

[0013] S01. Use a camera to take crack images on traffic facilities, annotate them with pixel-level labels, and build a crack dataset.

[0014] S02, build the HRLRF-Net model based on high resolution and large receptive field;

[0015] S03, train the HRLRF-Net model using the crack dataset;

[0016] S04. Apply the trained HRLRF-Net model for crack detection.

[0017] Further, in S02, the HRLRF-Net model based on high resolution and large receptive field includes:

[0018] High-resolution feature extractor: It has a diverse large receptive field feature extraction capability, used to extract and output crack feature information of the same size as the input image;

[0019] Output head: used to fuse the crack features of different semantic levels captured in the high-resolution feature extractor and output the final crack map, that is, the result of crack detection.

[0020] Preferably, the high-resolution feature extractor includes a high-resolution feature extraction channel and a medium-resolution feature extraction channel. The high-resolution feature extraction channel is composed of a convolutional layer with a stride (Stride) of 1 and three LRFAM (Large Receptive Field Attention Module) modules in cascade; the medium-resolution feature extraction channel includes a convolutional layer with a stride (Stride) of 2 and a cascade structure of three LRFAM modules. Finally, the features of the medium-resolution extraction channel are aggregated into the high-resolution feature channel to improve the extraction capability of high-resolution features.

[0021] The high-resolution feature extraction channel is the input of the predicted image, and the size of the predicted image is set to W×H×3. The first internal connection is a 5×5 convolution layer, and the number of convolution channels is C a , with a step size of 1, and performs BN (BatchNormalization) and ReLU operations to output the high-resolution initial feature F h0 , F h0 ∈R W×H×Ca ; It is then connected to three LRFAM modules in series for further feature extraction, and a feature map F of the same size and channel is obtained at the LRFAM cascade output position h1 、F h2 and F h3 The high-resolution initial feature is also connected to a medium-resolution feature extraction channel, which passes through a 5×5 convolution layer with a step size of 2, reducing the height and width of the feature map by 1 / 2, but doubling the number of channels. The output medium-resolution initial feature is F m0 , F m0 ∈R H / 2×W / 2×2Ca ; Then it is also connected to three LRFAM modules in series for further feature extraction, and also outputs feature maps of the same size and channel. Finally, the feature F at the output of the three LRFAM cascades of the medium resolution channel is m1 、F m2 、F m3 , through 1×1 convolution channel conversion and 2 times nearest neighbor interpolation upsampling, and the corresponding high-resolution channel feature F h1 、F h2 、F h3 Pixel-by-pixel addition is performed to achieve feature fusion. Finally, the high-resolution feature extractor outputs high-resolution feature maps F at three different stages. o1 、F o2 and F o3 .

[0022] Furthermore, the LRFAM is composed of a HybridASPA (Hybrid Atrous Spatial PyramidAttention Module) module and a ConvFFN module.

[0023] The HybridASPA module consists of a cascaded dilated convolution group and a channel attention module, and has a strong ability to extract crack features with large receptive fields. By cascading multiple dilated convolutions with exponentially increasing dilation rates, the convolution process of a large convolution kernel is simulated, so multiple features can be extracted from dilated convolution layers at different positions, and these features are regarded as the output of convolution layers with different convolution kernel sizes, thereby realizing multi-scale feature extraction. Subsequently, these features are approximated as multi-scale feature fusion by element-by-element addition. Although this stacking structure can realize large receptive fields and multi-scale convolution operations, all points on the feature map contribute the same to the input. Therefore, a channel attention mechanism is introduced in the module to dynamically adjust the weights of the feature maps, so that the model can more effectively utilize multi-scale information, thereby enhancing the feature representation capability.

[0024] Furthermore, in the HybridASPA module, the cascaded dilated convolution group is composed of four dilated convolution layers with a step size of 1 and a convolution kernel of 3×3, and the corresponding expansion rates are set according to the exponential growth method, which are 1, 3, 9 and 27 respectively, which is equivalent to realizing the feature extraction function of a large receptive field; let the input feature of HybridASPA be F i ,F i ∈R W×H×Ci , the number of input channels and output channels of each dilated convolution are the same; in order to reduce the amount of calculation, each dilated convolution layer consists of a stride of 1 and a channel number of C i / 2 1×1 convolution and a stride of 1, the number of channels is C i The output is connected to the BN layer and the ReLU activation function. The 1×1 convolution is used for channel compression, and then the 3×3 dilated convolution is used to increase the number of channels, and finally the number of input and output channels of each dilated convolution layer is kept consistent; at the input of the first dilated convolution and the feature output of each dilated convolution layer, a jump connection is made to the output of the last dilated convolution, and the features of all dilated convolution layers are added element by element to create feature fusion under different receptive fields; the output feature of the cascaded dilated convolution is F s ∈R W×H×Ci , whose input and output feature channels have the same number and size.

[0025] The channel attention module is constructed by globally focusing on the feature map F sThe weight of each channel is adjusted so that the network can better understand the global semantic information in the image, which is especially helpful for processing complex background scenes of cracks and fineness extraction. The improved efficient compression and excitation module is used here, which consists of an average pooling layer and a 1×1 convolution layer. The average pooling layer mainly generates pooling features F through average pooling operations along the channel axis. pavg , where F pavg ∈R W×H×1 , followed by a 1×1 convolutional layer and normalized by the Sigmoid function; the calculation process is as follows: A sag (X i )=σ(f 1×1 (F favg )), where σ represents the Sigmoid function, f 1×1 is a 3×3 convolutional layer; finally, the attention-guided feature map F sag ∈R W×H×Ci Calculated as: in Represents element-wise multiplication.

[0026] In the ConvFFN module, a pair of 1×3 and 3×1 one-dimensional convolutions are used to refine the features. The output is connected to the BN layer and the ReLU activation function. The output feature is F o ∈R W×H×Ci This type of one-dimensional convolution has the advantages of fewer parameters and less computation.

[0027] The output head of the HRLRF-Net model adopts 1×1 convolution with an output channel number of 1, which has the advantages of simple structure and small computation amount; the input of the output head is the high-resolution feature map F of three different stages of the high-resolution feature extractor o1 、F o2 and F o3 These features are first input into a 1×1 convolution with an output channel of 1 and a stride of 1 to reduce the number of feature channels to 1; then, these three channels are stacked (concatenate), and finally connected to a 1×1 convolution with a channel of 1 for feature fusion, and the output is connected to the Sigmoid activation function as the final output end to output the final crack map. By setting the threshold to 0.3, the output pixel value greater than 0.3 is considered a crack, otherwise it is background.

[0028] The technical concept of the present invention is: in view of the problems of large amount of calculation and insufficient feature expression ability in the high-resolution convolution network without subsampling, a HRLRF-Net (High Resolution and Large Receptive Field Network) model is proposed, which can retain the advantage of spatial accuracy and improve the calculation efficiency and detection accuracy; in terms of calculation efficiency, the present invention first realizes large receptive field feature extraction without significantly increasing the amount of calculation by cascading dilated convolutions with different dilation rates; secondly, by reducing the number of channels of high-resolution convolution, using 1×1 convolution to compress the dilated convolution channels, and then using 3×3 dilated convolution to increase the number of channels, the amount of calculation is further reduced. In terms of detection accuracy, firstly, the medium-resolution feature extraction channel branch is added, and the medium-resolution features of each stage are aggregated into the high-resolution feature channel, so that the model can use global and local information at the same time, thereby improving the accuracy and robustness of crack detection; secondly, a simple and efficient channel attention mechanism is introduced to dynamically adjust the weight of the channel feature map, more effectively use multi-scale information, and enhance the feature representation ability of the model. By integrating the above four methods, the proposed HRLRF-Net achieves a good balance between detection delay and detection accuracy and has the ability to detect cracks of different sizes.

[0029] Compared with the prior art, the benefits of the present invention are mainly reflected in:

[0030] (1) The method of the present invention adopts a high-resolution convolutional network technology route. Through a simple and effective method, the spatial accuracy advantage of the high-resolution convolutional network is retained, and the problems of large computational complexity and insufficient feature expression ability of the high-resolution convolutional network are solved through an optimization method, thereby meeting the requirements of full-size crack detection;

[0031] (2) The constructed HybridASPA module has both large receptive field and strong feature extraction capability. It realizes multi-scale feature extraction by mixing and connecting dilated convolutions with different dilation rates, and adds a channel attention mechanism to dynamically adjust the feature map weights to enhance the feature representation capability.

[0032] (3) Multi-resolution feature fusion: add a medium-resolution feature extraction channel to the high-resolution feature channel and integrate the features of each stage into the high-resolution feature. Since medium-resolution features usually contain more global and abstract information and can capture accurate spatial information, while high-resolution features contain more local details and can capture more detailed features, by fusing the two resolution features, global and local information can be comprehensively utilized, thereby improving the accuracy and robustness of crack detection.

[0033] (4) Flexible network structure,The overall setting of HRLRF-Net is highly flexible. Since the high-resolution feature extractor is similar to a matrix structure and all LRFAM structures are exactly the same, the width and depth of the backbone network can be quickly adjusted to meet the needs of different detection tasks, thereby achieving a balance between detection accuracy, speed and resource consumption.

[0034] In summary, based on the HRLRF-Net model, the present invention adopts the combination of dilated convolution hybrid cascade and attention mechanism, and the fusion of high-resolution features and medium-resolution features to enhance the feature extraction capability of the high-resolution crack detection network and realize the detection capability of cracks of different sizes. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the overall process of the crack detection method with high resolution and large receptive field according to an embodiment of the present invention;

[0036] Figure 2 is a structural diagram of the HRLRF-Net model of an embodiment of the present invention;

[0037] Figure 3 It is a structural diagram of the LRFAM module of an embodiment of the present invention;

[0038] Figure 4 Schematic diagram of the equivalent receptive field and attention mechanism of the HybridASPA module of an embodiment of the present invention;

[0039] Figure 5 Schematic diagram of equivalent receptive field simulation of the HybridASPA module according to an embodiment of the present invention;

[0040] Figure 6 This is a comparison chart of crack detection effects between the embodiments of the present invention and other methods. DETAILED DESCRIPTION

[0041] The present invention will be further described below in conjunction with the accompanying drawings.

[0042] Reference Figure 1 to Figure 5 , a crack detection method based on high-resolution and large receptive field convolutional network, comprising the following steps:

[0043] S01: Use a camera to capture crack data on traffic facilities, annotate them with pixel-level labels, and construct a crack dataset. The process is as follows:

[0044] (1.1) Take images of surface cracks on bridges, walls, and roads to ensure that the images are clear and the crack details are obvious. The crack images taken should include various backgrounds and lighting conditions to truly reflect the crack morphology in traffic facilities, which will help the trained model perform well in different practical application scenarios.

[0045] (1.2) Use the Labelme annotation tool to annotate the acquired crack image and manually draw the crack area in the image, trying to fit the crack edge as closely as possible to ensure the accuracy of the annotation.

[0046] (1.3) The annotated crack images are divided into a training set and a validation set in a ratio of 8:2.

[0047] S02: Build a HRLRF-Net model based on high resolution and large receptive field;

[0048] Figure 2 This is a schematic diagram of the structure of HRLRF-Net. The overall structure of the network mainly consists of two parts:

[0049] Lower layer: High-resolution feature extractor, which consists of high-resolution feature extraction channel and medium-resolution feature extraction channel, and finally outputs a set of high-resolution feature maps with the same size as the input image. Both channels have a large receptive field and can capture the global context feature information of cracks; the high-resolution channel focuses on extracting features containing more local details and is good at detecting slender cracks; the medium-resolution channel focuses on extracting more global and abstract information. By fusing medium-resolution features with high-resolution features, the model can comprehensively utilize global and local information, thereby improving the accuracy and robustness of crack detection.

[0050] The upper layer: the output head is responsible for fusing the multi-level features of the feature extractor and outputting the final crack detection results. Since all 1×1 convolution operations with an output channel number of 1 are used, the amount of calculation is very small.

[0051] The processing flow of the HRLRF-Net model is described as follows in the order from input to output:

[0052] (2.1) Image preprocessing: The image captured by the camera is set to a size of W×H×3 and normalized, that is, the input image is divided by 255, and then the corresponding channel mean is subtracted from each channel and divided by the channel standard deviation. The normalized image will be input into the high-resolution feature extractor.

[0053] (2.2) High-resolution feature extractor: It consists of a high-resolution feature extraction channel and a medium-resolution feature extraction channel.

[0054] High-resolution feature extraction channel: First, a 5×5 convolutional layer with a channel number of 32 and a stride of 1 is passed, and BN and ReLU operations are performed to obtain the initial features F of the input image. h0 , where F h0 ∈R W×H×32Next, three cascaded LRFAM models are used to further extract features, and the feature map F is obtained at the LRFAM cascade output position. h1 、F h2 and F h3 , and they all have the same size and number of channels.

[0055] Medium-resolution feature extraction channel: Connect the 5×5 convolution output of the high-resolution channel to the medium-resolution feature extraction channel, that is, input the initial feature map to the medium-resolution channel at the same time. This channel first passes through a 5×5 convolution with a channel number of 64 and a stride of 2, and performs BN and ReLU operations to output the medium-resolution initial feature F m0 ∈R H / 2×W / 2×64 , the height and width of the feature map are half of the original, while the number of channels is doubled. Subsequently, the features are further extracted through three cascaded LRFAMs to obtain the feature map F m0 、F m1 and F m2 Finally, these features are upsampled by 1×1 convolution channel conversion and 2 times nearest neighbor interpolation, and then combined with the high-resolution channel features F h0 、F h1 and F h2 Add element by element to perform feature fusion. The number of output channels of the 1×1 convolution is 32, and the output is connected to the BN and ReLU activation functions.

[0056] Figure 3 The schematic diagram of the LRFAM module describes its internal structure in detail. Table 1 shows the detailed parameters of LRFAM. The number of channels, height and width of the input and output feature maps of the entire module remain unchanged.

[0057] The LRFAM module consists of two parts: the HybridASPA module and the ConvFFN module. The specific parameters are shown in Table 1.

[0058]

[0059]

[0060] Table 1

[0061] The HybridASPA module consists of a cascaded dilated convolution group and a channel attention module, which is used to extract and fuse features with different receptive fields, and introduces a channel attention mechanism to enhance feature representation. The cascaded dilated convolution group is composed of multiple dilated convolutions with a step size of 1 and the same convolution kernel size in series. The receptive field characteristic study shows that when the dilation rate is set to an exponentially growing value k, the dilation rate is increased by 1. iWhen , the network can generate the maximum receptive field without grid effect, which means that all points on the feature map have the same contribution to the input. Where i represents the i-th dilated convolution. Specifically, four dilated convolutions with a kernel size of 3 are connected in series, and their dilation rates are set to 1, 3, 9, and 27 respectively. This design can efficiently capture diverse receptive fields and has fewer module parameters, thus achieving a balance between accuracy and speed.

[0062] The structure is described in the order from input to output as follows: First, it consists of four dilated convolution layers with a stride of 1 and a convolution kernel of 3×3, and the dilation rates are set to 1, 3, 9, and 27, respectively, which are equivalent to the feature extraction function of a large receptive field. The number of input channels and output channels of each dilated convolution is the same, and there is an output channel number C i / 2 1×1 convolution and an output channel number C i The 1×1 convolution is used for channel compression, while the 3×3 dilated convolution is used to increase the number of channels to reduce the amount of computation. The output of each dilated convolution layer is connected to a BN and ReLU activation function. At the input a of the first dilated convolution layer and the feature outputs b, c, and d of each dilated convolution layer, a jump connection is made to the output e of the last dilated convolution, and the outputs of all dilated convolution layers are added element by element. Figure 3 Medium Representation, forming an aggregation of multi-scale features.

[0063] Figure 4 The left side shows the equivalent receptive field diagram of the HybridASPA cascaded hole convolution group. This cascade method is equivalent to superimposing the features after convolution operations with different receptive field sizes, thereby approximately achieving multi-scale feature fusion. Figure 5 The simulation results of the equivalent receptive field of the HybridASPA module are shown. The dark area represents the calculated receptive field size and response. The simulation graph is calculated using the Receptivefield software, which aims to estimate the width and height of the receptive field by backpropagating the gradient of the selected feature map output to the input. This helps to understand the feature extraction capability of the model and its multi-scale feature fusion effect. As can be seen from the figure, the feature maps F at points b, c, d, and e on the HybridASPA module b 、F c 、F d and F e The contributions to the input are the same, and the effective receptive field sizes are 3, 9, 27, and 81, respectively, which are equivalent to convolution operations with convolution kernel sizes of 3, 9, 27, and 81.

[0064] The channel attention module enables the network to better understand the global semantic information in the image by adjusting the weight of each channel of the feature map globally. This is especially helpful for processing complex background scenes of cracks and fineness extraction. The improved efficient squeeze and excitation module is used here, including:

[0065] Global pooling: Use an average pooling layer to generate pooling features F through average pooling operations along the channel axis pavg , where F pavg ∈R W×H×1 . Convolutional normalization: Use a 1×1 convolutional layer to normalize F pavg Processed and normalized by Sigmoid function. The calculation process is summarized as follows: sag (X i )=σ(f 1×1 (F pavg )), where σ represents the Sigmoid function, f 1×1 is a 1×1 convolutional layer. Finally, the attention-guided feature map F sag ∈R W×H×Ci , calculated as: in Represents element-wise multiplication. Figure 4 The right side shows a schematic diagram of the effect of the attention mechanism, showing how the weights of the feature map are dynamically adjusted through the channel attention mechanism.

[0066] The ConvFFN module is a convolution-based Feed-Forward Network (FFN) for feature transformation and extraction to enhance the model’s expressiveness. ConvFFN consists of a pair of 1×3 and 3×1 one-dimensional convolutions, which are connected to the BN layer and the ReLU activation function after the output. The final output feature is F o ∈R W×H×Ci This one-dimensional convolution has the advantages of fewer parameters and less computation.

[0067] (2.3) In the output stage, the HRLRF-Net model extracts features F at different stages from the LRFAM of the high-resolution channel. o1 、F o2 and F o3 As the input of the output head, the operation is as follows: the three features are processed by a 1×1 convolution with an output channel number of 1 and a stride of 1, so that the number of channels of each feature is reduced to 1; then, the three channel features are stacked (concatenate); finally, they are connected to a 1×1 convolution with a channel number of 1 and a stride of 1 for feature fusion, and the final crack mapping output is generated through the Sigmoid activation function.

[0068] S03: Use the crack dataset to train the HRLRF-Net model;

[0069] In step S01, a crack dataset with pixel-level labels has been constructed and trained end-to-end under a supervised learning method to enhance the learning ability of HRLRF-Net under different scale changes. At the same time, a multi-layer supervision strategy is applied during the training process:

[0070] A Sigmoid activation function and a binary cross entropy loss (BCE Loss) function are added to each LRFAM side output branch on the high-resolution channel. This approach generates multiple losses and shortens the path of gradient backpropagation, enabling the model to automatically learn and combine complementary information from various layers of the convolutional neural network (CNN), thereby improving the optimization and generalization capabilities of pixel-level prediction tasks.

[0071] In this embodiment, the hardware is a compatible computer, using two Nvidia RTX3090 GPUs with a total video memory of 48G. The software is an Ubuntu 20.04 64-bit operating system, and the HRLRF-Net model is built on the Pytorch deep learning framework using the Python language.

[0072] Model training parameter settings: image size normalized to 400×400, Adam optimizer used, weight decay to 0.000001, batch-sizes set to 12, initial learning rate set to 0.005, and reduced by 30% at the 30th, 60th and 90th epochs, training cycle: fixed 150 training epochs, and select the model with the best verification performance in 150 epochs as the test model. The optimal model parameters W obtained by training m ;

[0073] S04: Apply the trained HRLRF-Net model for crack detection.

[0074] The predicted image x is used as input, and after being processed by the HRLRF-Net model, the pixel-level crack detection result is output. The formula is expressed as: crack_map =F seg (x; W m );

[0075] Precision, recall, F1 score, and intersection-over-union (IoU) are used to evaluate the crack detection results. The definitions of the evaluation indicators are as follows:

[0076] 1)

[0077] 2)

[0078] 3)

[0079] 4)

[0080] Among them, TP is the number of pixels correctly detected as cracks, FP is the number of pixels in the non-crack area mistakenly detected as cracks, FN is the number of pixels not correctly detected as cracks, and N is the number of samples in the validation set. In the experiment, the output threshold is set to 0.3, and the area with an output pixel value greater than 0.3 is regarded as a crack, otherwise it is regarded as background.

[0081] This example uses the public large-scale dataset CrackSeg9K as experimental data, which contains 9159 image samples and labels. CrackSeg9K is a collection of public crack datasets such as Masonry, CFD, CrackTree200, Volker, DeepCrack, Ceramic, SDNET2018, Rissbilder, Crack500 and GAPS384, mainly including cracks of building walls, road pavements and bridges. The experiment divides the dataset into a training set of 7332 samples and a test set of 1827 samples.

[0082] The present invention compares three common crack detection models: DeepCrack, HACNet and CrackFormer. Table 2 shows the comparison results of different algorithms in terms of detection accuracy, speed, etc. on the CrackSeg9K dataset.

[0083]

[0084]

[0085] Table 2

[0086] The HRLRF-Net model of the present invention achieves an F1 score of 77.6%, which is 3.6%, 3.2% and 1.2% higher than DeepCrack, HACNet and CrackFormer respectively. In terms of IoU score, the HRLRF-Net model also improves by 2.4%, 2.1% and 0.7% respectively compared with other models. In terms of the number of model parameters, HRLRF-Net is significantly smaller than DeepCrack and CrackFormer of the encoding-decoding architecture; although HRLRF-Net is slightly inferior to HACNet in prediction speed, it is still more than 3 times faster than DeepCrack and CrackFormer. Therefore, considering the detection effect and calculation speed, the HRLRF-Net model can better meet the needs of actual crack detection.

[0087] Qualitative test results such as Figure 6 As shown in the figure, several samples with complex scenes are selected. Each image has uneven grayscale, light and shadow occlusion, and cracks of different thicknesses. From the comparison of the first, second, third, and fourth rows, it can be seen that the HRLRF-Net model is closer to the actual situation in terms of the accuracy and completeness of the crack edge. The fifth and sixth rows of the figure show the visualization results of the slender crack samples. The HRLRF-Net model can more accurately predict slender and shallow cracks, showing its robustness and superiority under different crack conditions.

[0088] This embodiment provides a specific implementation of the present invention, which is intended to help understand or implement the present invention. The protection scope of the present invention should not be considered to be limited to the specific form described in this embodiment, but should protect the technical features of the present invention within the spirit of the present invention and the protection scope of the claims.

Claims

1. A crack detection method based on high-resolution and large receptive field convolutional network, characterized in that: The method comprises the following steps: S01. Use a camera to take crack images on traffic facilities, annotate them with pixel-level labels, and build a crack dataset. S02, build the HRLRF-Net model based on high resolution and large receptive field; S03, train the HRLRF-Net model using the crack dataset; S04. Apply the trained HRLRF-Net model for crack detection.

2. A crack detection method based on a high-resolution and large receptive field convolutional network as claimed in claim 1, characterized in that: In S02, the HRLRF-Net model based on high resolution and large receptive field includes: High-resolution feature extractor with diverse large receptive field feature extraction capabilities, used to extract and output crack feature information of the same size as the input image; The output head is used to fuse the crack features at different semantic levels captured by the high-resolution feature extractor and output the final crack map, which is the result of crack detection.

3. A crack detection method based on a high-resolution and large receptive field convolutional network as claimed in claim 2, characterized in that: The high-resolution feature extractor includes a high-resolution feature extraction channel and a medium-resolution feature extraction channel. The high-resolution feature extraction channel consists of a convolution layer with a step size of 1 and three LRFAM modules in cascade; the medium-resolution feature extraction channel includes a convolution layer with a step size of 2 and a cascade structure of three LRFAM modules. Finally, the features of the medium-resolution extraction channel are aggregated into the high-resolution feature channel to improve the extraction capability of high-resolution features.

4. A crack detection method based on a high-resolution and large receptive field convolutional network as claimed in claim 3, characterized in that: The high-resolution feature extraction channel is the input of the predicted image. The size of the predicted image is set to W×H×3, and the first internal connection is a 5×5 convolution layer with C convolution channels. a , with a step size of 1, and performs BN and ReLU operations to output the high-resolution initial feature F h0 , F h0 ∈R W×H×Ca ; It is then connected to three LRFAM modules in series for further feature extraction, and a feature map F of the same size and channel is obtained at the LRFAM cascade output position h1 、F h2 and F h3 The high-resolution initial feature is also connected to a medium-resolution feature extraction channel, which passes through a 5×5 convolution layer with a step size of 2, reducing the height and width of the feature map by 1 / 2, but doubling the number of channels. The output medium-resolution initial feature is F m0 , F m0 ∈R H / 2×W / 2×2Ca ; Then it is also connected to three LRFAM modules in series for further feature extraction, and also outputs feature maps of the same size and channel. Finally, the feature F at the cascade output of the three LRFAMs of the medium resolution channel is m1 、F m2 、F m3 , through 1×1 convolution channel conversion and 2 times nearest neighbor interpolation upsampling, and the corresponding high-resolution channel feature F h1 、F h2 、F h3 Pixel by pixel addition to achieve feature fusion, and finally the high-resolution feature extractor outputs three different stages of high-resolution feature maps F o1 、F o2 and F o3 .

5. A crack detection method based on a high-resolution and large receptive field convolutional network as claimed in claim 2, characterized in that: The LRFAM consists of a HybridASPA module and a ConvFFN module.

6. A crack detection method based on a high-resolution and large receptive field convolutional network as claimed in claim 5, characterized in that: The HybridASPA module consists of a cascaded dilated convolution group and a channel attention module, and has a strong ability to extract crack features with large receptive fields. By connecting multiple dilated convolutions with exponentially increasing dilation rates in series, the convolution process of a large convolution kernel is simulated, so multiple features can be extracted from dilated convolution layers at different positions, and these features are regarded as convolution layer outputs with different convolution kernel sizes, thereby realizing multi-scale feature extraction. Subsequently, these features are approximated as multi-scale feature fusion by element-by-element addition. Although this stacked structure can realize large receptive field and multi-scale convolution operations, all points on the feature map contribute the same to the input. Therefore, a channel attention mechanism is introduced in the module to dynamically adjust the weights of the feature maps so that the model can more effectively utilize multi-scale information.

7. A crack detection method based on a high-resolution and large receptive field convolutional network as claimed in claim 5, characterized in that: In the HybridASPA module, the cascaded dilated convolution group is composed of four dilated convolution layers with a step size of 1 and a convolution kernel of 3×3, and the corresponding expansion rates are set according to the exponential growth method, which are 1, 3, 9 and 27 respectively, which is equivalent to realizing the feature extraction function of a large receptive field. Assume that the input feature of HybridASPA is F i ,F i ∈R W×H×Ci , the number of input channels and output channels of each dilated convolution is the same; each dilated convolution layer consists of a stride of 1 and a number of channels C i / 2 1×1 convolution and a stride of 1, the number of channels is C i The output is connected to the BN layer and the ReLU activation function, where the 1×1 convolution is used for channel compression, and then the 3×3 dilated convolution is used to increase the number of channels, and finally the number of input and output channels of each dilated convolution layer is kept consistent; at the input of the first dilated convolution and the feature output of each dilated convolution layer, a jump connection is made to the output of the last dilated convolution, and the features of all dilated convolution layers are added element by element to create feature fusion under different receptive fields; the output feature of the cascaded dilated convolution is F s ∈R W×H×Ci , whose input and output feature channels have the same number and size.

8. A crack detection method based on a high-resolution and large receptive field convolutional network as claimed in claim 5, characterized in that: The channel attention module is constructed by globally focusing on the feature map F s The weight of each channel is adjusted so that the network can better understand the global semantic information in the image. The improved efficient compression excitation module is used here, which consists of an average pooling layer and a 1×1 convolution layer. The average pooling layer generates the pooling feature F through the average pooling operation along the channel axis. pavg , where F pavg ∈R W×H×1 , followed by a 1×1 convolutional layer and normalized by the Sigmoid function; the calculation process is as follows: A sag (X i )=σ(f 1×1 (F pavg )), where σ represents the Sigmoid function, f 1×1 is a 3×3 convolutional layer; finally, the attention-guided feature map F sag ∈R W×H×Ci Calculated as: in Represents element-wise multiplication.

9. A crack detection method based on a high-resolution and large receptive field convolutional network as claimed in claim 5, characterized in that: In the ConvFFN module, a pair of 1×3 and 3×1 one-dimensional convolutions are used to refine the features. The output is connected to the BN layer and the ReLU activation function. The output feature is F o ∈R W×H×Ci .

10. A crack detection method based on a high-resolution and large receptive field convolutional network as claimed in claim 2, characterized in that: The output head of the HRLRF-Net model adopts 1×1 convolution with an output channel number of 1, which has a simple structure and small computational complexity; the input of the output head is the high-resolution feature map F of three different stages of the high-resolution feature extractor o1 、F o2 and F o3 ,These features are first input into a 1×1 convolution with an output channel of 1 and a stride of 1, reducing the number of feature channels to 1; Then, these three channels are stacked and finally connected to a 1×1 convolution with a channel number of 1 for feature fusion. The output is connected to the Sigmoid activation function as the final output end to output the final crack map. By setting the threshold, the output pixel value greater than the set threshold is a crack, otherwise it is the background.

Images