Pavement crack detection method for different environments

By combining the brightness adaptive mechanism of deep learning and biovision system and Gabor differential blocking, a lightweight pavement crack detection model is proposed, which solves the problems of poor accuracy and low efficiency of pavement crack detection in different environments, and achieves high efficiency Improved crack feature extraction and detection performance.

CN118967638AActive Publication Date: 2024-11-15SOUTHWEST JIAOTONG UNIV

Patent Information

Application Number
CN202411093911.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-11-15
Estimated Expiration
2044-08-09

AI Technical Summary

Technical Problem

The existing pavement crack detection methods have poor detection accuracy in different environments, complex model structure, large number of parameters, low calculation efficiency, difficult to adapt to weather and light changes, and difficult to capture the shape and directional characteristics of pavement cracks.

Method used

A lightweight pavement crack detection model is proposed, combining the brightness adaptive mechanism of deep learning and biovision system, and through the brightness adaptive adjustment and feature processing module, the Gabor differential block and the probability weighted fusion module, the pavement cracks in different environments are realized Test.

Benefits of technology

It realizes efficient road surface crack detection in different environments, reduces the number of parameters and calculation overhead, extracts the shape and direction characteristics of the cracks, reduces background texture interference, and improves detection performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967638A_ABST
    Figure CN118967638A_ABST
Patent Text Reader

Abstract

The invention discloses a pavement crack detection method for different environments, and belongs to the technical field of image detection, and the method comprises the following steps: inputting a pavement crack image into a brightness adaptive adjustment and feature processing module, and carrying out the feature preliminary extraction to obtain a pavement crack preliminary feature; inputting the preliminary features of the pavement cracks into an encoder network for encoding step by step to obtain encoding features of the pavement cracks; and inputting the road surface crack coding features into a decoder network for feature weighted fusion to obtain final features of the road surface crack, and realizing road surface crack detection through the final features of the road surface crack. According to the invention, pavement crack detection in different environments is realized, and the problems of complex structure, large number of parameters and low efficiency of an existing model for pavement crack detection are solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of image detection, and in particular relates to a road surface crack detection method for different environments. Background Art

[0002] Road crack detection aims to extract and identify cracks from the road surface in order to evaluate the road condition and provide assistance for road maintenance and repair work. Timely and accurate detection of road cracks is of great significance, which can not only prevent the expansion of road damage and reduce the cost of road maintenance, but also further reduce the occurrence of traffic accidents and improve people's travel safety. However, since road cracks usually have large shape differences, diverse directions, poor continuity, and are easily disturbed by environmental factors (such as weather, light, and stains) during the actual collection process, it is difficult to improve the accuracy of current crack detection methods. This also makes road crack detection one of the challenging problems in visual tasks.

[0003] In recent years, deep learning technology has been widely used in computer vision tasks due to its excellent performance. Compared with early methods, methods based on convolutional neural networks and Transformer can achieve accurate crack extraction without manual feature engineering, while showing strong robustness and high detection performance. However, most current methods only use crack data acquired under standard ambient lighting conditions for training and testing, ignoring the impact of external environmental factors such as weather and lighting changes on crack detection. This omission limits the generalization ability of the model and the scope of applicable scenarios.

[0004] In addition, the diversity of pavement crack shapes and directions makes it difficult for previous methods to capture comprehensive crack information, which affects the detection and generalization performance of the model. In order to obtain better detection performance, previous studies often focus on the development of complex models, but ignore the actual application efficiency of these models. Therefore, many current deep learning methods face challenges such as high complexity, large parameter sets, low computational efficiency, and insufficient optimization for mobile platforms. Summary of the invention

[0005] The present invention provides a pavement crack detection method for different environments, realizes pavement crack detection in different environments, and solves the problems of complex model structure, large number of parameters and low efficiency in pavement crack detection.

[0006] In order to solve the above technical problems, the technical solution of the present invention is: a pavement crack detection method for different environments, comprising the following steps:

[0007] S1, inputting the pavement crack image into the brightness adaptive adjustment and feature processing module, performing preliminary feature extraction, and obtaining preliminary features of the pavement crack;

[0008] S2, inputting the preliminary features of the pavement cracks into the encoder network for step-by-step encoding to obtain the pavement crack encoding features;

[0009] S3. Input the pavement crack coding features into the decoder network for feature weighted fusion to obtain the final features of the pavement cracks, and realize pavement crack detection through the final features of the pavement cracks.

[0010] Furthermore, the brightness adaptive adjustment and feature processing module, the encoder network and the decoder network are sequentially connected to form a lightweight pavement crack detection model;

[0011] The loss function of the lightweight pavement crack detection model is expressed as:

[0012]

[0013] Among them, L(·) represents the cross entropy loss function, y represents the true label, p represents the predicted result, and y j represents the true label of the jth pixel, p j Represents the prediction result of the jth pixel.

[0014] Furthermore, the brightness adaptive adjustment and feature processing module includes a brightness adaptive unit, a cone cell processing unit and a rod cell processing unit; the brightness adaptive unit is connected to the cone cell processing unit and the rod cell processing unit respectively.

[0015] Furthermore, the brightness adaptation unit uses a brightness adaptation function to adjust the brightness of the input image;

[0016] The expression of the brightness adaptation function is:

[0017]

[0018] Among them, I a represents the output feature of the brightness adaptive unit, f(·) represents the brightness adaptive function, I represents the input image, ω represents the learnable parameter, V max represents the maximum brightness of the input image, and L represents the brightness of the input image;

[0019] The output characteristic of the brightness adaptive unit is a Includes R channel feature I aR , G channel feature I aG , B channel feature I aB and grayscale feature I ag ;

[0020] The expression of the brightness adaptive adjustment and feature processing module is:

[0021] I c =C 3×3 (C 3×3 (I aR ))+C 3×3 (C 3×3 (I aG ))+C 3×3 (C 3×3 (I aB ))

[0022] I r =C 3×3,4 (C 3×3,2 (I ag ))

[0023] I o =I c +I r

[0024] Among them, I c represents the output feature of the cone processing unit, I r represents the output characteristics of the rod cell processing unit, I o Indicates the preliminary characteristics of pavement cracks, C 3×3 (·) represents a regular convolution with a kernel of 3, C 3×3,4 (·) represents the dilated convolution with a dilation rate of 4, C 3×3,2 (·) Dilated convolution with dilation rate 2.

[0025] Further, the encoder network includes four Gabor difference blocks and three maximum pooling layers;

[0026] The four Gabor difference blocks include a first Gabor difference block, a second Gabor difference block, a third Gabor difference block and a fourth Gabor difference block; the three maximum pooling layers include a first maximum pooling layer, a second maximum pooling layer and a third maximum pooling layer;

[0027] The first Gabor difference block, the first maximum pooling layer, the second Gabor difference block, the second maximum pooling layer, the third Gabor difference block, the third maximum pooling layer and the fourth Gabor difference block are connected in sequence;

[0028] The first Gabor difference block outputs a first coding feature; the second Gabor difference block outputs a second coding feature; the third Gabor difference block outputs a third coding feature; and the fourth Gabor difference block outputs a fourth coding feature;

[0029] The first coding feature, the second coding feature, the third coding feature and the fourth coding feature are collectively used as pavement crack coding features.

[0030] Furthermore, the expression of the Gabor difference block is:

[0031] C 1 =C 3×3 (GD_layer(I G ))

[0032] C 2 =C 3×3 (GD_layer(I G ))

[0033] W=Sigmoid(C 1×1 (C 1 ))

[0034] O G =C 2 +W×C 1

[0035] Among them, C 1 represents the first differential feature, C 2 represents the second differential feature, I G represents the input feature of Gabor difference block, GD_layer(·) represents Gabor difference layer, C 3×3 (·) represents a regular convolution with a kernel of 3, C 1×1 (·) represents a regular convolution with a convolution kernel of 1, W represents a weight parameter, Sigmoid (·) represents an activation function, and O G Represents the output features of the Gabor difference block.

[0036] Furthermore, the expression of the Gabor difference layer is:

[0037] O max =max(Cat(Gcd*I GD_layer ,Gad*I GD_layer ))

[0038] O min =min(Cat(Gcd*I GD_layer ,Gad*I GD_layer ))

[0039] O GD_layer =C 1×1 (Cat(O max ,O min ))

[0040] Among them, O max represents the maximum difference feature, O minrepresents the minimum difference feature, max(·) represents the maximum value, min(·) represents the minimum value, Gcd represents the first convolution kernel composed of the center difference and the Gabor filter, Gad represents the second convolution kernel composed of the angle difference and the Gabor filter, * represents the convolution operation, I GD_layer represents the input feature of Gabor difference layer, Cat(·) represents the concatenation operation, O GD_layer Represents the output features of Gabor difference layer.

[0041] Further, the decoder network includes six probability weighted fusion modules;

[0042] The six probability weighted fusion modules include a first probability weighted fusion module, a second probability weighted fusion module, a third probability weighted fusion module, a fourth probability weighted fusion module, a fifth probability weighted fusion module and a sixth probability weighted fusion module;

[0043] The input features of the first probability weighted fusion module are the first coding feature and the second coding feature; the input features of the second probability weighted fusion module are the second coding feature and the third coding feature; the input features of the third probability weighted fusion module are the third coding feature and the fourth coding feature;

[0044] The input features of the fourth probability weighted fusion module are the output features of the first probability weighted fusion module and the output features of the second probability weighted fusion module; the input features of the fifth probability weighted fusion module are the output features of the second probability weighted fusion module and the output features of the third probability weighted fusion module; the input features of the sixth probability weighted fusion module are the output features of the fourth probability weighted fusion module and the output features of the fifth probability weighted fusion module;

[0045] The output features of the sixth probability weighted fusion module are input into a 1×1 convolution to obtain the final features of the pavement cracks.

[0046] Furthermore, the expression of the probability weighted fusion module is:

[0047] O PWF =PW(I 1 )+Upsample(PW(I 2 ))

[0048] Among them, O PWF represents the output feature of the probability weighted fusion module, PW(·) represents the probability weight unit, and I 1 represents the first input feature of the probability weighted fusion module, I 2 represents the second input feature of the probability weighted fusion module, and Upsample(·) represents the upsampling operation.

[0049] Furthermore, the expression of the probability weight unit is:

[0050] C out =C 3×3,2 (C 3×3 (I PW ))

[0051] PW out =Sigmoid(C 1×1 (C out ))×C out

[0052] σ=mean(Sigmoid(C 1×1 (C out )))

[0053] PW fusion =PW out +σ×C out

[0054] Among them, C out represents the expansion characteristic, C 3×3,2 (·) Dilated convolution with a dilation rate of 2, C 3×3 (·) represents a regular convolution with a kernel of 3, I PW Represents the input features of the probability weight unit, PW out represents the weight matrix, C 1×1 (·) represents a regular convolution with a convolution kernel of 1, Sigmoid(·) represents the activation function, σ represents the mean of the weight matrix, mean(·) represents the average value, PW fusion Represents the output features of the probability weight unit.

[0055] The beneficial effects of the present invention are as follows: (1) a lightweight pavement crack detection model is proposed, which combines deep learning technology with the brightness adaptation mechanism in the biological vision system, realizes adaptive adjustment and feature processing of input images in different environments, and has higher efficiency, fewer parameters and lower computational overhead;

[0056] (2) Through the brightness adaptive feature processing module, the brightness adaptation mechanism of the biological visual system is simulated to adjust the brightness and process the features of the input image;

[0057] (3) In response to the significant changes in crack shape and the challenges of diverse directions, a Gabor difference block is proposed to simulate the response of the biological visual system to visual stimuli of different directions and frequencies. It can extract the shape and direction of the crack while suppressing the interference of background texture, thus realizing the extraction of crack shape and direction features.

[0058] (4) The probability weighted fusion module is used to pair and fuse adjacent side output features, which promotes effective feature extraction and reduces the influence of background texture. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 The present invention is a flow chart of the pavement crack detection method for different environments.

[0060] Figure 2 This is a structural diagram of the lightweight pavement crack detection model of the present invention.

[0061] Figure 3 It is a structural diagram of the brightness adaptive adjustment and feature processing module of the present invention.

[0062] Figure 4 This is a structural diagram of the Gabor difference block of the present invention.

[0063] Figure 5 It is a structural diagram of the Gabor difference layer of the present invention.

[0064] Figure 6 It is a structural diagram of the probability weighted fusion module of the present invention.

[0065] Figure 7 Schematic diagram of the ablation experiment results of the present invention on the BJN260 dataset.

[0066] Figure 8 It is a schematic diagram of the results of the present invention before and after the brightness adaptation function processing of some BJN260 and Rain365 images.

[0067] Fig. 9 This is a comparison chart of the image clarity results of some BJN260 and Rain365 images before and after being processed by the brightness adaptive function of the present invention.

[0068] Fig.10 Comparison of the output results of different methods on four datasets. DETAILED DESCRIPTION

[0069] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

[0070] Example 1

[0071] like Figure 1As shown, the present invention provides a pavement crack detection method for different environments, comprising the following steps:

[0072] S1, inputting the pavement crack image into the brightness adaptive adjustment and feature processing module, performing preliminary feature extraction, and obtaining preliminary features of the pavement crack;

[0073] S2, inputting the preliminary features of the pavement cracks into the encoder network for step-by-step encoding to obtain the pavement crack encoding features;

[0074] S3. Input the pavement crack coding features into the decoder network for feature weighted fusion to obtain the final features of the pavement cracks, and realize pavement crack detection through the final features of the pavement cracks.

[0075] The brightness adaptive adjustment and feature processing module, the encoder network and the decoder network are connected in sequence to form a lightweight pavement crack detection model;

[0076] The loss function of the lightweight pavement crack detection model is expressed as:

[0077]

[0078] Among them, L(·) represents the cross entropy loss function, y represents the true label, p represents the predicted result, and y j represents the true label of the j-th pixel, p j Represents the prediction result of the jth pixel.

[0079] In this embodiment 1, crack detection is regarded as a binary classification task to distinguish crack pixels from non-crack pixels. The lightweight pavement crack detection model PCDNet is as follows: Figure 2 shown.

[0080] In this embodiment 1, BA-FP is a brightness adaptive adjustment and feature processing module, GD_Block is a Gabor difference block, PWF is a probability weighted fusion module, and MaxPooling is a maximum pooling layer.

[0081] PCDNet is divided into three parts. The first part is the brightness adaptive adjustment and feature processing module of the light-dark adaptation mechanism of cones and rods. This module realizes the preliminary extraction of input features by adaptively adjusting the brightness of the input image and feature processing, and also provides the basic conditions for the subsequent extraction of effective features.

[0082] The second part is an encoder network composed of Gabor difference blocks. After the brightness adjustment and processing of the first part, the input image is used as the input of the second part and processed step by step by the encoder network composed of Gabor difference blocks. The encoder network extracts effective features of the crack image, including shape and direction features, by stacking Gabor difference blocks. The Gabor difference block is inspired by the response of the biological visual system to visual stimuli of different directions and frequencies.

[0083] Then comes the third partial decoder network consisting of a probability weighted fusion module. The input of the decoder network is the four lateral outputs S1, S2, S3 and S4 of the encoder network, i.e., the encoded features. The encoded features are fused two by two through the processing of the probability weighted fusion module, which reduces the background texture while enhancing the output of effective features. Finally, a 1×1 convolution is performed to obtain the final crack output of the decoder network.

[0084] like Figure 3 As shown, the brightness adaptive adjustment and feature processing module includes a brightness adaptive unit, a cone cell processing unit and a rod cell processing unit; the brightness adaptive unit is connected to the cone cell processing unit and the rod cell processing unit respectively.

[0085] The brightness adaptation unit uses a brightness adaptation function to adjust the brightness of the input image;

[0086] The expression of the brightness adaptation function is:

[0087]

[0088] Among them, I a represents the output feature of the brightness adaptive unit, f(·) represents the brightness adaptive function, I represents the input image, ω represents the learnable parameter, V max represents the maximum brightness of the input image, and L represents the brightness of the input image;

[0089] The output characteristic I of the brightness adaptation unit a Includes R channel feature I aR , G channel feature I aG , B channel feature I aB and grayscale feature I ag ;

[0090] The expression of the brightness adaptive adjustment and feature processing module is:

[0091] I c =C 3×3 (C 3×3 (I aR ))+C 3×3 (C3×3 (I aG ))+C 3×3 (C 3×3 (I aB ))

[0092] I r =C 3×3,4 (C 3×3,2 (I ag ))

[0093] I o =I c +I r

[0094] Among them, I c represents the output feature of the cone processing unit, I r represents the output characteristics of the rod cell processing unit, I o Indicates the preliminary characteristics of pavement cracks, C 3×3 (·) represents a regular convolution with a kernel of 3, C 3×3,4 (·) represents the dilated convolution with a dilation rate of 4, C 3×3,2 (·) Dilated convolution with dilation rate 2.

[0095] In this embodiment 1, the brightness adaptive adjustment and feature processing module realizes the brightness adjustment and preliminary feature extraction of the crack image under different environments, including the brightness adaptive function proposed in this paper, the cone cell processing unit and the rod cell processing unit.

[0096] Among them, the brightness adaptation function realizes the adjustment of the brightness of the input image. The cone cell processing unit realizes the characteristics of three types of cone cells in the biological visual system processing three different colors of visual information through channel division and convolution operations. The rod cell processing unit realizes the characteristics of rod cells being sensitive to black and white vision by performing grayscale processing and convolution processing on the adjusted image.

[0097] Finally, by fusing the outputs of the two, the coordination between rods and cones was achieved, and the preliminary feature processing results were obtained. In addition, considering the different numbers of cones and rods in the retina, different receptive field sizes were set in the preliminary feature processing process.

[0098] The encoder network includes four Gabor difference blocks and three maximum pooling layers;

[0099] The four Gabor difference blocks include a first Gabor difference block, a second Gabor difference block, a third Gabor difference block and a fourth Gabor difference block; the three maximum pooling layers include a first maximum pooling layer, a second maximum pooling layer and a third maximum pooling layer;

[0100] The first Gabor difference block, the first maximum pooling layer, the second Gabor difference block, the second maximum pooling layer, the third Gabor difference block, the third maximum pooling layer and the fourth Gabor difference block are connected in sequence;

[0101] The first Gabor difference block outputs a first coding feature; the second Gabor difference block outputs a second coding feature; the third Gabor difference block outputs a third coding feature; and the fourth Gabor difference block outputs a fourth coding feature;

[0102] The first coding feature, the second coding feature, the third coding feature and the fourth coding feature are collectively used as pavement crack coding features.

[0103] Studies have shown that the biological visual system responds to visual stimuli of different directions and frequencies. This response is reflected in the fact that neurons in the visual cortex are sensitive to edges or motion directions of specific directions and respond to visual stimuli of specific frequencies. This response characteristic of neurons enables the biological visual system to better perceive and distinguish visual information of different directions and frequencies, thereby effectively processing and understanding visual signals in the environment. In addition, the Gabor filter has a selective response to edges or textures of different directions and frequencies, and can extract features of specific directions and frequencies in the image, thereby helping the model to better understand the structure and content of the image. Therefore, this embodiment 1 designs a Gabor difference block based on this.

[0104] like Figure 4 As shown, the expression of the Gabor difference block is:

[0105] C 1 =C 3×3 (GD_layer(I G ))

[0106] C 2 =C 3×3 (GD_layer(I G ))

[0107] W=Sigmoid(C 1×1 (C 1 ))

[0108] O G =C 2 +W×C 1

[0109] Among them, C 1 represents the first differential feature, C 2 represents the second differential feature, I Grepresents the input feature of Gabor difference block, GD_layer(·) represents Gabor difference layer, C 3×3 (·) represents a regular convolution with a kernel of 3, C 1×1 (·) represents a regular convolution with a convolution kernel of 1, W represents a weight parameter, Sigmoid (·) represents an activation function, and O G Represents the output features of the Gabor difference block.

[0110] like Figure 5 As shown, the expression of the Gabor difference layer is:

[0111] O max =max(Cat(Gcd*I GD_layer ,Gad*I GD_layer ))

[0112] O min =min(Cat(Gcd*I GD_layer ,Gad*I GD_layer ))

[0113] O GD_layer =C 1×1 (Cat(O max ,O min ))

[0114] Among them, O max represents the maximum difference feature, O min represents the minimum difference feature, max(·) represents the maximum value, min(·) represents the minimum value, Gcd represents the first convolution kernel composed of the center difference and the Gabor filter, Gad represents the second convolution kernel composed of the angle difference and the Gabor filter, * represents the convolution operation, I GD_layer represents the input feature of Gabor difference layer, Cat(·) represents the concatenation operation, O GD_layer Represents the output features of Gabor difference layer.

[0115] In this embodiment 1, in order to solve the problem of diverse crack shapes and directions in current crack detection, a new Gabor difference block is developed by combining Gabor filter, probability weighting and residual technology.

[0116] like Figure 4As shown in the figure, the input features first need to be processed by Gabor difference layering (i.e., GD_layer) to extract the effective feature information, including the shape and direction features of the cracks. Then, the features processed by Gabor difference layering are further processed by two parallel 3×3 convolutional layers, where the output of one convolutional layer is subjected to 1×1 convolution and sigmoid function to obtain a weight matrix for weighting the output of the pair of convolutional layers.

[0117] The weighted output is then fused with the output of another 3×3 convolutional layer to reduce the interference of background texture and supplement effective features, thus obtaining a more complete feature output. Finally, the final output of the Gabor difference block is obtained after another 3×3 convolutional layer.

[0118] The decoder network includes six probability weighted fusion modules;

[0119] The six probability weighted fusion modules include a first probability weighted fusion module, a second probability weighted fusion module, a third probability weighted fusion module, a fourth probability weighted fusion module, a fifth probability weighted fusion module and a sixth probability weighted fusion module;

[0120] The input features of the first probability weighted fusion module are the first coding feature and the second coding feature; the input features of the second probability weighted fusion module are the second coding feature and the third coding feature; the input features of the third probability weighted fusion module are the third coding feature and the fourth coding feature;

[0121] The input features of the fourth probability weighted fusion module are the output features of the first probability weighted fusion module and the output features of the second probability weighted fusion module; the input features of the fifth probability weighted fusion module are the output features of the second probability weighted fusion module and the output features of the third probability weighted fusion module; the input features of the sixth probability weighted fusion module are the output features of the fourth probability weighted fusion module and the output features of the fifth probability weighted fusion module;

[0122] The output features of the sixth probability weighted fusion module are input into a 1×1 convolution to obtain the final features of the pavement cracks.

[0123] like Figure 6 As shown, the expression of the probability weighted fusion module is:

[0124] O PWF =PW(I 1 )+Upsample(PW(I 2 ))

[0125] Among them, O PWFrepresents the output feature of the probability weighted fusion module, PW(·) represents the probability weight unit, and I 1 represents the first input feature of the probability weighted fusion module, I 2 represents the second input feature of the probability weighted fusion module, and Upsample(·) represents the upsampling operation.

[0126] The expression of the probability weight unit is:

[0127] C out =C 3×3,2 (C 3×3 (I PW ))

[0128] PW out =Sigmoid(C 1×1 (C out ))×C out

[0129] σ=mean(Sigmoid(C 1×1 (C out )))

[0130] PW fusion =PW out +σ×C out

[0131] Among them, C out represents the expansion characteristic, C 3×3,2 (·) Dilated convolution with a dilation rate of 2, C 3×3 (·) represents a regular convolution with a kernel of 3, I PW Represents the input features of the probability weight unit, PW out represents the weight matrix, C 1×1 (·) represents a regular convolution with a convolution kernel of 1, Sigmoid(·) represents the activation function, σ represents the mean of the weight matrix, mean(·) represents the average value, PW fusion Represents the output features of the probability weight unit.

[0132] In this embodiment 1, a probability weighted fusion module (PWF) is used to construct a lightweight decoder network, such as Figure 2 Shown in blue.

[0133] From the structure diagram of the probability weighted fusion module, it can be seen that the module is divided into two parallel paths, each path contains a probability weight unit (PW), and one of the paths contains an additional upsampling layer to restore the feature map with small resolution to the size of the adjacent feature map. In the probability weighted fusion module, the probability weight unit is used to further process the input features and calculate its weight matrix to weight the output.

[0134] The probability weight unit first uses a regular convolution layer with a convolution kernel of 3×3 and a dilated convolution layer with a convolution kernel of 3×3 and a dilation rate of 2 to process the features output by the encoder. Among them, the regular 3×3 convolution layer and the dilated convolution layer achieve a larger receptive field with fewer parameters, so that the PWF module can further extract the detailed information in the input features. Then, the weight matrix is ​​calculated using a 1×1 convolution layer and a Sigmoid activation function. By using the Sigmoid activation function, the size of each weight in the weight matrix is ​​limited to [0,1], and the weight can be dynamically adjusted according to the different inputs. Then, the weight matrix is ​​used to weight the output of the previous dilated convolution layer, and in order to prevent the loss of effective information during the weighted processing, the mean σ of the weight matrix is ​​calculated. Finally, the output of the dilated convolution layer is weighted again using σ, and the result of the two weighted fusions is used as the final output of the probability weighted fusion module.

[0135] Example 2

[0136] In this Example 2, the proposed PCDNet is experimentally analyzed and verified in detail. Four publicly available datasets are used in this experiment: BJN260, Rain365, Sun520, and DeepCrack. BJN260 contains data collected under night conditions, while Rain365 contains data taken after rain. On the other hand, Sun520 and DeepCrack contain data captured under normal weather conditions.

[0137] At the same time, depending on the data set, we use the full image as the model input for the BJN260, Rain365, and Sun520 data sets, and we use 380×380 size images as input for the DeepCrack data set. PCDNe uses the Adam optimizer to optimize the network parameters, and the initial learning rate is set to 3×10 -4 In addition, the network parameters are updated with batch_size=2 in each iteration for 50 iterations in total. All crack detection models are trained from scratch. In the experiments, the models are saved and tested every five training epochs to obtain the best performance.

[0138] In this embodiment 2, several indicators commonly used in the field of crack detection and segmentation are used to evaluate our method, including accuracy (ACC), precision (P), recall (R) and F1-score. In addition, commonly used evaluation methods for lightweight models are also used, including parameter quantity, FLOPs and FPS.

[0139] In this embodiment 2, the BJN260 nighttime dataset is used to perform ablation analysis on the brightness adjustment and feature processing module based on the brightness adaptation mechanism and the Gabor difference block.

[0140] As shown in Table 1, it is a comparison table of the results before and after combining brightness adaptive adjustment with feature processing modules or Gabor difference blocks in PCDNET, showing the performance of PCDNet with and without these modules on the BJN260 dataset.

[0141] Table 1

[0142]

[0143]

[0144] Table 1 clearly shows that combining Gabor difference blocks and brightness adaptive adjustment with feature processing modules can improve the detection performance of the model.

[0145] In this embodiment 2, Figure 7 As shown, some images in the BJN260 dataset are randomly selected and their output is visualized. Figure 7 In the example, Without-GD_layer means that the GD_block in PCDNet does not use GD_layer, and Without-BA-FP means that PCDNet does not use the BA-FP module.

[0146] calculate Figure 7 The F1 score of each image in the results shows that the output of PCDNet is usually better than the output of Without-GD_layer and Without-BA-FP. It is worth noting that after observing some outputs of Without-GD_layer, Without-BA-FP and PCDNet, it is noted that the outputs of Without-GD_layer and Without-BA-FP may contain more feature information.

[0147] However, after comparing with the true contour, it is found that the feature information in the Without-GD_layer and Without-BA-FP outputs is not annotated in the label, but is irrelevant background texture. Analysis of this image shows that the integration of the BA-FP module and the Gabor difference block enhances the output of the model, provides more detailed and relevant information, and effectively reduces the redundant background texture. This further proves the effectiveness of the brightness adaptive adjustment and feature processing module and the Gabor difference block proposed in the present invention.

[0148] like Figure 8As shown, the output after applying brightness adaptation is compared with the original image. It is obvious that the brightness-adjusted output shows significantly enhanced brightness, making the crack features more prominent and facilitating the extraction of these features in subsequent stages of the network. This visual confirmation further reinforces the effectiveness of our proposed features in improving crack detection performance.

[0149] To further verify the observed results, some images were randomly selected from two non-traditional environmental datasets, BJN260 and Rain365, and evaluated using the Brenner gradient function and SMD (grayscale difference) function, and the following results were obtained: Fig. 9 The clarity results are shown. Fig. 9 It can be seen that the output clarity after being processed by the brightness adaptive function is higher than that of the original image, which further shows that the result processed by the BA-FP module is more conducive to the extraction of crack features.

[0150] This Example 2 also re-implements and evaluates the performance of existing methods on the BJN260 and Rain365 datasets after applying enhanced data augmentation technology, where the existing methods include HED (holistic nested edge detection method), RCF (rich convolutional feature method), PiDiNet (pixel difference network for efficient edge detection), TEED (a tiny and efficient model for edge detection), UNet (a concrete crack detection method based on U-net full convolutional network), DeepCrack (a crack segmentation method based on deep hierarchical feature learning), SegNet (a deep convolutional encoder-decoder image segmentation method), Enet (a deep neural network for real-time semantic segmentation), XYW-Net (bionic XYW parallel path edge detection network), CrackW-Net (a new convolutional neural network for pavement crack image segmentation), CrackSegNet (concrete crack image detection method) and CarNet (a pavement crack detection method based on a lightweight convolutional neural network). Table 2 is a quantitative comparison table of PCDNet and other methods on the BJN260 and Rain365 datasets.

[0151] Table 2

[0152]

[0153] As shown in Table 2, the method of the present invention ranks in the top three in terms of F1 score, accuracy (ACC), number of parameters and FLOPs. Therefore, the method of the present invention achieves the best overall performance among all the methods.

[0154] Finally, this Example 2 conducted experiments on two conventional environment datasets, Sun520 and DeepCrack. Table 3 is a quantitative comparison table of PCDNet and other methods on the Sun520 dataset, and Table 4 is a quantitative comparison table of PCDNet and other methods on the DeepCrack dataset.

[0155] Table 3

[0156]

[0157] Table 4

[0158]

[0159] As shown in Tables 3 and 4, even on the datasets under normal conditions, the proposed method performs well and achieves the best F1 score. Compared with other methods with better performance, PCDNet has fewer parameters and FLOPs, which also shows that the proposed method not only performs consistently on different datasets, but also achieves excellent detection performance in various environments. Fig.10 The output results in further demonstrate that PCDNet is highly competitive among all methods and can extract crack features accurately and efficiently.

[0160] In summary, in response to the challenges of diverse environmental conditions, significant changes in crack shapes, and diverse directions faced by current deep learning-based crack detection methods, this paper proposes a bio-inspired lightweight pavement crack detection model, called PCDNet, for pixel-level road crack detection under different environmental conditions. In order to solve the problem of diverse external environments, PCDNet combines deep learning technology with the brightness adaptation mechanism in biological vision systems to achieve adaptive adjustment and feature processing of input images in different environments.

[0161] In view of the significant changes in crack shape and the challenges of diverse directions, PCDNet simulates the response of the biological visual system to visual stimuli of different directions and frequencies, and realizes the extraction of crack shape and direction features. Finally, PCDNet uses the probability weighted fusion module (PWF) to pair and fuse adjacent side output features, which promotes effective feature extraction and reduces the influence of background texture.

[0162] In addition, compared with previous methods with complex structures, large number of parameters and low efficiency, PCDNet shows higher efficiency, fewer parameters and lower computational overhead. Through testing and evaluation on multiple public datasets, PCDNet achieves superior performance among lightweight methods and shows strong competitiveness with other transfer learning methods.

Claims

1. A pavement crack detection method for different environments, It is characterized in that The following steps are involved: S1, inputting the pavement crack image into the brightness adaptive adjustment and feature processing module, performing preliminary feature extraction, and obtaining preliminary features of the pavement crack; S2, inputting the preliminary features of the pavement cracks into the encoder network for step-by-step encoding to obtain the pavement crack encoding features; S3. Input the pavement crack coding features into the decoder network for feature weighted fusion to obtain the final features of the pavement cracks, and realize pavement crack detection through the final features of the pavement cracks.

2. The pavement crack detection method for different environments according to claim 1, It is characterized in that The brightness adaptive adjustment and feature processing module, the encoder network and the decoder network are connected in sequence to form a lightweight pavement crack detection model; The loss function of the lightweight pavement crack detection model is expressed as: Among them, L(·) represents the cross entropy loss function, y represents the true label, p represents the predicted result, and y j represents the true label of the jth pixel, p j Represents the prediction result of the jth pixel.

3. The pavement crack detection method for different environments according to claim 1, It is characterized in that The brightness adaptive adjustment and feature processing module includes a brightness adaptive unit, a cone cell processing unit and a rod cell processing unit; the brightness adaptive unit is connected to the cone cell processing unit and the rod cell processing unit respectively.

4. The pavement crack detection method for different environments according to claim 3, It is characterized in that The brightness adaptation unit uses a brightness adaptation function to adjust the brightness of the input image; The expression of the brightness adaptation function is: Among them, I a represents the output feature of the brightness adaptive unit, f(·) represents the brightness adaptive function, I represents the input image, ω represents the learnable parameter, V max represents the maximum brightness of the input image, and L represents the brightness of the input image; The output characteristic of the brightness adaptive unit is a Includes R channel feature I aR , G channel feature I aG , B channel feature I aB and grayscale feature I ag ; The expression of the brightness adaptive adjustment and feature processing module is: I c =C 3×3 (C 3×3 (I aR ))+C 3×3 (C 3×3 (I aG ))+C 3×3 (C 3×3 (I aB )) I r =C 3×3,4 (C 3×3,2 (I ag )) I o =I c +I r Among them, I c represents the output feature of the cone processing unit, I r represents the output characteristics of the rod cell processing unit, I o Indicates the preliminary characteristics of pavement cracks, C 3×3 (·) represents a regular convolution with a kernel of 3, C 3×3,4 (·) represents the dilated convolution with a dilation rate of 4, C 3×3,2 (·) Dilated convolution with dilation rate 2.

5. The pavement crack detection method for different environments according to claim 1, It is characterized in that The encoder network includes four Gabor difference blocks and three maximum pooling layers; The four Gabor difference blocks include a first Gabor difference block, a second Gabor difference block, a third Gabor difference block and a fourth Gabor difference block; the three maximum pooling layers include a first maximum pooling layer, a second maximum pooling layer and a third maximum pooling layer; The first Gabor difference block, the first maximum pooling layer, the second Gabor difference block, the second maximum pooling layer, the third Gabor difference block, the third maximum pooling layer and the fourth Gabor difference block are connected in sequence; The first Gabor difference block outputs a first coding feature; the second Gabor difference block outputs a second coding feature; the third Gabor difference block outputs a third coding feature; and the fourth Gabor difference block outputs a fourth coding feature; The first coding feature, the second coding feature, the third coding feature and the fourth coding feature are collectively used as pavement crack coding features.

6. The pavement crack detection method for different environments according to claim 5, It is characterized in that The expression of the Gabor difference block is: C 1 =C 3×3 (GD_layer(I G )) C 2 =C 3×3 (GD_layer(I G )) W=Sigmoid(C 1×1 (C 1 )) O G =C 2 +W×C 1 Among them, C 1 represents the first differential feature, C 2 represents the second differential feature, I G represents the input feature of Gabor difference block, GD_layer(·) represents Gabor difference layer, C 3×3 (·) represents a regular convolution with a kernel of 3, C 1×1 (·) represents a regular convolution with a convolution kernel of 1, W represents a weight parameter, Sigmoid (·) represents an activation function, and O G Represents the output features of the Gabor difference block.

7. The pavement crack detection method for different environments according to claim 6, It is characterized in that The expression of the Gabor difference layer is: O max =max(Cat(Gcd*I GD_layer ,Gad*I GD_layer )) O min =min(Cat(Gcd*I GD_layer ,Gad*I GD_layer )) About GD_layer =C 1×1 (Cat(O max ,O min )) Among them, O max represents the maximum difference feature, O min represents the minimum difference feature, max(·) represents the maximum value, min(·) represents the minimum value, Gcd represents the first convolution kernel composed of the center difference and the Gabor filter, Gad represents the second convolution kernel composed of the angle difference and the Gabor filter, * represents the convolution operation, I GD_layer represents the input feature of Gabor difference layer, Cat(·) represents the concatenation operation, O GD_layer Represents the output features of Gabor difference layer.

8. The pavement crack detection method for different environments according to claim 5, It is characterized in that The decoder network includes six probability weighted fusion modules; The six probability weighted fusion modules include a first probability weighted fusion module, a second probability weighted fusion module, a third probability weighted fusion module, a fourth probability weighted fusion module, a fifth probability weighted fusion module and a sixth probability weighted fusion module; The input features of the first probability weighted fusion module are the first coding feature and the second coding feature; the input features of the second probability weighted fusion module are the second coding feature and the third coding feature; the input features of the third probability weighted fusion module are the third coding feature and the fourth coding feature; The input features of the fourth probability weighted fusion module are the output features of the first probability weighted fusion module and the output features of the second probability weighted fusion module; the input features of the fifth probability weighted fusion module are the output features of the second probability weighted fusion module and the output features of the third probability weighted fusion module; the input features of the sixth probability weighted fusion module are the output features of the fourth probability weighted fusion module and the output features of the fifth probability weighted fusion module; The output features of the sixth probability weighted fusion module are input into a 1×1 convolution to obtain the final features of the pavement cracks.

9. The pavement crack detection method for different environments according to claim 8, It is characterized in that The expression of the probability weighted fusion module is: The PWF =PW(I 1 )+Upsample(PW(I 2 )) Among them, O PWF represents the output feature of the probability weighted fusion module, PW(·) represents the probability weight unit, and I 1 represents the first input feature of the probability weighted fusion module, I 2 represents the second input feature of the probability weighted fusion module, and Upsample(·) represents the upsampling operation.

10. The pavement crack detection method for different environments according to claim 9, It is characterized in that The expression of the probability weight unit is: C out =C 3×3,2 (C 3×3 (I PW )) PW out =Sigmoid(C 1×1 (C out ))×C out σ=mean(Sigmoid(C 1×1 (C out ))) PW fusion =PW out +σ×C out Among them, C out represents the expansion characteristic, C 3×3,2 (·) Dilated convolution with a dilation rate of 2, C 3×3 (·) represents a regular convolution with a kernel of 3, I PW Represents the input features of the probability weight unit, PW out represents the weight matrix, C 1×1 (·) represents a regular convolution with a convolution kernel of 1, Sigmoid(·) represents the activation function, σ represents the mean of the weight matrix, mean(·) represents the average value, PW fusion Represents the output features of the probability weight unit.

Citation Information

Patent Citations

  • Wood defect detection method based on improved DETR model

    CN116363109A

  • Lightweight crack segmentation method based on convolutional neural network

    CN116993737A

  • Pavement crack detection method, medium, and system

    WO2024092968A1

Cited By

  • Multi-mode crack detection method, medium and equipment

    CN120219342A