A method for detecting road cracks in different environments

Abstract

The application discloses a kind of road surface crack detection methods for different environments, belong to image detection technical field, including the following steps: road surface crack image is input to luminance self-adapting adjustment and feature processing module, preliminary feature extraction is carried out, and road surface crack preliminary feature is obtained;Road surface crack preliminary feature is input to encoder network and is encoded gradually, and road surface crack coding feature is obtained;Road surface crack coding feature is input to decoder network and is weighted and fused, and road surface crack final feature is obtained, and road surface crack detection is realized by road surface crack final feature.The application realizes the road surface crack detection under different environments, solves the problem that there is existing model structure complex, parameter quantity is big and the problem of low efficiency for road surface crack detection.

Description

Technical Field

[0001] This invention belongs to the field of image detection technology, specifically relating to a method for detecting road surface cracks in different environments. Background Technology

[0002] Road crack detection aims to extract and identify cracks in the road surface to assess road conditions and aid in road maintenance and repair. Timely and accurate crack detection is crucial, preventing the spread of road damage, reducing maintenance costs, minimizing traffic accidents, and improving public safety. However, the accuracy of current crack detection methods is limited by the significant differences in shape, orientation, and continuity of road cracks, as well as their susceptibility to environmental factors (such as weather, lighting, and dirt) during data collection. This makes road crack detection one of the most challenging problems in visual tasks.

[0003] In recent years, deep learning technology has been widely applied to computer vision tasks due to its outstanding performance. Compared with earlier methods, methods based on convolutional neural networks and Transformers can achieve accurate crack extraction without manual feature engineering, while exhibiting strong robustness and high detection performance. However, most current methods only use crack data acquired under standard ambient lighting conditions for training and testing, neglecting the impact of external environmental factors such as weather and lighting changes on crack detection. This oversight limits the model's generalization ability and the range of applicable scenarios.

[0004] Furthermore, the diversity of road surface crack shapes and orientations makes it difficult for previous methods to capture comprehensive crack information, thus affecting the detection and generalization performance of the models. To achieve better detection performance, previous research has often focused on developing complex models, neglecting the practical application efficiency of these models. Therefore, many current deep learning methods face challenges such as high complexity, large parameter sets, low computational efficiency, and insufficient optimization for mobile platforms. Summary of the Invention

[0005] This invention provides a method for detecting pavement cracks in different environments, which realizes the detection of pavement cracks under different environments and solves the problems of complex model structure, large number of parameters and low efficiency in existing pavement crack detection methods.

[0006] To solve the above-mentioned technical problems, the technical solution of the present invention is: a method for detecting road surface cracks in different environments, comprising the following steps:

[0007] S1. Input the road surface crack image into the brightness adaptive adjustment and feature processing module to perform preliminary feature extraction and obtain preliminary features of the road surface crack;

[0008] S2. Input the preliminary features of road surface cracks into the encoder network for step-by-step encoding to obtain the coded features of road surface cracks;

[0009] S3. Input the road surface crack coding features into the decoder network for feature weighted fusion to obtain the final road surface crack features, and realize road surface crack detection through the final road surface crack features.

[0010] Furthermore, the brightness adaptive adjustment and feature processing module, the encoder network, and the decoder network are connected in sequence to form a lightweight pavement crack detection model;

[0011] The expression for the loss function of the lightweight pavement crack detection model is as follows:

[0012]

[0013] Where L(·) represents the cross-entropy loss function, y represents the true label, and p represents the prediction result. j p represents the true label of the j-th pixel. j This represents the prediction result for the j-th pixel.

[0014] Furthermore, the brightness adaptive adjustment and feature processing module includes a brightness adaptive unit, a cone cell processing unit, and a rod cell processing unit; the brightness adaptive unit is connected to the cone cell processing unit and the rod cell processing unit, respectively.

[0015] Furthermore, the brightness adaptive unit uses a brightness adaptive function to adjust the brightness of the input image;

[0016] The expression for the brightness adaptive function is:

[0017]

[0018] Among them, I a Let f(·) represent the output features of the brightness adaptive unit, f(·) represent the brightness adaptive function, I represent the input image, ω represent the learnable parameters, and V represent the output features of the brightness adaptive unit. max This represents the maximum brightness of the input image, where L represents the brightness of the input image.

[0019] The output feature I of the brightness adaptive unit a Including R channel features I aR G channel characteristics I aG B-channel characteristics I aB and grayscale features I ag ;

[0020] The expression for the brightness adaptive adjustment and feature processing module is:

[0021] I c =C 3×3 (C 3×3 (I aR ))+C 3×3 (C 3×3 (I aG ))+C 3×3 (C 3×3 (I aB ))

[0022] I r =C 3×3,4 (C 3×3,2 (I ag ))

[0023] I o =I c +I r

[0024] Among them, I c I represents the output characteristics of the cone cell processing unit. r I represents the output characteristics of the rod cell processing unit. o Indicates the initial characteristics of road surface cracks, C 3×3 (·) represents a regular convolution with a kernel of 3, C 3×3,4 (·) represents a dilated convolution with a dilation rate of 4, C 3×3,2 (·) Dilated convolution with an inflation rate of 2.

[0025] Furthermore, the encoder network includes four Gabor differential blocks and three max-pooling layers;

[0026] The four Gabor difference blocks include a first Gabor difference block, a second Gabor difference block, a third Gabor difference block, and a fourth Gabor difference block; the three max pooling layers include a first max pooling layer, a second max pooling layer, and a third max pooling layer.

[0027] The first Gabor differential block, the first max pooling layer, the second Gabor differential block, the second max pooling layer, the third Gabor differential block, the third max pooling layer, and the fourth Gabor differential block are connected in sequence;

[0028] The first Gabor differential block outputs a first coding feature; the second Gabor differential block outputs a second coding feature; the third Gabor differential block outputs a third coding feature; and the fourth Gabor differential block outputs a fourth coding feature.

[0029] The first coding feature, the second coding feature, the third coding feature and the fourth coding feature together serve as the road surface crack coding features.

[0030] Furthermore, the expression for the Gabor differential block is:

[0031] C1 = C 3×3 (GD_layer(I G ))

[0032] C2 = C 3×3 (GD_layer(I G ))

[0033] W = Sigmoid(C 1×1 (C1))

[0034] O G =C2+W×C1

[0035] Where C1 represents the first difference feature, C2 represents the second difference feature, and I G GD_layer(·) represents the input features of the Gabor difference block, and C represents the Gabor difference layer. 3×3 (·) represents a regular convolution with a kernel of 3, C 1×1 (·) represents a regular convolution with a kernel of 1, W represents the weight parameters, Sigmoid(·) represents the activation function, and O G This represents the output characteristics of the Gabor differential block.

[0036] Furthermore, the expression for the Gabor difference stratification is:

[0037] O max =max(Cat(Gcd*I) GD_layer ,Gad*I GD_layer ))

[0038] O min =min(Cat(Gcd*I) GD_layer ,Gad*I GD_layer ))

[0039] O GD_layer =C 1×1 (Cat(O max O min ))

[0040] Among them, O max O represents the maximum difference feature. min The minimum difference feature is represented by `max(·)`, which represents the maximum value, and `min(·)` represents the minimum value. `Gcd` represents the first convolution kernel formed by combining the center difference and the Gabor filter, `Gad` represents the second convolution kernel formed by combining the angle difference and the Gabor filter, and `*` represents the convolution operation. GD_layerThis represents the input features for Gabor difference hierarchies, Cat(·) represents the concatenation operation, and O GD_layer This represents the output characteristics of Gabor difference stratification.

[0041] Furthermore, the decoder network includes six probability-weighted fusion modules;

[0042] The six probability weighted fusion modules include a first probability weighted fusion module, a second probability weighted fusion module, a third probability weighted fusion module, a fourth probability weighted fusion module, a fifth probability weighted fusion module, and a sixth probability weighted fusion module;

[0043] The input features of the first probability weighted fusion module are the first encoded feature and the second encoded feature; the input features of the second probability weighted fusion module are the second encoded feature and the third encoded feature; the input features of the third probability weighted fusion module are the third encoded feature and the fourth encoded feature.

[0044] The input features of the fourth probability weighted fusion module are the output features of the first probability weighted fusion module and the second probability weighted fusion module; the input features of the fifth probability weighted fusion module are the output features of the second probability weighted fusion module and the third probability weighted fusion module; the input features of the sixth probability weighted fusion module are the output features of the fourth probability weighted fusion module and the fifth probability weighted fusion module.

[0045] The output features of the sixth probability weighted fusion module are input into a 1×1 convolution to obtain the final features of road surface cracks.

[0046] Furthermore, the expression for the probability-weighted fusion module is:

[0047] O PWF =PW(I1)+Upsample(PW(I2))

[0048] Among them, O PWF PW(·) represents the output feature of the probability weighted fusion module, I1 represents the first input feature of the probability weighted fusion module, I2 represents the second input feature of the probability weighted fusion module, and Upsample(·) represents the upsampling operation.

[0049] Furthermore, the expression for the probability weight unit is:

[0050] C out =C 3×3,2 (C 3×3 (I PW ))

[0051] PW out=Sigmoid(C 1×1 (C out ))×C out

[0052] σ = mean(Sigmoid(C) 1×1 (C out )))

[0053] PW fusion =PW out +σ×C out

[0054] Among them, C out Indicating dilation characteristics, C 3×3,2 (·) Dilated convolution with a dilation rate of 2, C 3×3 (·) represents a regular convolution with a kernel of 3, I PW PW represents the input features of the probability weight unit. out Let C represent the weight matrix. 1×1 (·) represents a regular convolution with a kernel of 1, Sigmoid(·) represents the activation function, σ represents the mean of the weight matrix, mean(·) represents taking the average, and PW fusion This represents the output feature of the probability weight unit.

[0055] The beneficial effects of the present invention are: (1) A lightweight road surface crack detection model is proposed, which combines deep learning technology with the brightness adaptation mechanism in the biological vision system, realizes adaptive adjustment and feature processing of input images in different environments, and has higher efficiency, fewer parameters and lower computational overhead.

[0056] (2) The brightness adaptive feature processing module simulates the brightness adaptation mechanism of the biological visual system to adjust the brightness and process the features of the input image.

[0057] (3) In response to the significant changes in crack shape and the challenge of diverse directions, Gabor differential blocks were proposed to simulate the response of the biological visual system to visual stimuli of different directions and frequencies. This can extract the shape and direction of cracks while suppressing background texture interference, thus realizing the extraction of crack shape and direction features.

[0058] (4) The probability weighted fusion module is used to pair and fuse adjacent side output features, which promotes effective feature extraction and reduces the influence of background texture. Attached Figure Description

[0059] Figure 1 This is a flowchart of the pavement crack detection method of the present invention for different environments.

[0060] Figure 2This is a structural diagram of the lightweight pavement crack detection model of the present invention.

[0061] Figure 3 This is a structural diagram of the brightness adaptive adjustment and feature processing module of the present invention.

[0062] Figure 4 This is a structural diagram of the Gabor differential block of the present invention.

[0063] Figure 5 This is a structural diagram of the Gabor differential layering of the present invention.

[0064] Figure 6 This is a structural diagram of the probability weighted fusion module of the present invention.

[0065] Figure 7 This is a schematic diagram of the ablation experiment results of the present invention on the BJN260 dataset.

[0066] Figure 8 This is a schematic diagram showing the results before and after the brightness adaptive function processing of some BJN260 and Rain365 images according to the present invention.

[0067] Figure 9 This is a comparison of image sharpness results before and after the luminance adaptive function processing of some BJN260 and Rain365 images according to the present invention.

[0068] Figure 10 This is a comparison chart of the output results of different methods on four datasets. Detailed Implementation

[0069] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

[0070] Example 1

[0071] like Figure 1 As shown, the present invention provides a method for detecting pavement cracks in different environments, comprising the following steps:

[0072] S1. Input the road surface crack image into the brightness adaptive adjustment and feature processing module to perform preliminary feature extraction and obtain preliminary features of the road surface crack;

[0073] S2. Input the preliminary features of road surface cracks into the encoder network for step-by-step encoding to obtain the coded features of road surface cracks;

[0074] S3. Input the road surface crack coding features into the decoder network for feature weighted fusion to obtain the final road surface crack features, and realize road surface crack detection through the final road surface crack features.

[0075] The brightness adaptive adjustment and feature processing module, encoder network and decoder network are connected in sequence to form a lightweight pavement crack detection model.

[0076] The expression for the loss function of the lightweight pavement crack detection model is as follows:

[0077]

[0078] Where L(·) represents the cross-entropy loss function, y represents the true label, and p represents the prediction result. j p represents the true label of the j-th pixel. j This represents the prediction result for the j-th pixel.

[0079] In this embodiment 1, crack detection is treated as a binary classification task that distinguishes between cracked pixels and non-cracked pixels. The lightweight pavement crack detection model PCDNet is as follows: Figure 2 As shown.

[0080] Furthermore, in this embodiment 1, BA-FP is the brightness adaptive adjustment and feature processing module, GD_Block is the Gabor differential block, PWF is the probability weighted fusion module, and MaxPooling is the maximum pooling layer.

[0081] PCDNet is divided into three parts. The first part is the brightness adaptive adjustment and feature processing module for the brightness adaptation mechanism of cone and rod cells. This module realizes the initial extraction of input features by adaptively adjusting the brightness of the input image and performing feature processing, and also provides the basic conditions for the subsequent extraction of effective features.

[0082] The second part is an encoder network composed of Gabor differential blocks. After brightness adjustment and processing in the first part, the input image is used as input to the encoder network composed of Gabor differential blocks for progressive processing in the second part. The encoder network extracts effective features from the crack image, including shape and orientation features, by stacking Gabor differential blocks. The Gabor differential blocks are inspired by the biological visual system's response to visual stimuli of different directions and frequencies.

[0083] The third part, the decoder network, consists of a probability-weighted fusion module. The decoder network takes as input the four lateral outputs S1, S2, S3, and S4 from the encoder network—the encoded features. These encoded features are then fused pairwise through the probability-weighted fusion module, reducing background texture while enhancing the output of effective features. Finally, a 1×1 convolution process yields the final split output of the decoder network.

[0084] like Figure 3 As shown, the brightness adaptive adjustment and feature processing module includes a brightness adaptive unit, a cone cell processing unit, and a rod cell processing unit; the brightness adaptive unit is connected to the cone cell processing unit and the rod cell processing unit, respectively.

[0085] The brightness adaptive unit uses a brightness adaptive function to adjust the brightness of the input image;

[0086] The expression for the brightness adaptive function is:

[0087]

[0088] Among them, I a Let f(·) represent the output features of the brightness adaptive unit, f(·) represent the brightness adaptive function, I represent the input image, ω represent the learnable parameters, and V represent the output features of the brightness adaptive unit. max This represents the maximum brightness of the input image, where L represents the brightness of the input image.

[0089] The output feature I of the brightness adaptive unit a Including R channel features I aR G channel characteristics I aG B-channel characteristics I aB and grayscale features I ag ;

[0090] The expression for the brightness adaptive adjustment and feature processing module is:

[0091] I c =C 3×3 (C 3×3 (I aR ))+C 3×3 (C 3×3 (I aG ))+C 3×3 (C 3×3 (I aB ))

[0092] I r =C 3×3,4 (C 3×3,2 (I ag ))

[0093] I o =I c +I r

[0094] Among them, I c I represents the output characteristics of the cone cell processing unit. r I represents the output characteristics of the rod cell processing unit. o Indicates the initial characteristics of road surface cracks, C 3×3 (·) represents a regular convolution with a kernel of 3, C 3×3,4 (·) represents a dilated convolution with a dilation rate of 4, C 3×3,2 (·) Dilated convolution with an inflation rate of 2.

[0095] In this embodiment 1, the brightness adaptive adjustment and feature processing module realizes the brightness adjustment and preliminary feature extraction of crack images under different environments, including three parts: the brightness adaptive function proposed in this paper, the cone cell processing unit, and the rod cell processing unit.

[0096] The brightness adaptive function adjusts the brightness of the input image. The cone cell processing unit implements the characteristic of three types of cone cells processing three different colors of visual information in a biological vision system through channel division and convolution operations. The rod cell processing unit implements the characteristic of rod cells being sensitive to black and white vision by performing grayscale processing and convolution on the adjusted image.

[0097] Finally, by fusing the outputs of both, the cooperation between rod and cone cells was achieved, yielding preliminary feature processing results. Furthermore, considering the different numbers of cone and rod cells within the retina, different receptive field sizes were set during the preliminary feature processing.

[0098] The encoder network comprises four Gabor differential blocks and three max-pooling layers;

[0099] The four Gabor difference blocks include a first Gabor difference block, a second Gabor difference block, a third Gabor difference block, and a fourth Gabor difference block; the three max pooling layers include a first max pooling layer, a second max pooling layer, and a third max pooling layer.

[0100] The first Gabor differential block, the first max pooling layer, the second Gabor differential block, the second max pooling layer, the third Gabor differential block, the third max pooling layer, and the fourth Gabor differential block are connected in sequence;

[0101] The first Gabor differential block outputs a first coding feature; the second Gabor differential block outputs a second coding feature; the third Gabor differential block outputs a third coding feature; and the fourth Gabor differential block outputs a fourth coding feature.

[0102] The first coding feature, the second coding feature, the third coding feature and the fourth coding feature together serve as the road surface crack coding features.

[0103] Studies have shown that the biological visual system responds to visual stimuli of different directions and frequencies. This response is manifested in the sensitivity of neurons in the visual cortex to edges or motion directions in specific directions and their response to visual stimuli of specific frequencies. This responsive characteristic of neurons enables the biological visual system to better perceive and distinguish visual information of different directions and frequencies, thereby achieving effective processing and understanding of visual signals in the environment. In addition, Gabor filters have selective responses to edges or textures of different directions and frequencies, and can extract features of specific directions and frequencies from images, thereby helping the model to better understand the structure and content of images. Therefore, this embodiment 1 designs a Gabor differential block based on this.

[0104] like Figure 4 As shown, the expression for the Gabor differential block is:

[0105] C1 = C 3×3 (GD_layer(I G ))

[0106] C2 = C 3×3 (GD_layer(I G ))

[0107] W = Sigmoid(C 1×1 (C1))

[0108] O G =C2+W×C1

[0109] Where C1 represents the first difference feature, C2 represents the second difference feature, and I G GD_layer(·) represents the input features of the Gabor difference block, and C represents the Gabor difference layer. 3×3 (·) represents a regular convolution with a kernel of 3, C 1×1 (·) represents a regular convolution with a kernel of 1, W represents the weight parameters, Sigmoid(·) represents the activation function, and O G This represents the output characteristics of the Gabor differential block.

[0110] like Figure 5 As shown, the expression for the Gabor differential stratification is:

[0111] O max =max(Cat(Gcd*I) GD_layer ,Gad*I GD_layer ))

[0112] O min =min(Cat(Gcd*I) GD_layer ,Gad*I GD_layer ))

[0113] O GD_layer =C 1×1 (Cat(O max O min ))

[0114] Among them, O max O represents the maximum difference feature. min The minimum difference feature is represented by `max(·)`, which represents the maximum value, and `min(·)` represents the minimum value. `Gcd` represents the first convolution kernel formed by combining the center difference and the Gabor filter, `Gad` represents the second convolution kernel formed by combining the angle difference and the Gabor filter, and `*` represents the convolution operation. GD_layer This represents the input features for Gabor difference hierarchies, Cat(·) represents the concatenation operation, and O GD_layer This represents the output characteristics of Gabor difference stratification.

[0115] In this embodiment 1, in order to address the problem of diverse crack shapes and directions in current crack detection, a new Gabor differential block is developed by combining Gabor filters, probability weighting, and residual techniques.

[0116] like Figure 4 As shown, the input features first need to be processed by Gabor difference layer (i.e., GD_layer) to extract effective feature information, including the shape and orientation features of the cracks. Then, the features processed by Gabor difference layer are further processed by two parallel 3×3 convolutional layers. The output of one convolutional layer is then processed by a 1×1 convolution and a sigmoid function to obtain a weight matrix used to weight the outputs of the two convolutional layers.

[0117] The weighted output is then fused with the output of another 3×3 convolutional layer to reduce background texture interference and supplement effective features, resulting in a more complete feature output. Finally, it is processed by another 3×3 convolutional layer to obtain the final output of the Gabor difference block.

[0118] The decoder network includes six probability-weighted fusion modules;

[0119] The six probability weighted fusion modules include a first probability weighted fusion module, a second probability weighted fusion module, a third probability weighted fusion module, a fourth probability weighted fusion module, a fifth probability weighted fusion module, and a sixth probability weighted fusion module;

[0120] The input features of the first probability weighted fusion module are the first encoded feature and the second encoded feature; the input features of the second probability weighted fusion module are the second encoded feature and the third encoded feature; the input features of the third probability weighted fusion module are the third encoded feature and the fourth encoded feature.

[0121] The input features of the fourth probability weighted fusion module are the output features of the first probability weighted fusion module and the second probability weighted fusion module; the input features of the fifth probability weighted fusion module are the output features of the second probability weighted fusion module and the third probability weighted fusion module; the input features of the sixth probability weighted fusion module are the output features of the fourth probability weighted fusion module and the fifth probability weighted fusion module.

[0122] The output features of the sixth probability weighted fusion module are input into a 1×1 convolution to obtain the final features of road surface cracks.

[0123] like Figure 6 As shown, the expression for the probability-weighted fusion module is:

[0124] O PWF =PW(I1)+Upsample(PW(I2))

[0125] Among them, O PWF PW(·) represents the output feature of the probability weighted fusion module, I1 represents the first input feature of the probability weighted fusion module, I2 represents the second input feature of the probability weighted fusion module, and Upsample(·) represents the upsampling operation.

[0126] The expression for the probability weight unit is:

[0127] C out =C 3×3,2 (C 3×3 (I PW ))

[0128] PW out =Sigmoid(C 1×1 (C out ))×C out

[0129] σ = mean(Sigmoid(C) 1×1 (C out )))

[0130] PW fusion =PW out +σ×C out

[0131] Among them, C out Indicating dilation characteristics, C 3×3,2 (·) Dilated convolution with a dilation rate of 2, C 3×3 (·) represents a regular convolution with a kernel of 3, I PW PW represents the input features of the probability weight unit. out Let C represent the weight matrix. 1×1 (·) represents a regular convolution with a kernel of 1, Sigmoid(·) represents the activation function, σ represents the mean of the weight matrix, mean(·) represents taking the average, and PW fusion This represents the output feature of the probability weight unit.

[0132] In this embodiment 1, a lightweight decoder network is constructed using a probabilistic weighted fusion module (PWF), such as... Figure 2 The blue section is shown.

[0133] As can be seen from the structure diagram of the probabilistic weighted fusion module, this module is divided into two parallel paths, each containing a probabilistic weight unit (PW). One path additionally includes an upsampling layer to restore the smaller-resolution feature map to the size of its neighboring feature maps. In the probabilistic weighted fusion module, the probabilistic weight unit is used to further process the input features and calculate its weight matrix to weight the output.

[0134] The probabilistic weighting unit first processes the encoder output features using a regular convolutional layer with a 3×3 kernel and a dilated convolutional layer with a 3×3 kernel and a dilation rate of 2. The 3×3 regular convolutional layer and the dilated convolutional layer achieve a large receptive field with fewer parameters, enabling the PWF module to further extract detailed information from the input features. Next, a weight matrix is ​​calculated using a 1×1 convolutional layer and a sigmoid activation function. The sigmoid activation function limits the size of each weight in the weight matrix to [0,1], and allows for dynamic adjustment of weights based on different inputs. Then, the weight matrix is ​​used to weight the output of the previous dilated convolutional layer, and the mean σ of the weight matrix is ​​calculated to prevent loss of effective information during weighting. Finally, σ is used to weight the output of the dilated convolutional layer again, and the result of the two weighted fusions is used as the final output of the probabilistic weighted fusion module.

[0135] Example 2

[0136] In this second embodiment, a detailed experimental analysis and verification of the proposed PCDNet were conducted. This experiment used four publicly available datasets: BJN260, Rain365, Sun520, and DeepCrack. BJN260 contains data collected under nighttime conditions, while Rain365 contains data captured after rain. Sun520 and DeepCrack, on the other hand, contain data captured under normal weather conditions.

[0137] Furthermore, depending on the dataset, we used full images as model input for the BJN260, Rain365, and Sun520 datasets, while for the DeepCrack dataset, we used 380×380 images as input. PCDNe uses the Adam optimizer to optimize the network parameters, with an initial learning rate set to 3×10⁻⁶. -4 Furthermore, the network parameters were updated with a batch size of 2 in each iteration, for a total of 50 iterations. All crack detection models were trained from scratch, and in the experiments, the models were saved and tested every five training epochs to obtain optimal performance.

[0138] In this second embodiment, several metrics commonly used in crack detection and segmentation are employed to evaluate our method, including accuracy (ACC), precision (P), recall (R), and F1-score. In addition, evaluation methods commonly used in lightweight models are also used, including parameter count, FLOPs, and FPS.

[0139] In this embodiment 2, the BJN260 nighttime dataset was used to perform ablation analysis on the brightness adjustment and feature processing module based on the brightness adaptation mechanism and the Gabor differential block.

[0140] Table 1 shows a comparison of the results before and after combining the brightness adaptive adjustment and feature processing modules or Gabor difference blocks in PCDNet, illustrating the performance of PCDNet on the BJN260 dataset with and without these modules.

[0141] Table 1

[0142]

[0143]

[0144] Table 1 clearly shows that combining Gabor differential blocks with brightness adaptive adjustment and feature processing modules can improve the detection performance of the model.

[0145] In this embodiment 2, as Figure 7 As shown, a portion of images from the BJN260 dataset were randomly selected, and their output was visualized. Figure 7 In the code, Without-GD_layer means that GD_block in PCDNet does not use GD_layer, and Without-BA-FP means that PCDNet does not use the BA-FP module.

[0146] calculate Figure 7 The F1 score for each image was calculated, and the results show that PCDNet's output generally outperforms the outputs of Without-GD_layer and Without-BA-FP. Notably, after observing some of the outputs of Without-GD_layer, Without-BA-FP, and PCDNet, it was noted that the outputs of Without-GD_layer and Without-BA-FP may contain more feature information.

[0147] However, comparison with the true contour revealed that the feature information in the outputs of Without-GD_layer and Without-BA-FP was not annotated in the labels, but rather represented as irrelevant background texture. Analysis of this image shows that the integration of the BA-FP module and Gabor difference blocks enhances the model's output, providing more detailed and relevant information and effectively reducing redundant background texture. This further demonstrates the effectiveness of the proposed brightness adaptive adjustment and feature processing module, as well as the Gabor difference blocks.

[0148] like Figure 8 As shown, the output after applying brightness adaptive processing is compared with the original image. It is evident that the brightness-adjusted output exhibits significantly enhanced brightness, making crack features more prominent and facilitating their extraction in subsequent network stages. This visual confirmation further reinforces the effectiveness of our proposed feature in improving crack detection performance.

[0149] To further validate the observed results, some images were randomly selected from two non-traditional environmental datasets, BJN260 and Rain365, and evaluated using the Brenner gradient function and the SMD (grayscale difference) function, yielding the following results: Figure 9 The sharpness results are shown. From Figure 9 It can be seen that the output clarity after brightness adaptive processing is higher than that of the original image, further indicating that the result processed by the BA-FP module is more conducive to the extraction of crack features.

[0150] This embodiment 2 also reimplemented and evaluated the performance of existing methods on the BJN260 and Rain365 datasets after applying enhanced data augmentation techniques. The existing methods include HED (globally nested edge detection), RCF (rich convolutional feature method), PiDiNet (pixel difference network for efficient edge detection), TEED (small and efficient model for edge detection), UNet (concrete crack detection method based on U-net fully convolutional network), DeepCrack (crack segmentation method based on deep hierarchical feature learning), SegNet (deep convolutional encoder-decoder image segmentation method), Enet (a deep neural network for real-time semantic segmentation), XYW-Net (bionic XYW parallel path edge detection network), CrackW-Net (a novel convolutional neural network for pavement crack image segmentation), CrackSegNet (concrete crack image detection method), and CarNet (pavement crack detection method based on lightweight convolutional neural network). Table 2 is a quantitative comparison table of PCDNet and other methods on the BJN260 and Rain365 datasets.

[0151] Table 2

[0152]

[0153] As shown in Table 2, the method of this invention ranks among the top three in terms of F1 score, accuracy (ACC), number of parameters, and FLOPs. Therefore, the method of this invention achieves the best overall performance among all methods.

[0154] Finally, experiments were conducted on two conventional environment datasets, Sun520 and DeepCrack, in Example 2. Table 3 shows the quantitative comparison of PCDNet with other methods on the Sun520 dataset, and Table 4 shows the quantitative comparison of PCDNet with other methods on the DeepCrack dataset.

[0155] Table 3

[0156]

[0157] Table 4

[0158]

[0159] As shown in Tables 3 and 4, even on datasets under normal conditions, the method of this invention performs excellently, achieving the best F1 score. Compared to other better-performing methods, PCDNet has fewer parameters and fewer FLOPs, demonstrating that the method of this invention not only performs consistently on different datasets but also achieves excellent detection performance in various environments. Figure 10The output results further demonstrate that PCDNet is highly competitive among all methods and can accurately and efficiently extract crack features.

[0160] In summary, this invention addresses the challenges posed by current deep learning-based crack detection methods, including diverse environmental conditions, significant variations in crack shape, and diverse orientations. It proposes a bio-inspired, lightweight pavement crack detection model, PCDNet, for pixel-level road crack detection under various environmental conditions. To solve the problem of diverse external environments, PCDNet combines deep learning techniques with brightness adaptation mechanisms from biological vision systems, achieving adaptive adjustment and feature processing of input images in different environments.

[0161] To address the significant variations in crack shape and the challenge of diverse orientations, PCDNet simulates the response of biological visual systems to visual stimuli of different directions and frequencies, enabling the extraction of crack shape and orientation features. Finally, PCDNet utilizes a probabilistic weighted fusion (PWF) module to pair and fuse adjacent side output features, promoting efficient feature extraction and reducing the influence of background texture.

[0162] Furthermore, compared to previous methods with complex structures, large numbers of parameters, and low efficiency, PCDNet exhibits higher efficiency, fewer parameters, and lower computational overhead. Through testing and evaluation on multiple public datasets, PCDNet achieves superior performance among lightweight methods and demonstrates strong competitiveness with other transfer learning approaches.

Claims

1. A method for detecting pavement cracks in different environments, characterized in that, Includes the following steps: S1. Input the road surface crack image into the brightness adaptive adjustment and feature processing module to perform preliminary feature extraction and obtain preliminary features of the road surface crack; S2. Input the preliminary features of road surface cracks into the encoder network for step-by-step encoding to obtain the coded features of road surface cracks; S3. Input the road surface crack coding features into the decoder network for feature weighting and fusion to obtain the final road surface crack features, and use the final road surface crack features to realize road surface crack detection. The brightness adaptive adjustment and feature processing module, encoder network and decoder network are connected in sequence to form a lightweight pavement crack detection model. The expression for the loss function of the lightweight pavement crack detection model is as follows: in, Represents the cross-entropy loss function. Indicates the true label, Indicates the prediction result. Indicates the first The real label of each pixel Indicates the first Prediction results for each pixel; The brightness adaptive adjustment and feature processing module includes a brightness adaptive unit, a cone cell processing unit, and a rod cell processing unit; the brightness adaptive unit is connected to the cone cell processing unit and the rod cell processing unit, respectively. The brightness adaptive unit uses a brightness adaptive function to adjust the brightness of the input image; The expression for the brightness adaptive function is: in, This represents the output characteristics of the brightness adaptive unit. This represents the brightness adaptive function. Indicates the input image. Indicates learnable parameters, This represents the maximum brightness of the input image. Indicates the brightness of the input image; The output characteristics of the brightness adaptive unit include Channel characteristics , Channel characteristics , Channel characteristics and grayscale features ; The expression for the brightness adaptive adjustment and feature processing module is: in, This indicates the output characteristics of the cone cell processing unit. This indicates the output characteristics of the rod cell processing unit. This indicates the initial characteristics of road surface cracks. This represents a regular convolution with a kernel of 3. This represents a dilated convolution with an inflation rate of 4. Dilated convolution with an inflation rate of 2.

2. The method for detecting pavement cracks in different environments according to claim 1, characterized in that, The encoder network comprises four Gabor differential blocks and three max-pooling layers; The four Gabor difference blocks include a first Gabor difference block, a second Gabor difference block, a third Gabor difference block, and a fourth Gabor difference block; the three max pooling layers include a first max pooling layer, a second max pooling layer, and a third max pooling layer. The first Gabor differential block, the first max pooling layer, the second Gabor differential block, the second max pooling layer, the third Gabor differential block, the third max pooling layer, and the fourth Gabor differential block are connected in sequence; The first Gabor differential block outputs a first coding feature; the second Gabor differential block outputs a second coding feature; the third Gabor differential block outputs a third coding feature; and the fourth Gabor differential block outputs a fourth coding feature. The first coding feature, the second coding feature, the third coding feature and the fourth coding feature together serve as the road surface crack coding features.

3. The method for detecting pavement cracks in different environments according to claim 2, characterized in that, The expression for the Gabor differential block is: in, This represents the first difference feature. This represents the second difference feature. This represents the input features of a Gabor difference block. Indicates Gabor differential stratification. This represents a regular convolution with a kernel of 3. This represents a regular convolution with a kernel of 1. Represents the weight parameters. This represents the activation function. This represents the output characteristics of the Gabor differential block.

4. The method for detecting pavement cracks in different environments according to claim 3, characterized in that, The expression for the Gabor differential stratification is: in, Represents the maximum difference feature. Represents the minimum difference feature. This indicates taking the maximum value. This indicates taking the minimum value. This represents the first convolution kernel formed by combining the central difference filter and the Gabor filter. This represents the second convolution kernel, which is a combination of angular difference and Gabor filter. This represents the convolution operation. The input features representing Gabor difference hierarchies, This indicates a splicing operation. This represents the output characteristics of Gabor difference stratification.

5. The method for detecting pavement cracks in different environments according to claim 2, characterized in that, The decoder network includes six probability-weighted fusion modules; The six probability weighted fusion modules include a first probability weighted fusion module, a second probability weighted fusion module, a third probability weighted fusion module, a fourth probability weighted fusion module, a fifth probability weighted fusion module, and a sixth probability weighted fusion module; The input features of the first probability weighted fusion module are the first encoded feature and the second encoded feature; the input features of the second probability weighted fusion module are the second encoded feature and the third encoded feature; the input features of the third probability weighted fusion module are the third encoded feature and the fourth encoded feature. The input features of the fourth probability weighted fusion module are the output features of the first probability weighted fusion module and the second probability weighted fusion module; the input features of the fifth probability weighted fusion module are the output features of the second probability weighted fusion module and the third probability weighted fusion module; the input features of the sixth probability weighted fusion module are the output features of the fourth probability weighted fusion module and the fifth probability weighted fusion module. The output features of the sixth probability weighted fusion module are input into a 1×1 convolution to obtain the final features of road surface cracks.

6. The method for detecting pavement cracks in different environments according to claim 5, characterized in that, The expression for the probability-weighted fusion module is: in, This represents the output feature of the probability-weighted fusion module. Represents the probability weight unit. This represents the first input feature of the probability-weighted fusion module. This represents the second input feature of the probability-weighted fusion module. This indicates an upsampling operation.

7. The method for detecting pavement cracks in different environments according to claim 6, characterized in that, The expression for the probability weight unit is: in, Indicates the expansion characteristic, Dilated convolution with an inflation rate of 2 This represents a regular convolution with a kernel of 3. This represents the input features of the probability weight unit. Represents the weight matrix. This represents a regular convolution with a kernel of 1. This represents the activation function. Represents the mean of the weight matrix. This indicates taking the average value. This represents the output feature of the probability weight unit.

Images