Pavement crack segmentation and quantification method based on pyramid region attention

Abstract

The invention discloses a pavement crack segmentation and quantification method based on pyramid region attention, and relates to the technical field of computer vision. The road maintenance mileage is increased year by year, the capital and human input required for maintaining the road network is quite huge, and the efficient, accurate and robust pavement crack segmentation and quantification method has important significance in monitoring the pavement damage condition and reducing the road maintenance expenditure. According to the method, the long-distance dependency relationship can be effectively captured while the low calculation cost is kept, the segmentation precision of the pavement crack is improved, the crack damage rate is calculated according to the segmentation result, a basis is provided for evaluation of the road damage condition, the pavement maintenance decision-making efficiency is improved, and the maintenance expenditure is reduced.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a road surface crack recognition, detection and quantification method based on a deep learning algorithm. Background Art

[0002] Highways are an important part of a country's infrastructure, and the road surface will inevitably be damaged during use due to various internal and external factors, such as repeated traffic loads and severe weather conditions. Among them, cracks are one of the most common pavement diseases. If they are not discovered and repaired in time, they will further expand and deepen and cause more serious structural damage. Therefore, timely detection of early pavement cracks is a key measure to effectively prevent further deterioration of pavement performance and reduce the maintenance costs of highway management departments. Traditional pavement crack detection mainly relies on manual visual inspection, and the inspectors visually assess the pavement condition. Although this method is the simplest to operate, manual visual inspection is time-consuming and labor-intensive, and has the disadvantages of high labor costs, low detection efficiency, long cycle, and the detection results are greatly affected by subjective factors, and personnel safety issues are difficult to guarantee. With the rapid development of the computer industry, deep learning algorithms have emerged and begun to attract the attention of researchers. Thanks to multi-layer neural networks, deep learning models have shown great advantages in characterizing complex road features, and have better robustness and accuracy when used for pavement crack detection. Therefore, more and more scholars have begun to devote themselves to the research of pavement crack detection based on deep learning, and continue to optimize and innovate it.

[0003] At present, the pavement crack detection methods based on deep learning can be mainly divided into three categories, namely, image-level classification methods, object detection methods and pixel-level classification methods. The image-level classification method uses a deep learning model to divide the pavement image into crack images and non-crack images. This type of algorithm is simple to operate, has high classification accuracy and fast speed, but it can only determine whether a single image contains pavement cracks, and cannot obtain relatively accurate crack location information. The object detection method uses the object detection model to realize the positioning and classification of the detection target, and the detection result will indicate its position and category in the form of a prediction box. Although the detection box is more accurate than the entire image, the crack still only occupies a part of it, and the damaged area of ​​the pavement crack cannot be accurately calculated. Therefore, the pixel-level classification method has begun to attract the attention of more and more researchers. Most of these methods are based on the encoder-decoder structure, using convolutional neural networks to distinguish between pavement and cracks pixel by pixel, which is more helpful for calculating pavement damage information. However, the accuracy of the model is still insufficient for pavement images containing various noises and complex background textures. Although many studies have alleviated this problem by introducing an attention mechanism, especially the introduction of non-local blocks (NL) to enable the model to effectively capture long-distance dependencies, thereby promoting the extraction of crack pixels. However, the huge computational cost of NL will severely limit the inference speed of the model, which is not conducive to practical application. How to model long-distance dependencies without significantly increasing the amount of calculation and build an efficient and accurate pavement crack segmentation model is currently a difficult problem.

[0004] Nowadays, the mileage of highway maintenance increases year by year, and the financial and manpower investment required to maintain the road network is quite huge. An efficient, accurate and robust pavement crack segmentation and quantification method is of great significance for monitoring pavement damage and reducing road maintenance expenses. Summary of the invention

[0005] In view of this, the present invention provides a pavement crack segmentation and quantification method based on pyramid regional attention, which can effectively capture long-distance dependencies while maintaining low computational costs, improve the segmentation accuracy of pavement cracks, and calculate the crack damage rate based on the segmentation results, thereby providing a basis for the assessment of road damage conditions, improving the efficiency of pavement maintenance decision-making, and reducing maintenance expenses.

[0006] The present invention is achieved by adopting the following technical solutions:

[0007] S1. Collect pavement crack images, make data sets and perform data enhancement processing;

[0008] S2. Based on the U-Net network structure, ResNet34 is used in combination with BAM to replace its encoder part, and the Spatial Attention Module (SA) and the Pyramid Region Attention Module (PRAM) are introduced into its jump connection part to build a pavement crack segmentation model.

[0009] S3, using the training set and the verification set to train and verify the pavement crack segmentation model, and optimizing the training rate and effect by adding auxiliary branches;

[0010] S4, using the trained crack segmentation model to perform crack recognition on the test set images to verify the generalization ability of the model;

[0011] S5, using the trained crack segmentation model to identify cracks in the road surface image to be detected, and outputting pixel-level crack detection results;

[0012] S6. Extract the crack skeleton and quantify the length of the pixel-level crack detection result obtained in S5, and calculate the crack damage rate.

[0013] In one embodiment, the image collection and data enhancement described in S1 further includes:

[0014] S11, using the CFD dataset as the original dataset, which contains pavement cracks of different types, scenes and sizes, and the image size is 480×320;

[0015] S12, dividing the original data set in S11 into a training set, a validation set and a test set according to a certain ratio;

[0016] S13, the training set, validation set and test set obtained in S12 are respectively cropped into patches of size 320×320, with a cropping step of 20 pixels;

[0017] S14, the patch image obtained in S13 is respectively rotated 90° counterclockwise, 180° counterclockwise, 270° counterclockwise, horizontally flipped, and vertically flipped, and added to the corresponding training set, validation set, and test set;

[0018] In one embodiment, the pavement crack segmentation model described in S2 is established based on the U-Net algorithm, including three parts: an encoder, a skip connection, and a decoder. The encoder extracts features of the input image and performs downsampling processing to obtain image features of different downsampling scales, including:

[0019] S21. When the input image size is (320, 320), the input three-channel image is adjusted to the matrix X(1, 3, 320, 320), where 1 represents the number of images input into the network at one time; 3 represents the number of feature channels of the matrix; the first 320 represents the height of the matrix; and the second 320 represents the width of the matrix;

[0020] S22, the input matrix X first enters the ResNet34+BAM encoder, passes through a convolution layer with a kernel of 7, a stride of 2, and a padding of 3, a BN layer, and a ReLU activation function layer to obtain the matrix E1 (1, 64, 160, 160), E1 passes through a maximum pooling layer with a kernel of 3, a stride of 2, and a padding of 1 to obtain the matrix E2 (1, 64, 80, 80), E2 enters the Residual Block (64, 1), passes through a convolution layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolution layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer to obtain the matrix E3 (1, 64, 80, 80), E2 and E3 are element-wise added to obtain E4 (1, 64, 80, 80), E4 enters the ResidualBlock (64, 1), passes through a kernel of 3, s A convolutional layer with a tride of 1 and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1 and a padding of 1, and a BN layer are used to obtain the matrix E5 (1, 64, 80, 80). E4 and E5 are element-wise added to obtain the matrix E6 (1, 64, 80, 80). E6 enters the ResidualBlock (64, 1), passes through a convolutional layer with a kernel of 3, a stride of 1 and a padding of 1, and a BN layer. N layers, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer are used to obtain the matrix E7 (1, 64, 80, 80). E6 and E7 are element-wise added to obtain the matrix E8 (1, 64, 80, 80). E8 enters BAM, passes through a global average pooling layer, and a fully connected layer to obtain the matrix E9 (1, 4, 1, 1). E9 passes through a fully connected layer to obtain the matrix E10 (1, 64, 1, 1). At the same time, E8 passes through a kernel of 1, s The convolution layer with tride of 1 and padding of 0, a BN layer and a ReLU activation function layer obtain the matrix E11 (1, 4, 80, 80). E11 passes through a dilated convolution layer with a kernel of 3, a stride of 1, a padding of 4, and a dilation rate of 4, a BN layer, a ReLU activation function layer, a dilated convolution layer with a kernel of 3, a stride of 1, a padding of 4, and a dilation rate of 4, a BN layer and a ReLU activation function layer to obtain the matrix E12 (1, 4, 80, 80).E12 passes through a convolution layer with a kernel of 1, a stride of 1, and a padding of 0 to obtain the matrix E13 (1, 1, 80, 80). E10 and E13 are expanded to (1, 64, 80, 80) and element-wise added. After passing through a Sigmoid activation function layer, the matrix E14 (1, 64, 80, 80) is obtained. E14 and E8 are element-wise multiplied and then element-wise added with E8 to obtain the matrix E15 (1, 64, 80, 80). E15 enters the Residual Block (128, 2), passes through a convolution layer with a kernel of 3, a stride of 2, and a padding of 1, a BN layer, a ReLU activation function layer, a convolution layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer to obtain the matrix E16 (1, 128, 40, 40). E16 enters the Residual Block (128, 2). Block(128, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E17(1, 128, 40, 40) is obtained. E17 enters the Residual Block(128, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E18(1, 128, 40, 40) is obtained. E18 enters the Residual Block (128, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E19 (1, 128, 40, 40) is obtained. E19 enters BAM, after a global average pooling layer and a fully connected layer, the matrix E20 (1, 8, 1, 1) is obtained. E20 passes through a fully connected layer to obtain the matrix E21 (1, 128, 1, 1). At the same time, E19 passes through a convolutional layer with a kernel of 1, a stride of 1, and a padding of 0, a BN layer, and a ReLU activation function layer to obtain the matrix E22 (1, 8, 40, 40).E22 passes through a dilated convolution layer with a kernel of 3, a stride of 1, a padding of 4, and a dilation rate of 4, a BN layer, a ReLU activation function layer, a dilated convolution layer with a kernel of 3, a stride of 1, a padding of 4, and a dilation rate of 4, a BN layer, and a ReLU activation function layer to obtain the matrix E23 (1, 8, 40, 40). E23 passes through a convolution layer with a kernel of 1, a stride of 1, and a padding of 0 to obtain the matrix E24 (1, 1, 40, 40). E21 and E24 are expanded to (1, 128, 40, 40) and element-wise added. After passing through a Sigmoid activation function layer, the matrix E25 (1, 128, 40, 40) is obtained. E25 is element-wise multiplied with E19 and then element-wise added with E19 to obtain the matrix E26 (1, 128, 40, 40). E26 enters the Residual Block(256, 2), after a convolutional layer with a kernel of 3, a stride of 2, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E27(1, 256, 20, 20) is obtained. E27 enters the Residual Block(256, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E28(1, 256, 20, 20) is obtained. E28 enters the Residual Block(256, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E29(1, 256, 20, 20) is obtained. E29 enters the Residual Block(256, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E30(1, 256, 20, 20) is obtained. E30 enters the Residual Block(256, 1),After a convolutional layer with kernel 3, stride 1, and padding 1, a BN layer, a ReLU activation function layer, a convolutional layer with kernel 3, stride 1, and padding 1, and a BN layer, the matrix E31 (1, 256, 20, 20) is obtained. E31 enters the Residual Block(256, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E32(1, 256, 20, 20) is obtained. E32 enters BAM, and after a global average pooling layer and a fully connected layer, the matrix E33(1, 16, 1, 1) is obtained. E33 passes through a fully connected layer to obtain the matrix E34(1, 256, 1, 1). At the same time, E32 passes through a convolutional layer with a kernel of 1, a stride of 1, and a padding of 0, a BN layer, and a ReLU activation function layer to obtain the matrix E35(1, 16, 20, 20). E35 passes through a convolutional layer with a kernel of 3, a stride of 1, a padding of 4, and a dilation rate of 4, a BN layer, a ReLU activation function layer, a kernel of 3, a stride of 1, a padding of 4, a dilated convolution layer with a dilated rate of 4, a BN layer and a ReLU activation function layer to obtain the matrix E36 (1, 16, 20, 20). E36 passes through a convolution layer with a kernel of 1, a stride of 1, and a padding of 0 to obtain the matrix E37 (1, 1, 20, 20). E34 and E37 are expanded to (1, 256, 20, 20) and element-wise added. After passing through a Sigmoid activation function layer, the matrix E38 (1, 256, 20, 20) is obtained. E38 is element-wise multiplied with E32 and then element-wise added with E32 to obtain the matrix E39 (1, 256, 20, 20). E39 enters the Residual Block (512, 2), after a convolutional layer with kernel 3, stride 2, padding 1, a BN layer, a ReLU activation function layer, a convolutional layer with kernel 3, stride 1, padding 1, and a BN layer, the matrix E40 (1, 512, 10, 10) is obtained. E40 enters Residual Block (512, 1).After a convolutional layer with kernel 3, stride 1, and padding 1, a BN layer, a ReLU activation function layer, a convolutional layer with kernel 3, stride 1, and padding 1, and a BN layer, the matrix E41 (1, 512, 10, 10) is obtained. E41 enters the Residual Block(512, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E42(1, 512, 10, 10) is obtained. E42 enters BAM, and after a global average pooling layer and a fully connected layer, the matrix E43(1, 32, 1, 1) is obtained. E43 passes through a fully connected layer to obtain the matrix E44(1, 512, 1, 1). At the same time, E42 passes through a convolutional layer with a kernel of 1, a stride of 1, and a padding of 0, a BN layer, and a ReLU activation function layer to obtain the matrix E45(1, 32, 10, 10). E45 passes through a convolutional layer with a kernel of 3, a stride of 1, and a padding of 4. A dilated convolution layer with a dilation rate of 4, a BN layer, a ReLU activation function layer, a dilated convolution layer with a kernel of 3, a stride of 1, a padding of 4, and a dilation rate of 4, a BN layer, and a ReLU activation function layer are used to obtain the matrix E46 (1, 32, 10, 10). E46 passes through a convolution layer with a kernel of 1, a stride of 1, and a padding of 0 to obtain the matrix E47 (1, 1, 10, 10). E44 and E47 are expanded to (1, 512, 10, 10) and element-wise added. After passing through a Sigmoid activation function layer, the matrix E48 (1, 512, 10, 10) is obtained. E48 is element-wise multiplied with E42 and then element-wise added with E42 to obtain the matrix E49 (1, 512, 10, 10).

[0021] S23, matrices E1, E15, E26, E39 and E49 are collectively the outputs of the encoder, wherein E1, E15, E26, E39 are input to the skip connection part, and E49 is input to the decoder.

[0022] In one embodiment, the road crack segmentation model described in S2 further enhances the attention of the output of the encoder through the jump connection part to obtain the refined image features, including:

[0023] S24, matrix E1 enters SA, and global average pooling and global maximum pooling are performed on the feature channel dimension to obtain matrix A1 (1, 1, 160, 160) and matrix A2 (1, 1, 160, 160). A1 and A2 are concatenated on the feature channel dimension to obtain matrix A3 (1, 2, 160, 160). A3 passes through a convolution layer with a kernel of 7, a stride of 1, and a padding of 3 and a Sigmoid activation function layer to obtain matrix A4 (1, 1, 160, 160). A4 is expanded to (1, 64, 160, 160) and element-wise multiplied with E1, and then element-wise added with E1. The matrix A5 (1, 64, 160, 160) is obtained, E15 enters SA, and global average pooling and global maximum pooling are performed on the feature channel dimension to obtain matrix A6 (1, 1, 80, 80) and matrix A7 (1, 1, 80, 80). A6 and A7 are concatenated on the feature channel dimension to obtain matrix A8 (1, 2, 80, 80). A8 passes through a convolution layer with a kernel of 7, a stride of 1, and a padding of 3 and a Sigmoid activation function layer to obtain matrix A9 (1, 1, 80, 80). A9 is expanded to (1, 64, 80, 80) and element-wise multiplied with E15, and then elem ent-wise sum to obtain the matrix A10 (1, 64, 80, 80), E26 enters PRAM, and passes through four average pooling layers to obtain the matrices A11 (1, 128, 1, 1), A12 (1, 128, 2, 2), A13 (1, 128, 3, 3) and A14 (1, 128, 6, 6), A11 passes through a convolution layer with kernel 1, stride 1, and padding 0 to obtain the matrix A15 (1, 128, 1, 1), A15 passes through bilinear interpolation to obtain the matrix A16 (1, 128, 40, 40), A12 enters SimNL, and passes through kernel 1, stride 1, and padding 0 respectively. The convolution layer of 0 obtains matrices A17 (1, 64, 2, 2), A18 (1, 64, 2, 2) and A19 (1, 64, 2, 2). A17 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrices A20 (1, 1, 2, 2) and matrix A21 (1, 1, 2, 2). A20 and A21 are concatenated and adjusted on the feature channel dimension to obtain matrix A22 (1, 4, 2). A18 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A23 (1, 1, 2, 2) and matrix A24 (1, 1, 2, 2). A23 and A24 are concatenated and adjusted on the feature channel dimension to obtain matrix A25 (1, 2, 4).A22 and A25 perform matrix multiplication and pass through a Softmax activation function layer to obtain matrix A26 (1, 4, 4). A19 is adjusted to (1, 4, 64) and matrix multiplied with A26 and adjusted to obtain matrix A27 (1, 64, 2, 2). A27 passes through a convolution layer with kernel 1, stride 1, and padding 0, and a BN layer to obtain matrix A28 (1, 128, 2, 2). A28 passes through bilinear interpolation to obtain matrix A29 (1, 128, 40, 40). A13 enters SimNL and passes through convolution layers with kernel 1, stride 1, and padding 0 to obtain matrix A30 (1, 64, 3, 3 ), A31 (1, 64, 3, 3) and A32 (1, 64, 3, 3), A30 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A33 (1, 1, 3, 3) and matrix A34 (1, 1, 3, 3), A33 and A34 are concatenated and adjusted on the feature channel dimension to obtain matrix A35 (1, 9, 2), A31 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A36 (1, 1, 3, 3) and matrix A37 (1, 1, 3, 3), A36 and A37 are concatenated and adjusted on the feature channel dimension to obtain matrix A38 (1, 2, 9), A35 and A38 perform matrix multiplication and pass a Softma The x activation function layer obtains the matrix A39 (1, 9, 9), adjusts A32 to (1, 9, 64) and performs matrix multiplication with A39 and adjusts it to obtain the matrix A40 (1, 64, 3, 3), A40 passes through a convolution layer with a kernel of 1, a stride of 1, and a padding of 0, and a BN layer to obtain the matrix A41 (1, 128, 3, 3), A41 passes through bilinear interpolation to obtain the matrix A42 (1, 128, 40, 40), A14 enters SimNL, and passes through convolution layers with a kernel of 1, a stride of 1, and a padding of 0 to obtain the matrices A43 (1, 64, 6, 6), A44 (1, 64, 6, 6) and A45 (1, 64 , 6, 6), A43 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A46 (1, 1, 6, 6) and matrix A47 (1, 1, 6, 6), A46 and A47 are concatenated and adjusted on the feature channel dimension to obtain matrix A48 (1, 36, 2), A44 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A49 (1, 1, 6, 6) and matrix A50 (1, 1, 6, 6), A49 and A50 are concatenated and adjusted on the feature channel dimension to obtain matrix A51 (1, 2, 36), A48 and A51 perform matrix multiplication and pass through a Softmax activation function layer to obtain matrix A52 (1, 36, 36),A45 is adjusted to (1, 36, 64) and matrix multiplication is performed with A52 to obtain matrix A53 (1, 64, 6, 6). A53 passes through a convolution layer with a kernel of 1, a stride of 1, and a padding of 0, and a BN layer to obtain matrix A54 (1, 128, 6, 6). A54 is subjected to bilinear interpolation to obtain matrix A55 (1, 128, 40, 40). A16, A29, A42, A55 and E26 are concatenated on the feature channel to obtain matrix A56 (1, 640, 40, 40). A56 passes through a convolution layer with a kernel of 1, a stride of 1, and a padding of 0 to obtain matrix A57 (1, 128, 40, 40). ), E39 enters PRAM and passes through four average pooling layers to obtain matrices A58 (1, 256, 1, 1), A59 (1, 256, 2, 2), A60 (1, 256, 3, 3) and A61 (1, 256, 6, 6), A58 passes through a convolution layer with kernel 1, stride 1, and padding 0 to obtain matrix A62 (1, 256, 1, 1), A62 passes through bilinear interpolation to obtain matrix A63 (1, 256, 20, 20), A59 enters SimNL and passes through convolution layers with kernel 1, stride 1, and padding 0 to obtain matrix A64 (1, 128, 2, 2), A65 (1, 128, 2, 2) and A66 (1, 128, 2, 2), A64 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A67 (1, 1, 2, 2) and matrix A68 (1, 1, 2, 2), A67 and A68 are concatenated and adjusted on the feature channel dimension to obtain matrix A69 (1, 4, 2), A65 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A70 (1, 1, 2, 2) and matrix A71 (1, 1, 2, 2), matrix A70 and matrix A71 are concatenated and adjusted on the feature channel dimension to obtain matrix A72 (1, 2, 4), A69 and A72 perform matrix multiplication and pass through a Softmax activation function layer to obtain To A73 (1, 4, 4), A66 is adjusted to (1, 4, 128) and matrix multiplication is performed with A73 and adjusted to obtain matrix A74 (1, 128, 2, 2). A74 passes through a convolution layer with kernel 1, stride 1, and padding 0, and a BN layer to obtain matrix A75 (1, 256, 2, 2). A75 passes through bilinear interpolation to obtain matrix A76 (1, 256, 20, 20). A60 enters SimNL and passes through convolution layers with kernel 1, stride 1, and padding 0 to obtain matrices A77 (1, 128, 3, 3), A78 (1, 128, 3, 3) and A79 (1, 128, 3, 3).A77 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A80 (1, 1, 3, 3) and matrix A81 (1, 1, 3, 3), A80 and A81 are concatenated and adjusted on the feature channel dimension to obtain matrix A82 (1, 9, 2), A78 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A83 (1, 1, 3, 3) and matrix A84 (1, 1, 3, 3), A83 and A84 are concatenated and adjusted on the feature channel dimension to obtain matrix A85 (1, 2, 9), A82 and A85 perform matrix multiplication and pass through a Softmax activation function layer to obtain matrix A86 (1, 9, 9), and A79 is adjusted to (1, 9, 128) and performs matrix multiplication with A86 and adjusts it to obtain matrix A87 (1, 128, 3, 3). A87 passes through a convolution layer with kernel 1, stride 1, and padding 0, and a BN layer to obtain matrix A88 (1, 256, 3, 3). A88 passes through bilinear interpolation to obtain matrix A89 (1, 256, 20, 20). A61 enters SimNL and passes through convolution layers with kernel 1, stride 1, and padding 0 to obtain matrices A90 (1, 128, 6, 6), A91 (1, 128, 6, 6) and A92 (1, 128, 6, 6). A90 is respectively multiplied on the feature channel dimension. Global average pooling and global maximum pooling are performed to obtain matrix A93 (1, 1, 6, 6) and matrix A94 (1, 1, 6, 6). A93 and A94 are concatenated and adjusted in the feature channel dimension to obtain matrix A95 (1, 36, 2). A91 performs global average pooling and global maximum pooling in the feature channel dimension to obtain matrix A96 (1, 1, 6, 6) and matrix A97 (1, 1, 6, 6). A96 and A97 are concatenated and adjusted in the feature channel dimension to obtain matrix A98 (1, 2, 36). A95 and A98 are matrix multiplied and passed through a Softmax activation function layer to obtain matrix A99 (1, 36, 36). A92 is adjusted to (1, 36, 128) and Matrix A100 (1, 128, 6, 6) is obtained by matrix multiplication and adjustment with A99. A100 passes through a convolution layer with kernel 1, stride 1, and padding 0, and a BN layer to obtain matrix A101 (1, 256, 6, 6). A101 is bilinearly interpolated to obtain matrix A102 (1, 256, 20, 20). A63, A76, A89, A102 and E39 are concatenated on the feature channel to obtain matrix A103 (1, 1280, 20, 20). A103 passes through a convolution layer with kernel 1, stride 1, and padding 0 to obtain matrix A104 (1, 256, 20, 20).

[0024] S25, matrices A5, A10, A57, and A104 are collectively input into the decoder as the output of the skip connection part.

[0025] In one embodiment, the road crack segmentation model described in S2 performs feature integration and upsampling processing on the output of the encoder and the jump connection part through the decoder to obtain the final segmentation result, including:

[0026] S26, matrices A5, A10, A57, A104 and E49 enter the decoder, E49 enters the Upsample Block (256), and after bilinear interpolation, the matrix D1 (1, 512, 20, 20) is obtained. D1 passes through a convolution layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, and a ReLU activation function layer to obtain the matrix D2 (1, 256, 20, 20). A104 and D2 are concatenated in the feature channel dimension to obtain the matrix D3 (1, 512, 20, 20), and D3 enters the Upconv Block(256), after a convolutional layer with kernel 3, stride 1, padding 1, a BN layer and a ReLU activation function layer, the matrix D4(1, 256, 20, 20) is obtained. D4 passes through a convolutional layer with kernel 3, stride 1, padding 1, a BN layer and a ReLU activation function layer to obtain the matrix D5(1, 256, 20, 20). D5 enters Upsample Block(128), and after bilinear interpolation, the matrix D6(1, 256, 40, 40) is obtained. D6 passes through a convolutional layer with kernel 3, stride 1, padding 1, a BN layer and a ReLU activation function layer to obtain the matrix D7(1, 128, 40, 40). A57 and D7 are concatenated in the feature channel dimension to obtain the matrix D8(1, 256, 40, 40). D8 enters Upconv Block (128), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, and a ReLU activation function layer, the matrix D9 (1, 128, 40, 40) is obtained. D9 passes through a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, and a ReLU activation function layer to obtain the matrix D10 (1, 128, 40, 40). D10 enters Upsample Block (64), and after bilinear interpolation, the matrix D11 (1, 128, 80, 80) is obtained. D11 passes through a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, and a ReLU activation function layer to obtain the matrix D12 (1, 64, 80, 80). A10 and D12 are concatenated in the feature channel dimension to obtain D13 (1, 128, 80, 80). D13 enters Upconv Block (64), after a convolutional layer with kernel 3, stride 1, padding 1, a BN layer and a ReLU activation function layer, the matrix D14 (1, 64, 80, 80) is obtained.D14 passes through a convolution layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, and a ReLU activation function layer to obtain a matrix D15 (1, 64, 80, 80). D15 enters the Upsample Block (64) and obtains a matrix D16 (1, 64, 160, 160) through bilinear interpolation. D16 passes through a convolution layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, and a ReLU activation function layer to obtain a matrix D17 (1, 64, 160, 160). A5 and D17 are concatenated in the feature channel dimension to obtain a matrix D18 (1, 128, 160, 160). D18 enters the Upconv Block (64), after a convolution layer with kernel 3, stride 1, padding 1, a BN layer and a ReLU activation function layer, the matrix D19 (1, 64, 160, 160) is obtained. D19 passes through a convolution layer with kernel 3, stride 1, padding 1, a BN layer and a ReLU activation function layer to obtain the matrix D20 (1, 64, 160, 160). D20 enters Upconvlast Block (2), and after bilinear interpolation, the matrix D21 (1, 64, 320, 320) is obtained. D21 passes through a convolution layer with kernel 1, stride 1, padding 0 to obtain the final network output Y (1, 2, 320, 320), where 2 represents the category of each pixel in the image, i.e. crack and non-crack. ,

[0027] In one implementation, the auxiliary branch described in S3 only participates in auxiliary training and is cancelled in the test phase and actual prediction. The step of upsampling the encoder intermediate feature E26 through the auxiliary branch to obtain the final segmentation result includes:

[0028] S31, the input three-channel image is adjusted to the matrix X(1, 3, 320, 320), where 1 represents the number of images input into the network at one time; 3 represents the number of feature channels of the matrix; the first 320 represents the height of the matrix; and the second 320 represents the width of the matrix;

[0029] S32 and matrix E26 are bilinearly interpolated to obtain F1(1, 128, 160, 160), F1 is passed through a convolution layer with kernel 1, stride 1, and padding 0 to obtain F2(1, 64, 160, 160), F2 is passed through a bilinear interpolation to obtain F3(1, 64, 320, 320), F3 is passed through a convolution layer with kernel 1, stride 1, and padding 0 to obtain the auxiliary branch final output Y*(1, 2, 320, 320), where 2 represents the category of each pixel in the image, i.e., crack and non-crack.

[0030] In one embodiment, the model training and verification step described in S3 further includes:

[0031] S33, based on the training set images and corresponding labels, input the labeled images into the network model for training, and output the pixel-level segmentation results of the road cracks;

[0032] S34, calculating the main network loss and the auxiliary branch loss;

[0033] S35, optimizing network parameters according to the network loss calculated in S34;

[0034] S36. Based on the validation set images and corresponding labels, the labeled images are input into the network model for verification, the evaluation index F1 value is calculated, and the optimal weights are saved and updated.

[0035] In one embodiment, the network loss function further includes:

[0036] S37, the main network loss consists of cross entropy loss and Dice coefficient loss:

[0037]

[0038] where p ic Indicates the true label that the i-th pixel belongs to category c; q ic Indicates the main network output that the i-th pixel belongs to category c; n represents the total number of pixels in the image; k represents the category and is 2;

[0039] S38, auxiliary branch loss is composed of Dice coefficient loss:

[0040]

[0041] where p ic Indicates the true label that the i-th pixel belongs to category c; z ic Indicates that the i-th pixel belongs to the auxiliary branch output of category c; n represents the total number of pixels in the image; k represents the category and is 2.

[0042] In one embodiment, the step of quantifying the pavement cracks and calculating the damage rate described in S6 includes:

[0043] S61, extracting the crack skeleton in the crack segmentation image using the Hilditch algorithm;

[0044] S62, calculating the pixel length of the crack skeleton described in S61;

[0045] S63, calculating a conversion coefficient α between image pixels and actual size by measuring the size of the actual photographed object and its corresponding pixel value on the image for multiple times;

[0046] S64, calculating the true crack length according to the results of S62 and S63;

[0047] S65. Calculate the crack damage rate DR according to Formula 3 in accordance with the Highway Technical Condition Assessment Standard (JTG 5210-2018).

[0048]

[0049] Among them, w i is the conversion coefficient of the automated detection of the i-th type of pavement damage; i represents the type of pavement damage; i0 represents the total number of pavement damages in the detection area; A i is the cumulative area of ​​the i-th type of pavement damage (m 2 ); A is the road surface detection area (m 2 ).

[0050] The present invention at least includes the following beneficial effects: the present invention discloses a pavement crack segmentation and quantification method based on pyramid regional attention, the pavement crack segmentation model is more efficient and accurate, is based on the U-Net structure, uses the ResNet34 network combined with BAM as an encoder to achieve efficient feature extraction, introduces SA in the shallow skip connection of the network to enhance the crack space details, and introduces PRAM in the deep skip connection of the network to achieve multi-scale context information integration and long-distance dependency capture with low computational cost, thereby improving the detection accuracy while maintaining a faster reasoning speed; the crack segmentation model adds an auxiliary branch in the training stage to assist the encoder training, thereby improving the training rate and model accuracy; the crack quantification method is based on the detection result of the pavement crack segmentation model, and after extracting the crack skeleton in the crack segmentation image by the Hilditch algorithm, the length and damage rate of the crack are calculated, providing an important basis for the evaluation of road damage conditions and the formulation of maintenance decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a flow chart of a road crack segmentation and quantification method based on pyramid area attention provided by the present invention;

[0052] Figure 2 It is a network structure diagram of a pavement crack segmentation model based on pyramid area attention provided by the present invention;

[0053] Figure 3 It is a detailed diagram of the network structure of the pavement crack segmentation model based on pyramid area attention provided by the present invention;

[0054] Figure 4 It is an auxiliary branch network structure diagram of the pavement crack segmentation model based on pyramid area attention provided by the present invention;

[0055] Figure 5 It is a partial detection result diagram of the method of the present invention and the existing semantic segmentation model on the CFD data set;

[0056] Figure 6 This is a comparison of the crack skeleton effects using the Zhang-Suen algorithm and the Hilditch algorithm;

[0057] Figure 7 It is a comparison of crack skeleton width using Zhang-Suen algorithm and Hilditch algorithm;

[0058] Figure 8 This is a demonstration diagram of crack segmentation and quantification performed on some larger-sized actual pavement crack images by the method proposed in the present invention. DETAILED DESCRIPTION

[0059] In order to make the technical route and advantages of the present invention more clearly understood, the present invention is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0060] The present invention provides a road crack segmentation and quantification method based on pyramid area attention, the flow chart is as follows Figure 1 As shown, the specific steps include:

[0061] S100, collecting pavement crack images, making data sets and performing data enhancement processing;

[0062] It should be noted that, in one embodiment, the data set uses CFD, which contains pavement cracks of different types, scenes and sizes, with an image size of 480×320, and a total of 118 images. The specific steps of data set preparation and data enhancement processing include: (1) dividing into training set, validation set and test set according to 6:2:2; (2) cropping the training set, validation set and test set into patches of 320×320 size with a cropping step of 20 pixels, and rotating the obtained patch images by 90°, 180°, 270°, horizontal flipping and vertical flipping respectively, and then adding them to the corresponding training set, validation set and test set. The final data set used has a total of 6372 images, of which the training set includes 3834 images, the validation set includes 1296 images, and the test set includes 1242 images.

[0063] S200, based on the U-Net network structure, uses ResNet34 combined with BAM to replace its encoder part, and introduces SA and PRAM in its jump connection part to build a pavement crack segmentation model;

[0064] It should be noted that, in one embodiment, the pavement crack segmentation model includes three parts: an encoder, a skip connection, and a decoder. The encoder first extracts features from the input image and performs downsampling processing to obtain image features of different downsampling scales. These image features are then further enhanced through the skip connection part to obtain refined image features. Finally, the decoder performs feature integration and upsampling processing on the outputs of the encoder and the skip connection part to obtain the final pixel-level segmentation result. The specific network structure is as follows: Figure 2 and Figure 3 shown.

[0065] It should be further explained that, in one implementation, PRAM performs average pooling, self-attention enhancement and upsampling on the input feature map at four scales, which not only fully utilizes the global multi-scale context information and realizes the long-range dependency modeling of each scale feature map sub-region. At the same time, since the size of the feature map entering SimNL is reduced to a fixed value, the computational cost required for self-attention is greatly reduced.

[0066] In one embodiment, the size of the input image is (320, 320), and the forward propagation process of the network is as follows:

[0067] The input three-channel image is adjusted to the matrix X(1, 3, 320, 320), where 1 represents the number of images input into the network at one time; 3 represents the number of feature channels of the matrix; the first 320 represents the height of the matrix; and the second 320 represents the width of the matrix.

[0068] The input matrix X first enters the ResNet34+BAM encoder, passes through a convolution layer with a kernel of 7, a stride of 2, and a padding of 3, a BN layer, and a ReLU activation function layer to obtain the matrix E1 (1, 64, 160, 160), E1 passes through a maximum pooling layer with a kernel of 3, a stride of 2, and a padding of 1 to obtain the matrix E2 (1, 64, 80, 80), E2 enters the Residual Block (64, 1), passes through a convolution layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolution layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer to obtain the matrix E3 (1, 64, 80, 80), E2 and E3 are element-wise added to obtain E4 (1, 64, 80, 80), E4 enters the ResidualBlock (64, 1), passes through a kernel of 3, s A convolutional layer with a tride of 1 and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1 and a padding of 1, and a BN layer are used to obtain the matrix E5 (1, 64, 80, 80). E4 and E5 are element-wise added to obtain the matrix E6 (1, 64, 80, 80). E6 enters the ResidualBlock (64, 1), passes through a convolutional layer with a kernel of 3, a stride of 1 and a padding of 1, and a BN layer. N layers, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer are used to obtain the matrix E7 (1, 64, 80, 80). E6 and E7 are element-wise added to obtain the matrix E8 (1, 64, 80, 80). E8 enters BAM, passes through a global average pooling layer, and a fully connected layer to obtain the matrix E9 (1, 4, 1, 1). E9 passes through a fully connected layer to obtain the matrix E10 (1, 64, 1, 1). At the same time, E8 passes through a kernel of 1, s The convolution layer with tride of 1 and padding of 0, a BN layer and a ReLU activation function layer obtain the matrix E11 (1, 4, 80, 80). E11 passes through a dilated convolution layer with a kernel of 3, a stride of 1, a padding of 4, and a dilation rate of 4, a BN layer, a ReLU activation function layer, a dilated convolution layer with a kernel of 3, a stride of 1, a padding of 4, and a dilation rate of 4, a BN layer and a ReLU activation function layer to obtain the matrix E12 (1, 4, 80, 80).E12 passes through a convolution layer with a kernel of 1, a stride of 1, and a padding of 0 to obtain the matrix E13 (1, 1, 80, 80). E10 and E13 are expanded to (1, 64, 80, 80) and element-wise added. After passing through a Sigmoid activation function layer, the matrix E14 (1, 64, 80, 80) is obtained. E14 and E8 are element-wise multiplied and then element-wise added with E8 to obtain the matrix E15 (1, 64, 80, 80). E15 enters the Residual Block (128, 2), passes through a convolution layer with a kernel of 3, a stride of 2, and a padding of 1, a BN layer, a ReLU activation function layer, a convolution layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer to obtain the matrix E16 (1, 128, 40, 40). E16 enters the Residual Block (128, 2). Block(128, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E17(1, 128, 40, 40) is obtained. E17 enters the Residual Block(128, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E18(1, 128, 40, 40) is obtained. E18 enters the Residual Block (128, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E19 (1, 128, 40, 40) is obtained. E19 enters BAM, after a global average pooling layer and a fully connected layer, the matrix E20 (1, 8, 1, 1) is obtained. E20 passes through a fully connected layer to obtain the matrix E21 (1, 128, 1, 1). At the same time, E19 passes through a convolutional layer with a kernel of 1, a stride of 1, and a padding of 0, a BN layer, and a ReLU activation function layer to obtain the matrix E22 (1, 8, 40, 40).E22 passes through a dilated convolution layer with a kernel of 3, a stride of 1, a padding of 4, and a dilation rate of 4, a BN layer, a ReLU activation function layer, a dilated convolution layer with a kernel of 3, a stride of 1, a padding of 4, and a dilation rate of 4, a BN layer, and a ReLU activation function layer to obtain the matrix E23 (1, 8, 40, 40). E23 passes through a convolution layer with a kernel of 1, a stride of 1, and a padding of 0 to obtain the matrix E24 (1, 1, 40, 40). E21 and E24 are expanded to (1, 128, 40, 40) and element-wise added. After passing through a Sigmoid activation function layer, the matrix E25 (1, 128, 40, 40) is obtained. E25 is element-wise multiplied with E19 and then element-wise added with E19 to obtain the matrix E26 (1, 128, 40, 40). E26 enters the Residual Block(256, 2), after a convolutional layer with a kernel of 3, a stride of 2, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E27(1, 256, 20, 20) is obtained. E27 enters the Residual Block(256, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E28(1, 256, 20, 20) is obtained. E28 enters the Residual Block(256, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E29(1, 256, 20, 20) is obtained. E29 enters the Residual Block(256, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E30(1, 256, 20, 20) is obtained. E30 enters the Residual Block(256, 1),After a convolutional layer with kernel 3, stride 1, and padding 1, a BN layer, a ReLU activation function layer, a convolutional layer with kernel 3, stride 1, and padding 1, and a BN layer, the matrix E31 (1, 256, 20, 20) is obtained. E31 enters the Residual Block(256, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E32(1, 256, 20, 20) is obtained. E32 enters BAM, and after a global average pooling layer and a fully connected layer, the matrix E33(1, 16, 1, 1) is obtained. E33 passes through a fully connected layer to obtain the matrix E34(1, 256, 1, 1). At the same time, E32 passes through a convolutional layer with a kernel of 1, a stride of 1, and a padding of 0, a BN layer, and a ReLU activation function layer to obtain the matrix E35(1, 16, 20, 20). E35 passes through a convolutional layer with a kernel of 3, a stride of 1, a padding of 4, and a dilation rate of 4, a BN layer, a ReLU activation function layer, a kernel of 3, a stride of 1, a padding of 4, a dilated convolution layer with a dilated rate of 4, a BN layer and a ReLU activation function layer to obtain the matrix E36 (1, 16, 20, 20). E36 passes through a convolution layer with a kernel of 1, a stride of 1, and a padding of 0 to obtain the matrix E37 (1, 1, 20, 20). E34 and E37 are expanded to (1, 256, 20, 20) and element-wise added. After passing through a Sigmoid activation function layer, the matrix E38 (1, 256, 20, 20) is obtained. E38 is element-wise multiplied with E32 and then element-wise added with E32 to obtain the matrix E39 (1, 256, 20, 20). E39 enters the Residual Block (512, 2), after a convolutional layer with kernel 3, stride 2, padding 1, a BN layer, a ReLU activation function layer, a convolutional layer with kernel 3, stride 1, padding 1, and a BN layer, the matrix E40 (1, 512, 10, 10) is obtained. E40 enters Residual Block (512, 1).After a convolutional layer with kernel 3, stride 1, and padding 1, a BN layer, a ReLU activation function layer, a convolutional layer with kernel 3, stride 1, and padding 1, and a BN layer, the matrix E41 (1, 512, 10, 10) is obtained. E41 enters the Residual Block(512, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E42(1, 512, 10, 10) is obtained. E42 enters BAM, and after a global average pooling layer and a fully connected layer, the matrix E43(1, 32, 1, 1) is obtained. E43 passes through a fully connected layer to obtain the matrix E44(1, 512, 1, 1). At the same time, E42 passes through a convolutional layer with a kernel of 1, a stride of 1, and a padding of 0, a BN layer, and a ReLU activation function layer to obtain the matrix E45(1, 32, 10, 10). E45 passes through a convolutional layer with a kernel of 3, a stride of 1, and a padding of 4. A dilated convolution layer with a dilation rate of 4, a BN layer, a ReLU activation function layer, a dilated convolution layer with a kernel of 3, a stride of 1, a padding of 4, and a dilation rate of 4, a BN layer, and a ReLU activation function layer are used to obtain the matrix E46 (1, 32, 10, 10). E46 passes through a convolution layer with a kernel of 1, a stride of 1, and a padding of 0 to obtain the matrix E47 (1, 1, 10, 10). E44 and E47 are expanded to (1, 512, 10, 10) and element-wise added. After passing through a Sigmoid activation function layer, the matrix E48 (1, 512, 10, 10) is obtained. E48 is element-wise multiplied with E42 and then element-wise added with E42 to obtain the matrix E49 (1, 512, 10, 10).

[0069] Afterwards, matrices E1, E15, E26 and E39 are further enhanced. E1 enters SA and performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A1 (1, 1, 160, 160) and matrix A2 (1, 1, 160, 160). A1 and A2 are concatenated on the feature channel dimension to obtain matrix A3 (1, 2, 160, 160). A3 passes through a convolutional layer with a kernel of 7, a stride of 1, and a padding of 3 and a Sigmoid activation function layer to obtain matrix A4 (1, 1, 160, 160). A4 is expanded to (1, 64, 160, 160) and element-wise multiplied with E1. After the product, the matrix A5 (1, 64, 160, 160) is obtained by element-wise addition with E1. E15 enters SA, and global average pooling and global maximum pooling are performed on the feature channel dimension to obtain matrix A6 (1, 1, 80, 80) and matrix A7 (1, 1, 80, 80). A6 and A7 are concatenated on the feature channel dimension to obtain matrix A8 (1, 2, 80, 80). A8 passes through a convolution layer with a kernel of 7, a stride of 1, and a padding of 3, and a Sigmoid activation function layer to obtain matrix A9 (1, 1, 80, 80). A9 is expanded to (1, 64, 80, 80) and element-wise with E15. After the product of e, the matrix A10 (1, 64, 80, 80) is obtained by element-wise addition with E15. E26 enters PRAM and passes through four average pooling layers to obtain matrices A11 (1, 128, 1, 1), A12 (1, 128, 2, 2), A13 (1, 128, 3, 3) and A14 (1, 128, 6, 6). A11 passes through a convolution layer with kernel 1, stride 1, and padding 0 to obtain matrix A15 (1, 128, 1, 1). A15 passes through bilinear interpolation to obtain matrix A16 (1, 128, 40, 40). A12 enters SimNL and passes through kernel 1, stride 1, The convolutional layer with padding of 0 obtains matrices A17 (1, 64, 2, 2), A18 (1, 64, 2, 2) and A19 (1, 64, 2, 2). A17 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrices A20 (1, 1, 2, 2) and A21 (1, 1, 2, 2). A20 and A21 are concatenated and adjusted on the feature channel dimension to obtain matrix A22 (1, 4, 2). A18 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A23 (1, 1, 2, 2) and A24 (1, 1, 2, 2). A23 and A24 are concatenated and adjusted on the feature channel dimension to obtain matrix A25 (1, 2,4), A22 and A25 are matrix multiplied and passed through a Softmax activation function layer to obtain matrix A26 (1, 4, 4), A19 is adjusted to (1, 4, 64) and matrix multiplied with A26 and adjusted to obtain matrix A27 (1, 64, 2, 2), A27 passes through a convolution layer with kernel 1, stride 1, and padding 0, and a BN layer to obtain matrix A28 (1, 128, 2, 2), A28 is bilinearly interpolated to obtain matrix A29 (1, 128, 40, 40), A13 enters SimNL, and passes through convolution layers with kernel 1, stride 1, and padding 0 to obtain matrix A30 (1, 64, 3 , 3), A31 (1, 64, 3, 3) and A32 (1, 64, 3, 3), A30 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A33 (1, 1, 3, 3) and matrix A34 (1, 1, 3, 3), A33 and A34 are concatenated and adjusted on the feature channel dimension to obtain matrix A35 (1, 9, 2), A31 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A36 (1, 1, 3, 3) and matrix A37 (1, 1, 3, 3), A36 and A37 are concatenated and adjusted on the feature channel dimension to obtain matrix A38 (1, 2, 9), A35 and A38 perform matrix multiplication and pass a Softmax The ax activation function layer obtains the matrix A39 (1, 9, 9), adjusts A32 to (1, 9, 64) and performs matrix multiplication with A39 and adjusts it to obtain the matrix A40 (1, 64, 3, 3), A40 passes through a convolution layer with a kernel of 1, a stride of 1, and a padding of 0, and a BN layer to obtain the matrix A41 (1, 128, 3, 3), A41 passes through bilinear interpolation to obtain the matrix A42 (1, 128, 40, 40), A14 enters SimNL, and passes through convolution layers with a kernel of 1, a stride of 1, and a padding of 0 to obtain the matrices A43 (1, 64, 6, 6), A44 (1, 64, 6, 6) and A45 (1, 64 , 6, 6), A43 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A46 (1, 1, 6, 6) and matrix A47 (1, 1, 6, 6), A46 and A47 are concatenated and adjusted on the feature channel dimension to obtain matrix A48 (1, 36, 2), A44 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A49 (1, 1, 6, 6) and matrix A50 (1, 1, 6, 6), A49 and A50 are concatenated and adjusted on the feature channel dimension to obtain matrix A51 (1, 2, 36), A48 and A51 perform matrix multiplication and pass through a Softmax activation function layer to obtain matrix A52 (1, 36, 36),A45 is adjusted to (1, 36, 64) and matrix multiplication is performed with A52 to obtain matrix A53 (1, 64, 6, 6). A53 passes through a convolution layer with kernel 1, stride 1, and padding 0, and a BN layer to obtain matrix A54 (1, 128, 6, 6). A41 is bilinearly interpolated to obtain matrix A55 (1, 128, 40, 40). A16, A29, A42, A55 and E26 are concatenated on the feature channel to obtain matrix A56 (1, 640, 40, 40). A56 passes through a convolution layer with kernel 1, stride 1, and padding 0 to obtain matrix A57 (1, 128, 40, 40). ), E39 enters PRAM and passes through four average pooling layers to obtain matrices A58 (1, 256, 1, 1), A59 (1, 256, 2, 2), A60 (1, 256, 3, 3) and A61 (1, 256, 6, 6), A58 passes through a convolution layer with kernel 1, stride 1, and padding 0 to obtain matrix A62 (1, 256, 1, 1), A62 passes through bilinear interpolation to obtain matrix A63 (1, 256, 20, 20), A59 enters SimNL and passes through convolution layers with kernel 1, stride 1, and padding 0 to obtain matrix A64 (1, 128, 2, 2), A65 (1, 128, 2, 2) and A66 (1, 128, 2, 2), A64 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A67 (1, 1, 2, 2) and matrix A68 (1, 1, 2, 2), A67 and A68 are concatenated and adjusted on the feature channel dimension to obtain matrix A69 (1, 4, 2), A65 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A70 (1, 1, 2, 2) and matrix A71 (1, 1, 2, 2), matrix A70 and matrix A71 are concatenated and adjusted on the feature channel dimension to obtain matrix A72 (1, 2, 4), A69 and A72 perform matrix multiplication and pass through a Softmax activation function layer to obtain To A73 (1, 4, 4), A66 is adjusted to (1, 4, 128) and matrix multiplication is performed with A73 and adjusted to obtain matrix A74 (1, 128, 2, 2). A74 passes through a convolution layer with kernel 1, stride 1, and padding 0, and a BN layer to obtain matrix A75 (1, 256, 2, 2). A75 passes through bilinear interpolation to obtain matrix A76 (1, 256, 20, 20). A60 enters SimNL and passes through convolution layers with kernel 1, stride 1, and padding 0 to obtain matrices A77 (1, 128, 3, 3), A78 (1, 128, 3, 3) and A79 (1, 128, 3, 3).A77 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A80 (1, 1, 3, 3) and matrix A81 (1, 1, 3, 3), A80 and A81 are concatenated and adjusted on the feature channel dimension to obtain matrix A82 (1, 9, 2), A78 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A83 (1, 1, 3, 3) and matrix A84 (1, 1, 3, 3), A83 and A84 are concatenated and adjusted on the feature channel dimension to obtain matrix A85 (1, 2, 9), A82 and A85 perform matrix multiplication and pass through a Softmax activation function layer to obtain matrix A86 (1, 9, 9), and A79 is adjusted to (1, 9, 128) and performs matrix multiplication with A86 and adjusts it to obtain matrix A87 (1, 128, 3, 3). A87 passes through a convolution layer with kernel 1, stride 1, and padding 0, and a BN layer to obtain matrix A88 (1, 256, 3, 3). A88 passes through bilinear interpolation to obtain matrix A89 (1, 256, 20, 20). A61 enters SimNL and passes through convolution layers with kernel 1, stride 1, and padding 0 to obtain matrices A90 (1, 128, 6, 6), A91 (1, 128, 6, 6) and A92 (1, 128, 6, 6). A90 is respectively multiplied on the feature channel dimension. Global average pooling and global maximum pooling are performed to obtain matrix A93 (1, 1, 6, 6) and matrix A94 (1, 1, 6, 6). A93 and A94 are concatenated and adjusted in the feature channel dimension to obtain matrix A95 (1, 36, 2). A91 performs global average pooling and global maximum pooling in the feature channel dimension to obtain matrix A96 (1, 1, 6, 6) and matrix A97 (1, 1, 6, 6). A96 and A97 are concatenated and adjusted in the feature channel dimension to obtain matrix A98 (1, 2, 36). A95 and A98 are matrix multiplied and passed through a Softmax activation function layer to obtain matrix A99 (1, 36, 36). A92 is adjusted to (1, 36, 128) and Matrix A100 (1, 128, 6, 6) is obtained by matrix multiplication and adjustment with A99. A100 passes through a convolution layer with kernel 1, stride 1, and padding 0, and a BN layer to obtain matrix A101 (1, 256, 6, 6). A101 is bilinearly interpolated to obtain matrix A102 (1, 256, 20, 20). A63, A76, A89, A102 and E39 are concatenated on the feature channel to obtain matrix A103 (1, 1280, 20, 20). A103 passes through a convolution layer with kernel 1, stride 1, and padding 0 to obtain matrix A104 (1, 256, 20, 20).

[0070] Finally, matrices A5, A10, A57, A104 and E49 enter the decoder, E49 enters the Upsample Block (256), and after bilinear interpolation, the matrix D1 (1, 512, 20, 20) is obtained. D1 passes through a convolution layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, and a ReLU activation function layer to obtain the matrix D2 (1, 256, 20, 20). A104 and D2 are concatenated in the feature channel dimension to obtain the matrix D3 (1, 512, 20, 20), and D3 enters the Upconv Block(256), after a convolutional layer with kernel 3, stride 1, padding 1, a BN layer and a ReLU activation function layer, the matrix D4(1, 256, 20, 20) is obtained. D4 passes through a convolutional layer with kernel 3, stride 1, padding 1, a BN layer and a ReLU activation function layer to obtain the matrix D5(1, 256, 20, 20). D5 enters Upsample Block(128), and after bilinear interpolation, the matrix D6(1, 256, 40, 40) is obtained. D6 passes through a convolutional layer with kernel 3, stride 1, padding 1, a BN layer and a ReLU activation function layer to obtain the matrix D7(1, 128, 40, 40). A57 and D7 are concatenated in the feature channel dimension to obtain the matrix D8(1, 256, 40, 40). D8 enters Upconv Block (128), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, and a ReLU activation function layer, the matrix D9 (1, 128, 40, 40) is obtained. D9 passes through a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, and a ReLU activation function layer to obtain the matrix D10 (1, 128, 40, 40). D10 enters Upsample Block (64), and after bilinear interpolation, the matrix D11 (1, 128, 80, 80) is obtained. D11 passes through a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, and a ReLU activation function layer to obtain the matrix D12 (1, 64, 80, 80). A10 and D12 are concatenated in the feature channel dimension to obtain D13 (1, 128, 80, 80). D13 enters Upconv Block (64), after a convolutional layer with kernel 3, stride 1, padding 1, a BN layer and a ReLU activation function layer, the matrix D14 (1, 64, 80, 80) is obtained.D14 passes through a convolution layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, and a ReLU activation function layer to obtain a matrix D15 (1, 64, 80, 80). D15 enters the Upsample Block (64) and obtains a matrix D16 (1, 64, 160, 160) through bilinear interpolation. D16 passes through a convolution layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, and a ReLU activation function layer to obtain a matrix D17 (1, 64, 160, 160). A5 and D17 are concatenated in the feature channel dimension to obtain a matrix D18 (1, 128, 160, 160). D18 enters the Upconv Block (64), after a convolution layer with kernel 3, stride 1, padding 1, a BN layer and a ReLU activation function layer, the matrix D19 (1, 64, 160, 160) is obtained. D19 passes through a convolution layer with kernel 3, stride 1, padding 1, a BN layer and a ReLU activation function layer to obtain the matrix D20 (1, 64, 160, 160). D20 enters Upconvlast Block (2), and after bilinear interpolation, the matrix D21 (1, 64, 320, 320) is obtained. D21 passes through a convolution layer with kernel 1, stride 1, padding 0 to obtain the final network output Y (1, 2, 320, 320), where 2 represents the category of each pixel in the image, i.e. crack and non-crack. ,

[0071] S300, using the training set and the verification set to train and verify the pavement crack segmentation model, and optimizing the training rate and effect by adding auxiliary branches;

[0072] It should be noted that, in one embodiment, the model training and verification steps include: (1) based on the training set images and corresponding labels, the labeled images are input into the network model for training, and the pixel-level segmentation results of the pavement cracks are output; (2) the main network loss and the auxiliary branch loss are calculated; (3) the network parameters are optimized according to the calculated network loss; (4) based on the verification set images and corresponding labels, the labeled images are input into the network model for verification, the evaluation index F1 value is calculated, and the optimal weight is saved and updated.

[0073] In one embodiment, the optimizer of the model uses the Adam optimizer, which involves calculating the gradient of 8 samples per iteration to update the weights and applying L2 regularization with a weight decay factor of 0.00001. Considering that the decaying learning rate helps network training, a dynamic learning rate correction strategy based on model performance is adopted. The model is verified on the validation set every 60 weight updates, and the evaluation index F1 value is calculated. When no better F1 value is obtained for three consecutive times, the learning rate is reduced by 90%, and the initial learning rate is 0.0001.

[0074] In one embodiment, the main network loss consists of a cross entropy loss and a Dice coefficient loss:

[0075]

[0076] where p ic Indicates the true label that the i-th pixel belongs to category c; q ic Indicates the main network output that the i-th pixel belongs to category c; n represents the total number of pixels in the image; k represents the category and is 2;

[0077] The auxiliary branch loss consists of the Dice coefficient loss:

[0078]

[0079] where p ic Indicates the true label that the i-th pixel belongs to category c; z ic Indicates that the i-th pixel belongs to the auxiliary branch output of category c; n represents the total number of pixels in the image; k represents the category and is 2.

[0080] It should be further explained that, in one embodiment, the auxiliary branch only participates in auxiliary training and is cancelled in the test phase and actual prediction. The intermediate features of the encoder are upsampled by the auxiliary branch to obtain the final segmentation result. The specific network structure is as follows: Figure 4 shown.

[0081] In one embodiment, the size of the input image is (320, 320), and the forward propagation process of the auxiliary branch is as follows:

[0082] The input three-channel image is adjusted to the matrix X(1, 3, 320, 320), where 1 represents the number of images input into the network at one time; 3 represents the number of feature channels of the matrix; the first 320 represents the height of the matrix; and the second 320 represents the width of the matrix.

[0083] The input matrix X is forward propagated according to the main network to obtain the matrix E26 (1, 128, 40, 40). E26 is bilinearly interpolated to obtain F1 (1, 128, 160, 160). F1 is passed through a convolution layer with kernel 1, stride 1, and padding 0 to obtain F2 (1, 64, 160, 160). F2 is bilinearly interpolated to obtain F3 (1, 64, 320, 320). F3 is passed through a convolution layer with kernel 1, stride 1, and padding 0 to obtain the auxiliary branch final output Y* (1, 2, 320, 320), where 2 represents the category of each pixel in the image, i.e., crack and non-crack.

[0084] S400, using the trained crack segmentation model to perform crack recognition on the test set image to verify the generalization ability of the model;

[0085] It should be noted that, in one embodiment, the trained crack segmentation model is the model with the best performance on the validation set, and the auxiliary branch is removed during the test.

[0086] S500, using the trained crack segmentation model to perform crack recognition on the road surface image to be detected, and outputting a pixel-level crack detection result;

[0087] It should be noted that, in one embodiment, when the length or width of the image to be tested is less than 320 pixels, the image will be scaled before entering the model so that the length and width of the image meet two conditions: (1) greater than or equal to 320 pixels; (2) a multiple of 32.

[0088] S600, extracting crack skeletons and quantifying lengths of the pixel-level crack detection results, and calculating crack damage rates.

[0089] It should be noted that, in one embodiment, S600 further includes: (1) extracting the crack skeleton in the crack segmentation image using the Hilditch algorithm; (2) calculating the pixel length L of the crack skeleton c ; (3) Calculate the conversion coefficient α between image pixels and actual size by measuring the size of the actual object and its corresponding pixel value in the image multiple times; (4) Calculate the actual crack length; (5) Calculate the crack damage rate DR according to the Highway Technical Condition Assessment Standard (JTG 5210-2018)

[0090] It should be further explained that, in one embodiment, for a crack skeleton with a single pixel width, the length of the crack can be expressed as the sum of the lengths of each small skeleton. The specific steps include: (1) searching from the upper left corner to the lower right corner of the image, recording the starting point and adjacent end points of the first unrecorded crack skeleton and calculating their Euclidean distance; (2) repeating step 1 until all crack skeletons are calculated; (3) summing the lengths of all skeletons.

[0091] In one embodiment, the actual total length of the crack L cr for:

[0092] L cr =αL c #(6)

[0093] The cumulative area of ​​cracks A cr for:

[0094] A cr =L cr ×0.2#(7)

[0095] The road detection area is the actual area of ​​the detection image, which is:

[0096] A=αHαW#(8)

[0097] Where H and W are the pixel height and width of the detection image, respectively. Substitute A cr By adding A to formula 9, the damage rate DR of pavement cracks can be obtained.

[0098]

[0099] Among them, w i is the conversion factor of the automated detection of the i-th type of pavement damage. For the sake of uniformity, the conversion factor of the automated detection of cracks is taken as 2.0; i represents the type of pavement damage; i0 represents the total number of pavement diseases in the detection area; A i is the cumulative area of ​​the i-th type of pavement damage (m 2 ); A is the road surface detection area (m 2 ).

[0100] further, Figure 5Some prediction results of the proposed method and the existing semantic segmentation model on the CFD dataset are given. The results show that, considering the allowable error of 2 pixels, the detection results of FCNs, SegNet, U-Net and DeepLabv3+ all have false positive points far away from the true label, while PSPNet and the proposed method do not have this problem. In addition, since the proposed method makes full use of global multi-scale information and introduces SimNL to capture long-distance dependencies, its detection results are smoother and more detailed than other methods. Compared with DeepLabv3+, the proposed method is slightly inferior in the recall of mesh cracks, but has more advantages in accuracy. In general, regardless of lighting conditions, shadows, road markings and other noises, the proposed method can accurately extract most of the crack pixels. The specific quantitative results are shown in Table 1. Among these methods, the proposed method achieves the best precision and F1 value. In terms of inference speed, the proposed method only takes 25.89ms to process an image with a resolution of 480×320, which is also better in practical applications.

[0101] Table 1 Performance comparison of the proposed method and existing semantic segmentation models on CFD datasets

[0102]

[0103] Table 2 shows the performance comparison results of the method of the present invention and the existing crack segmentation model on the CFD data set. It can be seen that under the same allowable error, the method of the present invention has a higher F1 value and better performance than other crack segmentation algorithms.

[0104] Table 2 Performance comparison of the proposed method and existing crack segmentation models on CFD data sets

[0105]

[0106] Furthermore, in order to verify the effectiveness and superiority of the Hilditch algorithm for crack skeleton extraction, Figure 6 The skeleton extraction effects of the currently more commonly used Zhang-Suen algorithm and Hilditch algorithm for some crack segmentation results are compared. It can be seen that compared with the Zhang-Suen algorithm, the skeleton extracted by the Hilditch algorithm is not only closer to the real crack in direction, but also has no burrs and has a better effect. Figure 7 The detailed information of the skeleton after magnification is further displayed. It is not difficult to find that the width of the skeleton extracted by the Zhang-Suen algorithm cannot be guaranteed to be a single pixel, while the skeleton extracted by the Hilditch algorithm does not have this problem.

[0107] further, Figure 8The method proposed in the present invention is used to demonstrate crack segmentation and quantification of some larger-sized actual pavement crack images. Based on the estimated true length of the pavement cracks, combined with the image size and the automated detection conversion factor, the crack damage rate can be calculated using Formula 9, providing an important basis for evaluating road damage conditions and making maintenance decisions.

[0108] For those skilled in the art, the above embodiments and descriptions are only for illustrating the principles and optimal embodiments of the present invention. Some improvements may be made thereto without departing from the scope of the present invention, but these improvements should be within the scope of protection of the claims attached to the present invention.

Claims

1. A road crack segmentation and quantification method based on pyramid region attention, characterized in that: include: S1. Collect pavement crack images, make data sets and perform data enhancement processing; S2. Based on the U-Net network structure, ResNet34 is used in combination with BAM to replace its encoder part, and the spatial attention module SA and the pyramid regional attention module PRAM are introduced into its jump connection part to construct a pavement crack segmentation model; S3, using the training set and the verification set to train and verify the pavement crack segmentation model, and optimizing the training rate and effect by adding auxiliary branches; S4, using the trained crack segmentation model to perform crack recognition on the test set images to verify the generalization ability of the model; S5, using the trained crack segmentation model to identify cracks in the road surface image to be detected, and outputting pixel-level crack detection results; S6. Extract the crack skeleton and quantify the length of the pixel-level crack detection result obtained in S5, and calculate the crack damage rate.

2. The method according to claim 1, characterized in that S1 further includes: S11, using the CFD dataset as the original dataset, which contains pavement cracks of different types, scenes and sizes, and the image size is 480×320; S12, dividing the original data set in S11 into a training set, a validation set and a test set according to a certain ratio; S13, the training set, validation set and test set obtained in S12 are respectively cropped into patches of size 320×320, with a cropping step of 20 pixels; S14, the patch image obtained in S13 is rotated 90° counterclockwise, 180° counterclockwise, 270° counterclockwise, flipped horizontally, and flipped vertically, and added to the corresponding training set, validation set, and test set.

3. The method according to claim 1, characterized in that The pavement crack segmentation model described in S2 is established based on the U-Net algorithm, and includes three parts: an encoder, a skip connection, and a decoder. The encoder extracts features of the input image and performs downsampling processing to obtain image features of different downsampling scales, including: S21. When the input image size is (320, 320), the input three-channel image is adjusted to the matrix X(1, 3, 320, 320), where 1 represents the number of images input into the network at one time; 3 represents the number of feature channels of the matrix; the first 320 represents the height of the matrix; and the second 320 represents the width of the matrix; S22, the input matrix X first enters the ResNet34+BAM encoder, passes through a convolution layer with a kernel of 7, a stride of 2, and a padding of 3, a BN layer, and a ReLU activation function layer to obtain the matrix E1 (1, 64, 160, 160), E1 passes through a maximum pooling layer with a kernel of 3, a stride of 2, and a padding of 1 to obtain the matrix E2 (1, 64, 80, 80), E2 enters the Residual Block (64, 1), passes through a convolution layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolution layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer to obtain the matrix E3 (1, 64, 80, 80), E2 and E3 are element-wise added to obtain E4 (1, 64, 80, 80), E4 enters the ResidualBlock (64, 1), passes through a kernel of 3, s A convolutional layer with a tride of 1 and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1 and a padding of 1, and a BN layer are used to obtain the matrix E5 (1, 64, 80, 80). E4 and E5 are element-wise added to obtain the matrix E6 (1, 64, 80, 80). E6 enters the ResidualBlock (64, 1), passes through a convolutional layer with a kernel of 3, a stride of 1 and a padding of 1, and a BN layer. N layers, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer are used to obtain the matrix E7 (1, 64, 80, 80). E6 and E7 are element-wise added to obtain the matrix E8 (1, 64, 80, 80). E8 enters BAM, passes through a global average pooling layer, and a fully connected layer to obtain the matrix E9 (1, 4, 1, 1). E9 passes through a fully connected layer to obtain the matrix E10 (1, 64, 1, 1). At the same time, E8 passes through a kernel of 1, s The convolution layer with tride of 1 and padding of 0, a BN layer and a ReLU activation function layer obtain the matrix E11 (1, 4, 80, 80). E11 passes through a dilated convolution layer with a kernel of 3, a stride of 1, a padding of 4, and a dilation rate of 4, a BN layer, a ReLU activation function layer, a dilated convolution layer with a kernel of 3, a stride of 1, a padding of 4, and a dilation rate of 4, a BN layer and a ReLU activation function layer to obtain the matrix E12 (1, 4, 80, 80).E12 passes through a convolution layer with a kernel of 1, a stride of 1, and a padding of 0 to obtain the matrix E13 (1, 1, 80, 80). E10 and E13 are expanded to (1, 64, 80, 80) and element-wise added. After passing through a Sigmoid activation function layer, the matrix E14 (1, 64, 80, 80) is obtained. E14 and E8 are element-wise multiplied and then element-wise added with E8 to obtain the matrix E15 (1, 64, 80, 80). E15 enters the Residual Block (128, 2), passes through a convolution layer with a kernel of 3, a stride of 2, and a padding of 1, a BN layer, a ReLU activation function layer, a convolution layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer to obtain the matrix E16 (1, 128, 40, 40). E16 enters the Residual Block (128, 2). Block(128, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E17(1, 128, 40, 40) is obtained. E17 enters the Residual Block(128, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E18(1, 128, 40, 40) is obtained. E18 enters the Residual Block (128, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E19 (1, 128, 40, 40) is obtained. E19 enters BAM, after a global average pooling layer and a fully connected layer, the matrix E20 (1, 8, 1, 1) is obtained. E20 passes through a fully connected layer to obtain the matrix E21 (1, 128, 1, 1). At the same time, E19 passes through a convolutional layer with a kernel of 1, a stride of 1, and a padding of 0, a BN layer, and a ReLU activation function layer to obtain the matrix E22 (1, 8, 40, 40).E22 passes through a dilated convolution layer with a kernel of 3, a stride of 1, a padding of 4, and a dilation rate of 4, a BN layer, a ReLU activation function layer, a dilated convolution layer with a kernel of 3, a stride of 1, a padding of 4, and a dilation rate of 4, a BN layer, and a ReLU activation function layer to obtain the matrix E23 (1, 8, 40, 40). E23 passes through a convolution layer with a kernel of 1, a stride of 1, and a padding of 0 to obtain the matrix E24 (1, 1, 40, 40). E21 and E24 are expanded to (1, 128, 40, 40) and element-wise added. After passing through a Sigmoid activation function layer, the matrix E25 (1, 128, 40, 40) is obtained. E25 is element-wise multiplied with E19 and then element-wise added with E19 to obtain the matrix E26 (1, 128, 40, 40). E26 enters the Residual Block(256, 2), after a convolutional layer with a kernel of 3, a stride of 2, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E27(1, 256, 20, 20) is obtained. E27 enters the Residual Block(256, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E28(1, 256, 20, 20) is obtained. E28 enters the Residual Block(256, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E29(1, 256, 20, 20) is obtained. E29 enters the Residual Block(256, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E30(1, 256, 20, 20) is obtained. E30 enters the Residual Block(256, 1),After a convolutional layer with kernel 3, stride 1, and padding 1, a BN layer, a ReLU activation function layer, a convolutional layer with kernel 3, stride 1, and padding 1, and a BN layer, the matrix E31 (1, 256, 20, 20) is obtained. E31 enters the Residual Block(256, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E32(1, 256, 20, 20) is obtained. E32 enters BAM, and after a global average pooling layer and a fully connected layer, the matrix E33(1, 16, 1, 1) is obtained. E33 passes through a fully connected layer to obtain the matrix E34(1, 256, 1, 1). At the same time, E32 passes through a convolutional layer with a kernel of 1, a stride of 1, and a padding of 0, a BN layer, and a ReLU activation function layer to obtain the matrix E35(1, 16, 20, 20). E35 passes through a convolutional layer with a kernel of 3, a stride of 1, a padding of 4, and a dilation rate of 4, a BN layer, a ReLU activation function layer, a kernel of 3, a stride of 1, a padding of 4, a dilated convolution layer with a dilated rate of 4, a BN layer and a ReLU activation function layer to obtain the matrix E36 (1, 16, 20, 20). E36 passes through a convolution layer with a kernel of 1, a stride of 1, and a padding of 0 to obtain the matrix E37 (1, 1, 20, 20). E34 and E37 are expanded to (1, 256, 20, 20) and element-wise added. After passing through a Sigmoid activation function layer, the matrix E38 (1, 256, 20, 20) is obtained. E38 is element-wise multiplied with E32 and then element-wise added with E32 to obtain the matrix E39 (1, 256, 20, 20). E39 enters the Residual Block (512, 2), after a convolutional layer with kernel 3, stride 2, padding 1, a BN layer, a ReLU activation function layer, a convolutional layer with kernel 3, stride 1, padding 1, and a BN layer, the matrix E40 (1, 512, 10, 10) is obtained. E40 enters Residual Block (512, 1).After a convolutional layer with kernel 3, stride 1, and padding 1, a BN layer, a ReLU activation function layer, a convolutional layer with kernel 3, stride 1, and padding 1, and a BN layer, the matrix E41 (1, 512, 10, 10) is obtained. E41 enters the Residual Block(512, 1), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, a ReLU activation function layer, a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, and a BN layer, the matrix E42(1, 512, 10, 10) is obtained. E42 enters BAM, and after a global average pooling layer and a fully connected layer, the matrix E43(1, 32, 1, 1) is obtained. E43 passes through a fully connected layer to obtain the matrix E44(1, 512, 1, 1). At the same time, E42 passes through a convolutional layer with a kernel of 1, a stride of 1, and a padding of 0, a BN layer, and a ReLU activation function layer to obtain the matrix E45(1, 32, 10, 10). E45 passes through a convolutional layer with a kernel of 3, a stride of 1, and a padding of 4. A dilated convolution layer with a dilation rate of 4, a BN layer, a ReLU activation function layer, a dilated convolution layer with a kernel of 3, a stride of 1, a padding of 4, and a dilation rate of 4, a BN layer, and a ReLU activation function layer are used to obtain the matrix E46 (1, 32, 10, 10). E46 passes through a convolution layer with a kernel of 1, a stride of 1, and a padding of 0 to obtain the matrix E47 (1, 1, 10, 10). E44 and E47 are expanded to (1, 512, 10, 10) and element-wise added. After passing through a Sigmoid activation function layer, the matrix E48 (1, 512, 10, 10) is obtained. E48 is element-wise multiplied with E42 and then element-wise added with E42 to obtain the matrix E49 (1, 512, 10, 10). S23, matrices E1, E15, E26, E39 and E49 are collectively the outputs of the encoder, wherein E1, E15, E26, E39 are input to the skip connection part, and E49 is input to the decoder.

4. The method according to claim 1, characterized in that: The pavement crack segmentation model described in S2 is established based on the U-Net algorithm, and includes three parts: an encoder, a jump connection, and a decoder. The output of the encoder is further enhanced by the jump connection part to obtain the steps of refined image features, including: S24, matrix E1 enters SA, and global average pooling and global maximum pooling are performed on the feature channel dimension to obtain matrix A1 (1, 1, 160, 160) and matrix A2 (1, 1, 160, 160). A1 and A2 are concatenated on the feature channel dimension to obtain matrix A3 (1, 2, 160, 160). A3 passes through a convolution layer with a kernel of 7, a stride of 1, and a padding of 3 and a Sigmoid activation function layer to obtain matrix A4 (1, 1, 160, 160). A4 is expanded to (1, 64, 160, 160) and element-wise multiplied with E1, and then element-wise added with E1. The matrix A5 (1, 64, 160, 160) is obtained, E15 enters SA, and global average pooling and global maximum pooling are performed on the feature channel dimension to obtain matrix A6 (1, 1, 80, 80) and matrix A7 (1, 1, 80, 80). A6 and A7 are concatenated on the feature channel dimension to obtain matrix A8 (1, 2, 80, 80). A8 passes through a convolution layer with a kernel of 7, a stride of 1, and a padding of 3 and a Sigmoid activation function layer to obtain matrix A9 (1, 1, 80, 80). A9 is expanded to (1, 64, 80, 80) and element-wise multiplied with E15, and then elem ent-wise sum to obtain the matrix A10 (1, 64, 80, 80), E26 enters PRAM, and passes through four average pooling layers to obtain the matrices A11 (1, 128, 1, 1), A12 (1, 128, 2, 2), A13 (1, 128, 3, 3) and A14 (1, 128, 6, 6), A11 passes through a convolution layer with kernel 1, stride 1, and padding 0 to obtain the matrix A15 (1, 128, 1, 1), A15 passes through bilinear interpolation to obtain the matrix A16 (1, 128, 40, 40), A12 enters SimNL, and passes through kernel 1, stride 1, and padding 0 respectively. The convolution layer of 0 obtains matrices A17 (1, 64, 2, 2), A18 (1, 64, 2, 2) and A19 (1, 64, 2, 2). A17 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrices A20 (1, 1, 2, 2) and matrix A21 (1, 1, 2, 2). A20 and A21 are concatenated and adjusted on the feature channel dimension to obtain matrix A22 (1, 4, 2). A18 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A23 (1, 1, 2, 2) and matrix A24 (1, 1, 2, 2). A23 and A24 are concatenated and adjusted on the feature channel dimension to obtain matrix A25 (1, 2, 4).A22 and A25 perform matrix multiplication and pass through a Softmax activation function layer to obtain matrix A26 (1, 4, 4). A19 is adjusted to (1, 4, 64) and matrix multiplied with A26 and adjusted to obtain matrix A27 (1, 64, 2, 2). A27 passes through a convolution layer with kernel 1, stride 1, and padding 0, and a BN layer to obtain matrix A28 (1, 128, 2, 2). A28 passes through bilinear interpolation to obtain matrix A29 (1, 128, 40, 40). A13 enters SimNL and passes through convolution layers with kernel 1, stride 1, and padding 0 to obtain matrix A30 (1, 64, 3, 3 ), A31 (1, 64, 3, 3) and A32 (1, 64, 3, 3), A30 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A33 (1, 1, 3, 3) and matrix A34 (1, 1, 3, 3), A33 and A34 are concatenated and adjusted on the feature channel dimension to obtain matrix A35 (1, 9, 2), A31 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A36 (1, 1, 3, 3) and matrix A37 (1, 1, 3, 3), A36 and A37 are concatenated and adjusted on the feature channel dimension to obtain matrix A38 (1, 2, 9), A35 and A38 perform matrix multiplication and pass a Softma The x activation function layer obtains the matrix A39 (1, 9, 9), adjusts A32 to (1, 9, 64) and performs matrix multiplication with A39 and adjusts it to obtain the matrix A40 (1, 64, 3, 3), A40 passes through a convolution layer with a kernel of 1, a stride of 1, and a padding of 0, and a BN layer to obtain the matrix A41 (1, 128, 3, 3), A41 passes through bilinear interpolation to obtain the matrix A42 (1, 128, 40, 40), A14 enters SimNL, and passes through convolution layers with a kernel of 1, a stride of 1, and a padding of 0 to obtain the matrices A43 (1, 64, 6, 6), A44 (1, 64, 6, 6) and A45 (1, 64 , 6, 6), A43 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A46 (1, 1, 6, 6) and matrix A47 (1, 1, 6, 6), A46 and A47 are concatenated and adjusted on the feature channel dimension to obtain matrix A48 (1, 36, 2), A44 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A49 (1, 1, 6, 6) and matrix A50 (1, 1, 6, 6), A49 and A50 are concatenated and adjusted on the feature channel dimension to obtain matrix A51 (1, 2, 36), A48 and A51 perform matrix multiplication and pass through a Softmax activation function layer to obtain matrix A52 (1, 36, 36),A45 is adjusted to (1, 36, 64) and matrix multiplication is performed with A52 to obtain matrix A53 (1, 64, 6, 6). A53 passes through a convolution layer with a kernel of 1, a stride of 1, and a padding of 0, and a BN layer to obtain matrix A54 (1, 128, 6, 6). A54 is subjected to bilinear interpolation to obtain matrix A55 (1, 128, 40, 40). A16, A29, A42, A55 and E26 are concatenated on the feature channel to obtain matrix A56 (1, 640, 40, 40). A56 passes through a convolution layer with a kernel of 1, a stride of 1, and a padding of 0 to obtain matrix A57 (1, 128, 40, 40). ), E39 enters PRAM and passes through four average pooling layers to obtain matrices A58 (1, 256, 1, 1), A59 (1, 256, 2, 2), A60 (1, 256, 3, 3) and A61 (1, 256, 6, 6), A58 passes through a convolution layer with kernel 1, stride 1, and padding 0 to obtain matrix A62 (1, 256, 1, 1), A62 passes through bilinear interpolation to obtain matrix A63 (1, 256, 20, 20), A59 enters SimNL and passes through convolution layers with kernel 1, stride 1, and padding 0 to obtain matrix A64 (1, 128, 2, 2), A65 (1, 128, 2, 2) and A66 (1, 128, 2, 2), A64 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A67 (1, 1, 2, 2) and matrix A68 (1, 1, 2, 2), A67 and A68 are concatenated and adjusted on the feature channel dimension to obtain matrix A69 (1, 4, 2), A65 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A70 (1, 1, 2, 2) and matrix A71 (1, 1, 2, 2), matrix A70 and matrix A71 are concatenated and adjusted on the feature channel dimension to obtain matrix A72 (1, 2, 4), A69 and A72 perform matrix multiplication and pass through a Softmax activation function layer to obtain To A73 (1, 4, 4), A66 is adjusted to (1, 4, 128) and matrix multiplication is performed with A73 and adjusted to obtain matrix A74 (1, 128, 2, 2). A74 passes through a convolution layer with kernel 1, stride 1, and padding 0, and a BN layer to obtain matrix A75 (1, 256, 2, 2). A75 passes through bilinear interpolation to obtain matrix A76 (1, 256, 20, 20). A60 enters SimNL and passes through convolution layers with kernel 1, stride 1, and padding 0 to obtain matrices A77 (1, 128, 3, 3), A78 (1, 128, 3, 3), and A79 (1, 128, 3, 3).A77 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A80 (1, 1, 3, 3) and matrix A81 (1, 1, 3, 3), A80 and A81 are concatenated and adjusted on the feature channel dimension to obtain matrix A82 (1, 9, 2), A78 performs global average pooling and global maximum pooling on the feature channel dimension to obtain matrix A83 (1, 1, 3, 3) and matrix A84 (1, 1, 3, 3), A83 and A84 are concatenated and adjusted on the feature channel dimension to obtain matrix A85 (1, 2, 9), A82 and A85 perform matrix multiplication and pass through a Softmax activation function layer to obtain matrix A86 (1, 9, 9), and A79 is adjusted to (1, 9, 128) and performs matrix multiplication with A86 and adjusts it to obtain matrix A87 (1, 128, 3, 3). A87 passes through a convolution layer with kernel 1, stride 1, and padding 0, and a BN layer to obtain matrix A88 (1, 256, 3, 3). A88 passes through bilinear interpolation to obtain matrix A89 (1, 256, 20, 20). A61 enters SimNL and passes through convolution layers with kernel 1, stride 1, and padding 0 to obtain matrices A90 (1, 128, 6, 6), A91 (1, 128, 6, 6) and A92 (1, 128, 6, 6). A90 is respectively multiplied on the feature channel dimension. Global average pooling and global maximum pooling are performed to obtain matrix A93 (1, 1, 6, 6) and matrix A94 (1, 1, 6, 6). A93 and A94 are concatenated and adjusted in the feature channel dimension to obtain matrix A95 (1, 36, 2). A91 performs global average pooling and global maximum pooling in the feature channel dimension to obtain matrix A96 (1, 1, 6, 6) and matrix A97 (1, 1, 6, 6). A96 and A97 are concatenated and adjusted in the feature channel dimension to obtain matrix A98 (1, 2, 36). A95 and A98 are matrix multiplied and passed through a Softmax activation function layer to obtain matrix A99 (1, 36, 36). A92 is adjusted to (1, 36, 128) and Matrix A100 (1, 128, 6, 6) is obtained by matrix multiplication and adjustment with A99. A100 passes through a convolution layer with kernel 1, stride 1, and padding 0, and a BN layer to obtain matrix A101 (1, 256, 6, 6). A101 is bilinearly interpolated to obtain matrix A102 (1, 256, 20, 20). A63, A76, A89, A102 and E39 are concatenated on the feature channel to obtain matrix A103 (1, 1280, 20, 20). A103 passes through a convolution layer with kernel 1, stride 1, and padding 0 to obtain matrix A104 (1, 256, 20, 20). S25, matrices A5, A10, A57, and A104 are collectively input into the decoder as the output of the skip connection part.

5. The method according to claim 1, characterized in that The pavement crack segmentation model described in S2 is established based on the U-Net algorithm, and includes three parts: an encoder, a jump connection, and a decoder. The decoder integrates and upsamples the outputs of the encoder and the jump connection parts to obtain the final segmentation result, including: S26, matrices A5, A10, A57, A104 and E49 enter the decoder, E49 enters the Upsample Block (256), and after bilinear interpolation, the matrix D1 (1, 512, 20, 20) is obtained. D1 passes through a convolution layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, and a ReLU activation function layer to obtain the matrix D2 (1, 256, 20, 20). A104 and D2 are concatenated in the feature channel dimension to obtain the matrix D3 (1, 512, 20, 20), and D3 enters the Upconv Block(256), after a convolutional layer with kernel 3, stride 1, padding 1, a BN layer and a ReLU activation function layer, the matrix D4(1, 256, 20, 20) is obtained. D4 passes through a convolutional layer with kernel 3, stride 1, padding 1, a BN layer and a ReLU activation function layer to obtain the matrix D5(1, 256, 20, 20). D5 enters Upsample Block(128), and after bilinear interpolation, the matrix D6(1, 256, 40, 40) is obtained. D6 passes through a convolutional layer with kernel 3, stride 1, padding 1, a BN layer and a ReLU activation function layer to obtain the matrix D7(1, 128, 40, 40). A57 and D7 are concatenated in the feature channel dimension to obtain the matrix D8(1, 256, 40, 40). D8 enters Upconv Block (128), after a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, and a ReLU activation function layer, the matrix D9 (1, 128, 40, 40) is obtained. D9 passes through a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, and a ReLU activation function layer to obtain the matrix D10 (1, 128, 40, 40). D10 enters Upsample Block (64), and after bilinear interpolation, the matrix D11 (1, 128, 80, 80) is obtained. D11 passes through a convolutional layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, and a ReLU activation function layer to obtain the matrix D12 (1, 64, 80, 80). A10 and D12 are concatenated in the feature channel dimension to obtain D13 (1, 128, 80, 80). D13 enters Upconv Block (64), after a convolutional layer with kernel 3, stride 1, padding 1, a BN layer and a ReLU activation function layer, the matrix D14 (1, 64, 80, 80) is obtained.D14 passes through a convolution layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, and a ReLU activation function layer to obtain a matrix D15 (1, 64, 80, 80). D15 enters the Upsample Block (64) and obtains a matrix D16 (1, 64, 160, 160) through bilinear interpolation. D16 passes through a convolution layer with a kernel of 3, a stride of 1, and a padding of 1, a BN layer, and a ReLU activation function layer to obtain a matrix D17 (1, 64, 160, 160). A5 and D17 are concatenated in the feature channel dimension to obtain a matrix D18 (1, 128, 160, 160). D18 enters the Upconv Block (64), after a convolution layer with kernel 3, stride 1, padding 1, a BN layer and a ReLU activation function layer, the matrix D19 (1, 64, 160, 160) is obtained. D19 passes through a convolution layer with kernel 3, stride 1, padding 1, a BN layer and a ReLU activation function layer to obtain the matrix D20 (1, 64, 160, 160). D20 enters Upconvlast Block (2), and after bilinear interpolation, the matrix D21 (1, 64, 320, 320) is obtained. D21 passes through a convolution layer with kernel 1, stride 1, padding 0 to obtain the final network output Y (1, 2, 320, 320), where 2 represents the category of each pixel in the image, i.e. crack and non-crack. , 6. The method according to claim 1, characterized in that The auxiliary branch described in S3 only participates in auxiliary training and is cancelled in the test phase and actual prediction. The step of upsampling the encoder intermediate feature E26 through the auxiliary branch to obtain the final segmentation result includes: S31, the input three-channel image is adjusted to the matrix X(1, 3, 320, 320), where 1 represents the number of images input into the network at one time; 3 represents the number of feature channels of the matrix; the first 320 represents the height of the matrix; and the second 320 represents the width of the matrix; S32 and matrix E26 are bilinearly interpolated to obtain F1(1, 128, 160, 160), F1 is passed through a convolution layer with kernel 1, stride 1, and padding 0 to obtain F2(1, 64, 160, 160), F2 is passed through a bilinear interpolation to obtain F3(1, 64, 320, 320), F3 is passed through a convolution layer with kernel 1, stride 1, and padding 0 to obtain the auxiliary branch final output Y*(1, 2, 320, 320), where 2 represents the category of each pixel in the image, i.e., crack and non-crack.

7. The method according to claim 1, characterized in that The model training and verification steps described in S3 further include: S33, based on the training set images and corresponding labels, input the labeled images into the network model for training, and output the pixel-level segmentation results of the road cracks; S34, calculating the main network loss and the auxiliary branch loss; S35, optimizing network parameters according to the network loss calculated in S34; S36. Based on the validation set images and corresponding labels, the labeled images are input into the network model for verification, the evaluation index F1 value is calculated, and the optimal weights are saved and updated.

8. The method according to claim 7, characterized in that The network's loss function includes: S37, the main network loss consists of cross entropy loss and Dice coefficient loss: where p ic Indicates the true label that the i-th pixel belongs to category c; q ic Indicates the main network output that the i-th pixel belongs to category c; n represents the total number of pixels in the image; k represents the category and is 2; S38, auxiliary branch loss is composed of Dice coefficient loss: where p ic Indicates the true label that the i-th pixel belongs to category c; z ic Indicates that the i-th pixel belongs to the auxiliary branch output of category c; n represents the total number of pixels in the image; k represents the category and is 2.

9. The method according to claim 1, characterized in that: S6 further includes: S61, extracting the crack skeleton in the crack segmentation image using the Hilditch algorithm; S62, calculating the pixel length of the crack skeleton described in S61; S63, calculating a conversion coefficient α between image pixels and actual size by measuring the size of the actual photographed object and its corresponding pixel value on the image for multiple times; S64, calculating the true crack length according to the results of S62 and S63; S65. Calculate the crack damage rate DR according to Formula 3. Among them, w i is the conversion coefficient of the automated detection of the i-th type of pavement damage; i represents the type of pavement damage; i0 represents the total number of pavement damages in the detection area; A i is the cumulative area of ​​the i-th type of pavement damage (m 2 ); A is the road surface detection area (m 2 ).

Images