Pavement crack image detection method and device based on MDTA and GDFN

Through the pavement crack detection method based on MDTA and GDFN, the problems of noise sensitivity and low image quality in pavement crack detection are solved, and the pavement crack detection is achieved with higher accuracy. The model parameters are optimized through data enhancement and evaluation indicators, and the detection results are improved accuracy.

CN118351077BActive Publication Date: 2024-11-15WUHAN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410456898.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-16
Publication Date
2024-11-15
Estimated Expiration
2044-04-16

AI Technical Summary

Technical Problem

The existing pavement crack detection algorithm is sensitive to noise, and CNN has data length and width limitations when processing pavement disease images, resulting in inaccurate detection. Especially under the influence of factors such as vehicle vibration, uneven light and shadow, the image quality is low. , affects the accuracy of subsequent crack identification.

Method used

Using the pavement crack image detection method based on MDTA and GDFN, the pavement crack image data set is obtained and marked, and preprocessed such as data augmentation and normalization is performed, and the network model is trained and detected, and evaluation indicators such as CPA, MPA, MIOU and F1 are used. Optimize model parameters until the setting requirements are met to achieve accurate detection of the images to be detected.

Benefits of technology

Effectively remove image noise, improve the accuracy of road crack detection, improve the accuracy and completeness of detection results, and overcome the problems of noise sensitivity and low image quality in the prior art.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118351077B_ABST
    Figure CN118351077B_ABST
Patent Text Reader

Abstract

The present invention discloses a pavement crack image detection method and device based on MDTA and GDFN, and relates to the field of intelligent transportation technology. The method comprises obtaining a pavement crack image data set under different pavement conditions, and annotating the data set according to the pavement background and the crack morphology to obtain a training set, a verification set and a test set; preprocessing the pavement crack images in the training set, and training a pavement crack detection network model based on MDTA and GDFN through the training set; testing the trained pavement crack detection network model through the test set, and calculating an evaluation index according to the test result; based on the calculated evaluation index and when the evaluation index meets the set requirements, using the current pavement crack detection network model to perform crack detection on the image to be detected. The present application can effectively remove image noise, thereby improving the detection accuracy of pavement cracks.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation technology, and in particular to a pavement crack image detection method and device based on MDTA and GDFN. Background Art

[0002] As roads are used for longer and are overloaded, various road diseases occur. Most road diseases first manifest as cracks. The severity of road cracks not only reflects the health of the road, but also plays an important role in the formulation of road maintenance strategies. Therefore, accurate and effective identification and location of road cracks is an important part of intelligent transportation infrastructure.

[0003] At present, the purpose of the pavement crack detection algorithm is to clearly distinguish the cracks from the background in the image. Existing image detection-based technologies are divided into two categories: rule-driven methods and data-driven methods. Rule-driven algorithms mainly include threshold algorithms and edge detection algorithms. However, the use of threshold algorithms requires two conditions to be met: (1) the background fluctuation of the crack image is stable; (2) the crack pixels are more clearly distinguishable than the background pixels. The above two algorithms are sensitive to noise. Therefore, the rule-driven method that is highly sensitive to noise needs to be improved.

[0004] The data-driven crack detection algorithm learns the characteristics of different cracks from the data for classification. Artificial neural networks have been successfully used in pavement crack recognition. Convolutional neural networks (CNNs) have become the mainstream of crack detection. However, CNN has certain limitations on the length and width of data when processing input data, especially for image data and sequence data. In addition, pavement disease images are subject to vehicle vibration, uneven lighting, and shadows during the acquisition process, resulting in low image contrast and low quality. The CNN-based method cannot interact globally, resulting in poor continuity and rough detection. Pavement cracks are characterized by small size, but the image detection model pays less attention to edge pixels and non-edge pixels, resulting in inaccurate detection. The above problems seriously affect subsequent crack recognition accuracy and other issues. Therefore, there are still major limitations in the field of pavement crack detection. Summary of the invention

[0005] The present application provides a pavement crack image detection method and device based on MDTA and GDFN, which can effectively remove image noise, thereby improving the detection accuracy of pavement cracks.

[0006] In a first aspect, an embodiment of the present application provides a pavement crack image detection method based on MDTA and GDFN, and the pavement crack image detection method based on MDTA and GDFN includes:

[0007] Obtaining a data set of pavement crack images under different pavement conditions, and annotating the data set according to the pavement background and the morphology of the cracks to obtain a training set, a validation set, and a test set;

[0008] Preprocess the pavement crack images in the training set, and use the training set to train the pavement crack detection network model based on MDTA and GDFN;

[0009] The trained pavement crack detection network model is tested using the test set, and the evaluation index is calculated based on the test results;

[0010] Based on the calculated evaluation index and when the evaluation index meets the set requirements, the current pavement crack detection network model is used to perform crack detection on the image to be detected.

[0011] In combination with the first aspect, in one implementation,

[0012] The preprocessing includes data enhancement, data normalization, and negative sample addition;

[0013] The evaluation indicators include CPA, MPA, MIOU and F1.

[0014] In combination with the first aspect, in one implementation, the method of performing crack detection on the image to be detected based on the calculated evaluation index and when the evaluation index meets the set requirements, using the current pavement crack detection network model, specifically includes:

[0015] Determine whether the evaluation index calculated after testing the current pavement crack detection network model meets the set requirements:

[0016] If so, the current pavement crack detection network model is used to perform crack detection on the image to be detected;

[0017] If not, the parameters of the pavement crack detection network model are adjusted, the pavement crack detection network model is trained again with the training set, and the evaluation index is calculated again. This cycle is repeated until the calculated evaluation index meets the set requirements, and then the pavement crack detection network model whose evaluation index meets the set requirements is used to perform crack detection on the image to be detected.

[0018] In combination with the first aspect, in one implementation, the using the current pavement crack detection network model to perform crack detection on the image to be detected specifically includes:

[0019] Given an image I with a spatial resolution of H×W×3, a feature map is obtained by convolution with kernel_size=3, stride=1, and padding=1, and the feature map is normalized, dimensionally increased, and dimensionally reduced;

[0020] Repeat the normalization, dimension increase and dimension reduction of the feature map many times;

[0021] The feature maps obtained at different processing stages are connected, and the feature maps obtained after the connection operation are normalized, dimensioned, and reduced;

[0022] Detailed features are extracted from feature maps based on the Refinement module, and images are superimposed through convolution.

[0023] The feature maps of different resolutions are spliced, and the feature maps are upsampled and downsampled to generate the final result map, thereby realizing feature extraction of the image to be detected.

[0024] In combination with the first aspect, in one implementation, the image I with a given spatial resolution of H×W×3 is convolved with kernel_size=3, stride=1, and padding=1 to obtain a feature map, and the feature map is normalized, dimensionally increased, and dimensionally reduced, specifically including:

[0025] Given an image I with a spatial resolution of H×W×3, the feature map I1 is obtained by convolution with kernel_size=3, stride=1, and padding=1;

[0026] The feature map I1 is passed through the normalization layer to normalize the pixels. Then, based on MDTA, the convolution with kernel_size=1 is used to achieve dimension increase and the group convolution with kernel_size=3, stride=1, and padding=1 is used to divide the features into three parts, and Q, V, and K are obtained for self-attention calculation.

[0027] Use convolution with kernel_size=1, change the number of channels, save feature map I2, pass feature map I2 through the normalization layer to achieve pixel normalization, and set the hidden layer feature dimension equal to the input dimension multiplied by the expansion factor;

[0028] Based on GDFN, the features are divided into two parts through convolution with kernel_size=1, out_channel=hidden_features×2 to achieve dimensionality increase and group convolution with kernel_size=3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1, gating with GELU activation function, convolution with kernel_size=1 to achieve dimensionality reduction, changing the number of channels, and saving feature map I3;

[0029] Use convolution with kernel_size=3 to halve the number of channels of feature map I3, and use the PixelUnshuffle layer to increase the channel dimension to 4 times. At the same time, the size of the feature map is reduced to half of the original size, and the feature map I4 is saved.

[0030] In combination with the first aspect, in one implementation, the step of repeatedly normalizing, increasing dimension, and reducing dimension on the feature graph comprises:

[0031] The feature map I4 is passed through the normalization layer to achieve pixel normalization. Based on MDTA, the convolution with kernel_size=1 is used to achieve dimension increase and the group convolution with kernel_size=3, stride=1, and padding=1 is used to divide the features into three parts to obtain Q, V, and K for self-attention calculation.

[0032] Use convolution with kernel_size=1, change the number of channels, save feature map I5, pass feature map I5 through the normalization layer to achieve pixel normalization, and set the hidden layer feature dimension equal to the input dimension multiplied by the expansion factor;

[0033] Based on GDFN, the features are divided into two parts through convolution with kernel_size=1, out_channel=hidden_features×2 to achieve dimensionality increase and group convolution with kernel_size=3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1, gating with GELU activation function, and convolution with kernel_size=1 to achieve dimensionality reduction, change the number of channels, and save feature map I6;

[0034] The feature map I6 is passed through the normalization layer to achieve pixel normalization. Based on MDTA, the convolution with kernel_size=1 is used to achieve dimension increase and the group convolution with kernel_size=3, stride=1, and padding=1 is used to divide the features into three parts to obtain Q, V, and K for self-attention calculation.

[0035] Use convolution with kernel_size=1 to achieve dimensionality reduction, change the number of channels, and save feature map I7;

[0036] Pass the feature map I7 through the normalization layer to achieve pixel normalization, and set the hidden layer feature dimension equal to the input dimension multiplied by the expansion factor;

[0037] Based on GDFN, the features are divided into two parts through convolution with kernel_size=1, out_channel=hidden_features×2 to achieve dimensionality increase and group convolution with kernel_size=3×3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1, and gating with GELU activation function. Convolution with kernel_size=1 is used to achieve dimensionality reduction, the number of channels is changed, and the feature map I8 is saved;

[0038] The feature map I8 is convolved with kernel_size = 1×1 to achieve dimensionality reduction, and then the PixelUnshuffle layer is used to increase the channel dimension to 4 times, while the size of the feature map is reduced to half of the original size, and the feature map I9 is ​​saved;

[0039] Execute the first step, which is to pass the feature map I9 through the normalization layer to realize pixel normalization, based on MDTA, through kernel_size = 1 convolution to achieve dimensionality increase and kernel_size = 3, stride = 1, padding = 1 group convolution, divide the features into three parts, obtain Q, V, K for self-attention calculation, use kernel_size = 1 convolution, change the number of channels, and save the feature map I 10 , the feature map I 10 After the normalization layer, the pixels are normalized, and the hidden layer feature dimension is set equal to the input dimension multiplied by the expansion factor. Based on GDFN, the convolution with kernel_size=1, out_channel=hidden_features×2 is used to increase the dimension, and the group convolution with kernel_size=3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1 is used to divide the features into two parts. The GELU activation function is used for gating, and the convolution with kernel_size=1 is used to reduce the dimension. The number of channels is changed and the feature map I is saved. 11 , repeat the first step twice again and save the final feature map I 12 ;

[0040] The feature map I 12 After kernel_size=1×1 convolution, the dimension is reduced. Then the PixelUnshuffle layer is used to increase the channel dimension to 4 times. At the same time, the size of the feature map is reduced to half of the original size. The feature map I is saved. 13 ;

[0041] Execute the second step, which is to transform the feature map I 13 After the normalization layer, the pixels are normalized. Based on MDTA, the convolution with kernel_size=1 is used to achieve dimension increase and the group convolution with kernel_size=3, stride=1, and padding=1 is used to divide the features into three parts. Q, V, and K are obtained for self-attention calculation. The convolution with kernel_size=1 is used to change the number of channels and save the feature map I. 14 , the feature map I 14 After the normalization layer, the pixels are normalized, and the hidden layer feature dimension is set equal to the input dimension multiplied by the expansion factor. Based on GDFN, the convolution of kernel_size=1, out_channel=hidden_features×2 is used to achieve dimensionality increase and the group convolution of kernel_size=3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1 is used to divide the features into two parts. The GELU activation function is used for gating, and the convolution of kernel_size=1 is used. The second step is repeated three times again, and the final feature map I is saved. 15 ;

[0042] The feature map I 15 After kernel_size=3×3 convolution, the channel dimension is increased to twice the original one, and then PixelUnshuffle is used to reduce the number of channels to 1 / 4 of the original one. At the same time, the size of the feature map is expanded to twice the original one, and the feature map I is saved. 16 .

[0043] In combination with the first aspect, in one implementation, the feature maps obtained at different processing stages are connected, and the feature maps obtained after the connection operation are normalized, dimensioned, and reduced, specifically including:

[0044] The feature map I 12 and feature map I 15 Perform the connection operation to obtain the feature map I 17 ;

[0045] Execute the third step, the third step is to convert the feature map I 17After the normalization layer, the pixels are normalized. Based on MDTA, the convolution with kernel_size=1 is used to achieve dimension increase and the group convolution with kernel_size=3, stride=1, and padding=1 is used to divide the features into three parts. Q, V, and K are obtained for self-attention calculation. The convolution with kernel_size=1 is used to change the number of channels and save the feature map I. 18 , the feature map I 18 After the normalization layer, the pixels are normalized, and the hidden layer feature dimension is set equal to the input dimension multiplied by the expansion factor. Based on GDFN, the convolution of kernel_size=1, out_channel=hidden_features×2 is used to achieve dimensionality increase and the group convolution of kernel_size=3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1 is used to divide the features into two parts. The GELU activation function is used for gating, and the convolution of kernel_size=1 is used to achieve dimensionality reduction. The number of channels is changed, and the feature map is saved. The third step is repeated twice to save the final feature map I 19 ;

[0046] The feature map I 19 After convolution with kernel_size=3, the channel dimension is increased to twice the original one, and then PixelUnshuffle is used to reduce the number of channels to 1 / 4 of the original one. At the same time, the size of the feature map is expanded to twice the original one, and the feature map I is saved. 20 ;

[0047] The feature map I8 and feature map I 20 Perform the connection operation to obtain the feature map I 21 ;

[0048] Execute the fourth step, the fourth step is to convert the feature map I 21 After the normalization layer, the pixels are normalized. Based on MDTA, the convolution with kernel_size=1 is used to achieve dimension increase and the group convolution with kernel_size=3, stride=1, and padding=1 is used to divide the features into three parts. Q, V, and K are obtained for self-attention calculation. The convolution with kernel_size=1 is used to change the number of channels and save the feature map I. 22 , the feature map I 22After the normalization layer, the pixels are normalized, and the hidden layer feature dimension is set equal to the input dimension multiplied by the expansion factor. Based on GDFN, the convolution of kernel_size=1, out_channel=hidden_features×2 is used to achieve dimensionality increase and the group convolution of kernel_size=3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1 is used to divide the features into two parts. The GELU activation function is used for gating, and the convolution of kernel_size=1 is used to achieve dimensionality reduction. The number of channels is changed, the feature map is saved, and the fourth step is repeated once again to save the final feature map I 23 ;

[0049] The feature map I 23 After convolution with kernel_size=3, the channel dimension is increased to twice the original one, and then PixelUnshuffle is used to reduce the number of channels to 1 / 4 of the original one. At the same time, the size of the feature map is expanded to twice the original one, and the feature map I is saved. 24 ;

[0050] Connect feature map I3 and feature map I9 to obtain feature map I 25 ;

[0051] The feature map I 25 After the normalization layer, the pixels are normalized. Based on MDTA, the convolution with kernel_size=1 is used to achieve dimension increase and the group convolution with kernel_size=3, stride=1, and padding=1 is used to divide the features into three parts. Q, V, and K are obtained for self-attention calculation. The convolution with kernel_size=1 is used to change the number of channels and save the feature map I. 26 , the feature map I 26 After the normalization layer, the pixels are normalized, and the hidden layer feature dimension is set equal to the input dimension multiplied by the expansion factor. Based on GDFN, the convolution of kernel_size=1, out_channel=hidden_features×2 is used to achieve dimensionality increase and the group convolution of kernel_size=3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1 is used to divide the features into two parts. The GELU activation function is used for gating, and the convolution of kernel_size=1 is used to achieve dimensionality reduction. The number of channels is changed and the final feature map I is saved. 27 .

[0052] In combination with the first aspect, in one implementation, extracting detail features from the feature map based on the Refinement module and performing image superposition through convolution specifically includes:

[0053] The feature map I 27 After the Refinement module is used to extract detail features, image superposition is achieved through convolution with kernel_size=3 to obtain feature map I 28 ;

[0054] The feature map I 28 After three downsamplings with kernel_size=3, feature map I is obtained. 29 , save the feature maps of each downsampling, respectively feature map I 30 , Feature Map I 31 , Feature Map I 32 ;

[0055] The feature map I 32 Use slices to split the image into multiple slices, map them to one-dimensional vectors through linear mapping according to the given size, output a vector sequence or two-dimensional matrix, input the vector sequence or two-dimensional matrix into the Transformer layer for 12 iterations, and save the feature map I 33 .

[0056] In combination with the first aspect, in one implementation, the feature maps of different resolutions are spliced, and the feature maps are upsampled and downsampled to generate a final result map to achieve feature extraction of the image to be detected, specifically including:

[0057] Based on each layer of the U-Net decoder and each layer of the CNN, the feature maps of different resolutions are spliced ​​using skip connections, and the feature maps I 30 , Feature Map I 31 , Feature Map I 32 As input;

[0058] The feature map I 33 After upsampling and feature map I 32 Perform concatenation, convolution with kernel_size=3 and upsampling, and save feature map I 34 ;

[0059] The feature map I 34 After upsampling and feature map I 31 Perform concatenation, convolution with kernel_size=3 and upsampling, and save feature map I 35 ;

[0060] The feature map I35 After upsampling and feature map I 30 Perform concatenation, convolution with kernel_size=3 and upsampling, and save feature map I 36 ;

[0061] The feature map I 36 After convolution with kernel_size=3, the final result graph is generated to achieve feature extraction of the image to be detected.

[0062] In a second aspect, an embodiment of the present application provides a pavement crack image detection device based on MDTA and GDFN, comprising:

[0063] An acquisition module is used to acquire a data set of pavement crack images under different pavement conditions, and annotate the data set according to the pavement background and the morphology of the cracks to obtain a training set, a validation set, and a test set;

[0064] A training module, which is used to pre-process the pavement crack images in the training set and train the pavement crack detection network model based on MDTA and GDFN through the training set;

[0065] A testing module, which is used to test the trained pavement crack detection network model through a test set and calculate evaluation indicators based on the test results;

[0066] The execution module is used to perform crack detection on the image to be detected using the current pavement crack detection network model based on the calculated evaluation index and when the evaluation index meets the set requirements.

[0067] The beneficial effects brought by the technical solution provided in the embodiments of the present application include:

[0068] A pavement crack image dataset under different pavement conditions is obtained, and the dataset is annotated according to the pavement background and the morphology of the cracks to obtain a training set, a validation set and a test set. Then, the pavement crack images in the training set are preprocessed, and a pavement crack detection network model based on MDTA and GDFN is trained by the training set. Then, the trained pavement crack detection network model is tested by the test set, and an evaluation index is calculated according to the test result. Then, based on the calculated evaluation index and when the evaluation index meets the set requirements, the current pavement crack detection network model is used to perform crack detection on the image to be detected. In view of the problems of noise diversity and low image quality of pavement crack images, the image noise can be effectively removed, thereby improving the detection accuracy of pavement cracks. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 This is a flow chart of the pavement crack image detection method based on MDTA and GDFN in this application;

[0070] Figure 2 This is a schematic diagram of the functional modules of the pavement crack image detection device based on MDTA and GDFN in this application. DETAILED DESCRIPTION

[0071] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0072] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0073] In the first aspect, the embodiment of the present application provides a pavement crack image detection method based on MDTA and GDFN, which can effectively remove image noise and improve the detection accuracy of pavement cracks in view of the noise diversity and low image quality of pavement crack images. MDTA, i.e. multi-Dconv head transposed attention, is an improved attention mechanism used to aggregate local and non-local related pixels, so as to efficiently perform high-resolution image processing; GDFN, i.e. gated-Dconv feed-forward network, is used to perform controllable feature transformation, i.e. suppress low-information features and retain only useful information.

[0074] In one embodiment, referring to Figure 1 , Figure 1 FIG. 1 is a flow chart of the pavement crack image detection method based on MDTA and GDFN in this application. Figure 1 As shown, the pavement crack image detection method based on MDTA and GDFN includes:

[0075] S1: Obtain a data set of pavement crack images under different pavement conditions, and annotate the data set according to the pavement background and the morphology of the cracks to obtain a training set, a validation set, and a test set;

[0076] In this application, labelImg software can be used to label the data set actively. The labeled data set is randomly divided into a training set, a validation set, and a test set in a ratio of 7:2:1.

[0077] S2: Preprocess the pavement crack images in the training set, and use the training set to train the pavement crack detection network model based on MDTA and GDFN;

[0078] In this application, preprocessing includes data enhancement, data normalization, and negative sample addition. The training set is preprocessed to enhance the richness of the pavement crack image data in the training set.

[0079] S3: The trained pavement crack detection network model is tested using a test set, and evaluation indicators are calculated based on the test results; in this application, the evaluation indicators include CPA (category pixel accuracy), MPA (category average pixel accuracy), MIOU (average intersection over union) and F1 (harmonic mean of precision and recall).

[0080] S4: Based on the calculated evaluation index and when the evaluation index meets the set requirements, the current pavement crack detection network model is used to perform crack detection on the image to be detected.

[0081] Furthermore, in one embodiment, based on the calculated evaluation index and when the evaluation index meets the set requirements, the current pavement crack detection network model is used to perform crack detection on the image to be detected, specifically including:

[0082] Determine whether the evaluation index calculated after testing the current pavement crack detection network model meets the set requirements:

[0083] If so, the current pavement crack detection network model is used to perform crack detection on the image to be detected;

[0084] If not, the parameters of the pavement crack detection network model are adjusted, the pavement crack detection network model is trained again with the training set, and the evaluation index is calculated again. This cycle is repeated until the calculated evaluation index meets the set requirements, and then the pavement crack detection network model whose evaluation index meets the set requirements is used to perform crack detection on the image to be detected.

[0085] Furthermore, in one embodiment, crack detection is performed on the image to be detected using the current pavement crack detection network model, specifically including:

[0086] S401: Given an image I with a spatial resolution of H×W×3, a feature map is obtained by convolution with kernel_size=3, stride=1, and padding=1, and the feature map is normalized, dimensionally increased, and dimensionally reduced;

[0087] S402: Repeat the normalization, dimension increase, and dimension reduction processing on the feature map for multiple times;

[0088] S403: Connect the feature maps obtained in different processing stages, and normalize, increase dimension, and reduce dimension on the feature maps obtained after the connection operation;

[0089] S404: extracting detail features from the feature map based on the Refinement module, and performing image superposition through convolution;

[0090] S405: splicing feature maps of different resolutions, and performing upsampling and downsampling processing on the feature maps to generate a final result map, thereby realizing feature extraction of the image to be detected.

[0091] Furthermore, in one embodiment, given an image I with a spatial resolution of H×W×3, a feature map is obtained by convolution with kernel_size=3, stride=1, and padding=1, and the feature map is normalized, dimensionally increased, and dimensionally reduced, specifically including:

[0092] S4011: Given an image I with a spatial resolution of H×W×3, a feature map I1 is obtained through convolution with kernel_size=3, stride=1, and padding=1, and the feature map I1 is used as the input of the image recovery module; for H×W×3, H represents the height of the image, W represents the width of the image, and 3 represents the number of channels of the image; kernel_size represents the size of the convolution kernel, stride represents the step size in the convolution operation, and padding represents the padding in the convolution;

[0093] S4012: The feature map I1 is passed through a normalization layer to normalize pixels, and then based on MDTA, a convolution with kernel_size=1 is performed to achieve dimension increase and a group convolution with kernel_size=3, stride=1, and padding=1 is performed to divide the features into three parts, and Q, V, and K are obtained for self-attention calculation; wherein Q represents the target word matrix, K represents the keyword matrix, and V represents the original feature;

[0094] S4013: Use convolution with kernel_size=1, change the number of channels, save feature map I2, pass feature map I2 through a normalization layer to achieve pixel normalization, and set the hidden layer feature dimension equal to the input dimension multiplied by the expansion factor;

[0095] S4014: Based on GDFN, the features are divided into two parts through convolution with kernel_size=1, out_channel=hidden_features×2 to achieve dimensionality increase and group convolution with kernel_size=3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1, and gating with GELU activation function, and convolution with kernel_size=1 to achieve dimensionality reduction, change the number of channels, and save feature map I3; where out_channel represents the number of output channels, hidden_features represents the number of input features, and in_channel represents the number of input channels;

[0096] S4015: Use convolution with kernel_size=3 to halve the number of channels of feature map I3, use the PixelUnshuffle layer to increase the channel dimension to 4 times, and at the same time reduce the size of the feature map to half of the original size, and save the feature map I4.

[0097] Furthermore, in one embodiment, the feature graph is normalized, dimensioned, and reduced multiple times, specifically including:

[0098] S4021: The feature map I4 is passed through a normalization layer to achieve pixel normalization. Based on MDTA, a convolution with kernel_size=1 is performed to achieve dimension increase and a group convolution with kernel_size=3, stride=1, and padding=1 is performed to divide the feature into three parts, and Q, V, and K are obtained for self-attention calculation.

[0099] S4022: Use convolution with kernel_size=1, change the number of channels, save feature map I5, pass feature map I5 through a normalization layer to achieve pixel normalization, and set the hidden layer feature dimension equal to the input dimension multiplied by the expansion factor;

[0100] S4023: Based on GDFN, the features are divided into two parts through convolution with kernel_size=1, out_channel=hidden_features×2 to achieve dimensionality increase and group convolution with kernel_size=3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1, and gating with GELU activation function, and convolution with kernel_size=1 to achieve dimensionality reduction, change the number of channels, and save feature map I6;

[0101] S4024: The feature map I6 is passed through a normalization layer to achieve pixel normalization. Based on MDTA, a convolution with kernel_size=1 is performed to achieve dimension increase and a group convolution with kernel_size=3, stride=1, and padding=1 is performed to divide the feature into three parts, and Q, V, and K are obtained for self-attention calculation.

[0102] S4025: Use convolution with kernel_size=1 to achieve dimensionality reduction, change the number of channels, and save feature map I7;

[0103] S4026: passing the feature map I7 through a normalization layer to achieve pixel normalization, and setting the hidden layer feature dimension equal to the input dimension multiplied by the expansion factor;

[0104] S4027: Based on GDFN, the features are divided into two parts through convolution with kernel_size=1, out_channel=hidden_features×2 to achieve dimensionality increase and group convolution with kernel_size=3×3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1, and gating with GELU activation function, and convolution with kernel_size=1 to achieve dimensionality reduction, change the number of channels, and save feature map I8;

[0105] S4028: Reduce the dimension of feature map I8 by convolution with kernel_size=1×1, and then use PixelUnshuffle layer to increase the channel dimension to 4 times, while reducing the size of the feature map to half of the original size, and save feature map I9;

[0106] S4029: Execute the first step, which is to pass the feature map I9 through the normalization layer to achieve pixel normalization, based on MDTA, through kernel_size=1 convolution to achieve dimensionality increase and kernel_size=3, stride=1, padding=1 group convolution, divide the features into three parts, obtain Q, V, K for self-attention calculation, use kernel_size=1 convolution, change the number of channels, and save the feature map I 10 , the feature map I 10After the normalization layer, the pixels are normalized, and the hidden layer feature dimension is set equal to the input dimension multiplied by the expansion factor. Based on GDFN, the convolution with kernel_size=1, out_channel=hidden_features×2 is used to increase the dimension, and the group convolution with kernel_size=3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1 is used to divide the features into two parts. The GELU activation function is used for gating, and the convolution with kernel_size=1 is used to reduce the dimension. The number of channels is changed and the feature map I is saved. 11 , repeat the first step twice again and save the final feature map I 12 ;

[0107] S40210: Feature map I 12 After kernel_size=1×1 convolution, the dimension is reduced. Then the PixelUnshuffle layer is used to increase the channel dimension to 4 times. At the same time, the size of the feature map is reduced to half of the original size. The feature map I is saved. 13 ;

[0108] S40211: Execute the second step, which is to convert the feature map I 13 After the normalization layer, the pixels are normalized. Based on MDTA, the convolution with kernel_size=1 is used to achieve dimension increase and the group convolution with kernel_size=3, stride=1, and padding=1 is used to divide the features into three parts. Q, V, and K are obtained for self-attention calculation. The convolution with kernel_size=1 is used to change the number of channels and save the feature map I. 14 , the feature map I 14 After the normalization layer, the pixels are normalized, and the hidden layer feature dimension is set equal to the input dimension multiplied by the expansion factor. Based on GDFN, the convolution of kernel_size=1, out_channel=hidden_features×2 is used to achieve dimensionality increase and the group convolution of kernel_size=3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1 is used to divide the features into two parts. The GELU activation function is used for gating, and the convolution of kernel_size=1 is used. The second step is repeated three times again, and the final feature map I is saved. 15 ;

[0109] S40212: Feature map I 15After kernel_size=3×3 convolution, the channel dimension is increased to twice the original one, and then PixelUnshuffle is used to reduce the number of channels to 1 / 4 of the original one. At the same time, the size of the feature map is expanded to twice the original one, and the feature map I is saved. 16 .

[0110] Furthermore, in one embodiment, the feature maps obtained at different processing stages are connected, and the feature maps obtained after the connection operation are normalized, dimensioned, and reduced, specifically including:

[0111] S4031: Feature map I 12 and feature map I 15 Perform the connection operation to obtain the feature map I 17 ;

[0112] S4032: Execute the third step, which is to convert the feature map I 17 After the normalization layer, the pixels are normalized. Based on MDTA, the convolution with kernel_size=1 is used to achieve dimension increase and the group convolution with kernel_size=3, stride=1, and padding=1 is used to divide the features into three parts. Q, V, and K are obtained for self-attention calculation. The convolution with kernel_size=1 is used to change the number of channels and save the feature map I. 18 , the feature map I 18 After the normalization layer, the pixels are normalized, and the hidden layer feature dimension is set equal to the input dimension multiplied by the expansion factor. Based on GDFN, the convolution of kernel_size=1, out_channel=hidden_features×2 is used to achieve dimensionality increase and the group convolution of kernel_size=3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1 is used to divide the features into two parts. The GELU activation function is used for gating, and the convolution of kernel_size=1 is used to achieve dimensionality reduction. The number of channels is changed, and the feature map is saved. The third step is repeated twice to save the final feature map I 19 ;

[0113] S4033: Feature map I 19 After convolution with kernel_size=3, the channel dimension is increased to twice the original one, and then PixelUnshuffle is used to reduce the number of channels to 1 / 4 of the original one. At the same time, the size of the feature map is expanded to twice the original one, and the feature map I is saved. 20 ;

[0114] S4034: Combine feature map I8 and feature map I 20 Perform the connection operation to obtain the feature map I 21 ;

[0115] S4035: Execute the fourth step, which is to convert the feature map I 21 After the normalization layer, the pixels are normalized. Based on MDTA, the convolution with kernel_size=1 is used to achieve dimension increase and the group convolution with kernel_size=3, stride=1, and padding=1 is used to divide the features into three parts. Q, V, and K are obtained for self-attention calculation. The convolution with kernel_size=1 is used to change the number of channels and save the feature map I. 22 , the feature map I 22 After the normalization layer, the pixels are normalized, and the hidden layer feature dimension is set equal to the input dimension multiplied by the expansion factor. Based on GDFN, the convolution of kernel_size=1, out_channel=hidden_features×2 is used to achieve dimensionality increase and the group convolution of kernel_size=3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1 is used to divide the features into two parts. The GELU activation function is used for gating, and the convolution of kernel_size=1 is used to achieve dimensionality reduction. The number of channels is changed, the feature map is saved, and the fourth step is repeated once again to save the final feature map I 23 ;

[0116] S4036: Feature map I 23 After convolution with kernel_size=3, the channel dimension is increased to twice the original one, and then PixelUnshuffle is used to reduce the number of channels to 1 / 4 of the original one. At the same time, the size of the feature map is expanded to twice the original one, and the feature map I is saved. 24 ;

[0117] S4037: Connect the feature map I3 and the feature map I9 to obtain the feature map I 25 ;

[0118] S4038: Feature map I 25 After the normalization layer, the pixels are normalized. Based on MDTA, the convolution with kernel_size=1 is used to achieve dimension increase and the group convolution with kernel_size=3, stride=1, and padding=1 is used to divide the features into three parts. Q, V, and K are obtained for self-attention calculation. The convolution with kernel_size=1 is used to change the number of channels and save the feature map I.26 , the feature map I 26 After the normalization layer, the pixels are normalized, and the hidden layer feature dimension is set equal to the input dimension multiplied by the expansion factor. Based on GDFN, the convolution of kernel_size=1, out_channel=hidden_features×2 is used to achieve dimensionality increase and the group convolution of kernel_size=3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1 is used to divide the features into two parts. The GELU activation function is used for gating, and the convolution of kernel_size=1 is used to achieve dimensionality reduction. The number of channels is changed and the final feature map I is saved. 27 .

[0119] Furthermore, in one embodiment, detail features are extracted from the feature map based on the Refinement module, and image superposition is performed through convolution, specifically including:

[0120] S4041: Feature map I 27 After the Refinement module is used to extract detail features, image superposition is achieved through convolution with kernel_size=3 to obtain feature map I 28 ;

[0121] S4042: Feature map I 28 After three downsamplings with kernel_size=3, feature map I is obtained. 29 , save the feature maps of each downsampling, respectively feature map I 30 , Feature Map I 31 , Feature Map I 32 ;

[0122] S4041: Feature map I 32 Use slices to split the image into multiple slices, map them to one-dimensional vectors through linear mapping according to the given size, output a vector sequence or two-dimensional matrix, input the vector sequence or two-dimensional matrix into the Transformer layer (the basic building block in the Transformer model) for 12 iterations, and save the feature map I 33 .

[0123] Furthermore, in one embodiment, feature maps of different resolutions are spliced, and the feature maps are upsampled and downsampled to generate a final result map, thereby realizing feature extraction of the image to be detected, which specifically includes:

[0124] S4051: Based on each layer of the U-Net (an image segmentation network) decoder and each layer of the CNN (Convolutional Neural Networks), the feature maps of different resolutions are spliced ​​using skip connections, respectively. 30 , Feature Map I 31 , Feature Map I 32 As input;

[0125] S4052: Feature map I 33 After upsampling and feature map I 32 Perform concatenation, convolution with kernel_size=3 and upsampling, and save feature map I 34 ;

[0126] S4053: Feature map I 34 After upsampling and feature map I 31 Perform concatenation, convolution with kernel_size=3 and upsampling, and save feature map I 35 ;

[0127] S4054: Feature map I 35 After upsampling and feature map I 30 Perform concatenation, convolution with kernel_size=3 and upsampling, and save feature map I 36 ;

[0128] S4055: Feature map I 36 After convolution with kernel_size=3, the final result graph is generated to achieve feature extraction of the image to be detected.

[0129] The pavement crack image detection method based on MDTA and GDFN in this application is compared with FCN, PSPNet, deeplab V3+, U-Net, and TransUNet, and the four evaluation indicators of PA, MPA, MIOU, and F1 are used to verify the experimental results. The experimental evaluation accuracy results are shown in Table 1 below. From a visual point of view, the crack edges of the result image detected by the method of this application are complete and clear, with less noise and better detection results. It can be seen from the table that the three indicators of the method proposed in this application are greater than other methods. In order to verify the effectiveness of the introduced loss function, this application provides an ablation experiment to compare the evaluation indicators of the model without loss function and the model with loss function. The experimental evaluation accuracy results are shown in Table 2 below.

[0130] Table 1 Experimental evaluation accuracy results

[0131]

[0132] Table 2 Evaluation accuracy results of ablation experiments

[0133]

[0134] The pavement crack image detection method based on MDTA and GDFN in the embodiment of the present application can effectively remove image noise to solve the problems of noise diversity and low image quality of pavement crack images, thereby improving the detection accuracy of pavement cracks.

[0135] In a second aspect, an embodiment of the present application also provides a pavement crack image detection device based on MDTA and GDFN.

[0136] In one embodiment, referring to Figure 2 , Figure 2 This is a functional module diagram of the pavement crack image detection device based on MDTA and GDFN in this application. Figure 2 As shown, the pavement crack image detection device based on MDTA and GDFN includes an acquisition module, a training module, a testing module and an execution module.

[0137] The acquisition module is used to obtain a data set of pavement crack images under different pavement conditions, and annotate the data set according to the pavement background and the morphology of the cracks to obtain a training set, a validation set and a test set; the training module is used to preprocess the pavement crack images in the training set, and train a pavement crack detection network model based on MDTA and GDFN through the training set; the testing module is used to test the trained pavement crack detection network model through the test set, and calculate the evaluation index according to the test results; the execution module is used to use the current pavement crack detection network model to perform crack detection on the image to be detected based on the calculated evaluation index and when the evaluation index meets the set requirements.

[0138] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit "first", "second" and "third" to different types.

[0139] In the description of the embodiments of the present application, "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a specific way.

[0140] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; the “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0141] In some processes described in the embodiments of the present application, multiple operations or steps that appear in a specific order are included, but it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of the present application or in parallel, and the sequence number of the operation is only used to distinguish the different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed in sequence or in parallel, and these operations or steps may be combined.

[0142] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present application.

[0143] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A pavement crack image detection method based on MDTA and GDFN, It is characterized in that The pavement crack image detection method based on MDTA and GDFN includes: Obtaining a data set of pavement crack images under different pavement conditions, and annotating the data set according to the pavement background and the morphology of the cracks to obtain a training set, a validation set, and a test set; The pavement crack images in the training set are preprocessed, and the pavement crack detection network model based on MDTA and GDFN is trained through the training set; The trained pavement crack detection network model is tested using the test set, and the evaluation index is calculated based on the test results; Based on the calculated evaluation index and when the evaluation index meets the set requirements, the current pavement crack detection network model is used to perform crack detection on the image to be detected; The method of using the current pavement crack detection network model to perform crack detection on the image to be detected specifically includes: Given an image I with a spatial resolution of H×W×3, a feature map is obtained by convolution with kernel_size=3, stride=1, and padding=1, and the feature map is normalized, dimensionally increased, and dimensionally reduced; Repeat the normalization, dimension increase and dimension reduction of the feature map many times; The feature maps obtained at different processing stages are connected, and the feature maps obtained after the connection operation are normalized, dimensioned, and reduced; Detailed features are extracted from feature maps based on the Refinement module, and images are superimposed through convolution. The feature maps of different resolutions are spliced, and the feature maps are upsampled and downsampled to generate the final result map, thereby realizing feature extraction of the image to be detected.

2. A pavement crack image detection method based on MDTA and GDFN as claimed in claim 1, Features: The preprocessing includes data enhancement, data normalization, and negative sample addition; The evaluation indicators include CPA, MPA, MIOU and F1.

3. A pavement crack image detection method based on MDTA and GDFN as claimed in claim 1, It is characterized in that The method of using the current pavement crack detection network model to perform crack detection on the image to be detected based on the calculated evaluation index and when the evaluation index meets the set requirements specifically includes: Determine whether the evaluation index calculated after testing the current pavement crack detection network model meets the set requirements: If so, the current pavement crack detection network model is used to perform crack detection on the image to be detected; If not, the parameters of the pavement crack detection network model are adjusted, the pavement crack detection network model is trained again with the training set, and the evaluation index is calculated again. This cycle is repeated until the calculated evaluation index meets the set requirements, and then the pavement crack detection network model whose evaluation index meets the set requirements is used to perform crack detection on the image to be detected.

4. A pavement crack image detection method based on MDTA and GDFN as claimed in claim 1, It is characterized in that The given image I with a spatial resolution of H×W×3 is convolved with kernel_size=3, stride=1, and padding=1 to obtain a feature map, and the feature map is normalized, dimensionally increased, and dimensionally reduced, specifically including: Given an image I with a spatial resolution of H×W×3, the feature map I is obtained by convolution with kernel_size=3, stride=1, and padding=1. 1 ; The feature map I 1 After the normalization layer, the pixels are normalized, and then based on MDTA, the convolution with kernel_size=1 is used to achieve dimension increase and the group convolution with kernel_size=3, stride=1, and padding=1 is used to divide the features into three parts, and Q, V, and K are obtained for self-attention calculation; Use convolution with kernel_size=1, change the number of channels, and save feature map I 2 , the feature map I 2 After the normalization layer, the pixels are normalized and the hidden layer feature dimension is set equal to the input dimension multiplied by the expansion factor; Based on GDFN, the features are divided into two parts through convolution with kernel_size=1, out_channel=hidden_features×2 to achieve dimensionality increase and group convolution with kernel_size=3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1. The GELU activation function is used for gating, and convolution with kernel_size=1 is used to achieve dimensionality reduction. The number of channels is changed and the feature map I is saved. 3 ; Use kernel_size=3 convolution to transform feature map I 3 The number of channels is halved, and the PixelUnshuffle layer is used to increase the channel dimension to 4 times. At the same time, the size of the feature map is reduced to half of the original size, and the feature map I is saved. 4 .

5. A pavement crack image detection method based on MDTA and GDFN as claimed in claim 4, It is characterized in that The process of repeatedly normalizing, increasing dimension, and reducing dimension of the feature graph includes: The feature map I 4 After the normalization layer, the pixels are normalized. Based on MDTA, the convolution with kernel_size=1 is used to increase the dimension, and the group convolution with kernel_size=3, stride=1, and padding=1 is used to divide the features into three parts, and Q, V, and K are obtained for self-attention calculation. Use convolution with kernel_size=1, change the number of channels, and save feature map I 5 , the feature map I 5 After the normalization layer, the pixels are normalized and the hidden layer feature dimension is set equal to the input dimension multiplied by the expansion factor; Based on GDFN, the features are divided into two parts through convolution with kernel_size=1, out_channel=hidden_features×2 to achieve dimensionality increase and group convolution with kernel_size=3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1. The GELU activation function is used for gating, and convolution with kernel_size=1 is used to achieve dimensionality reduction. The number of channels is changed and the feature map I is saved. 6 ; The feature map I 6 After the normalization layer, the pixels are normalized. Based on MDTA, the convolution with kernel_size=1 is used to increase the dimension, and the group convolution with kernel_size=3, stride=1, and padding=1 is used to divide the features into three parts, and Q, V, and K are obtained for self-attention calculation. Use convolution with kernel_size=1 to achieve dimensionality reduction, change the number of channels, and save feature map I 7 ; The feature map I 7 After the normalization layer, the pixels are normalized and the hidden layer feature dimension is set equal to the input dimension multiplied by the expansion factor; Based on GDFN, the features are divided into two parts through convolution with kernel_size=1, out_channel=hidden_features×2 to achieve dimensionality increase and group convolution with kernel_size=3×3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1. The GELU activation function is used for gating, and convolution with kernel_size=1 is used to achieve dimensionality reduction. The number of channels is changed and the feature map I is saved. 8 ; The feature map I 8 After kernel_size=1×1 convolution, the dimension is reduced. Then the PixelUnshuffle layer is used to increase the channel dimension to 4 times. At the same time, the size of the feature map is reduced to half of the original size. The feature map I is saved. 9 ; Execute the first step, which is to transform the feature map I 9 After the normalization layer, the pixels are normalized. Based on MDTA, the convolution with kernel_size=1 is used to achieve dimension increase and the group convolution with kernel_size=3, stride=1, and padding=1 is used to divide the features into three parts. Q, V, and K are obtained for self-attention calculation. The convolution with kernel_size=1 is used to change the number of channels and save the feature map I. 10 , the feature map I 10 After the normalization layer, the pixels are normalized, and the hidden layer feature dimension is set equal to the input dimension multiplied by the expansion factor. Based on GDFN, the convolution with kernel_size=1, out_channel=hidden_features×2 is used to increase the dimension, and the group convolution with kernel_size=3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1 is used to divide the features into two parts. The GELU activation function is used for gating, and the convolution with kernel_size=1 is used to reduce the dimension. The number of channels is changed and the feature map I is saved. 11 , repeat the first step twice again and save the final feature map I 12 ; The feature map I 12 After kernel_size=1×1 convolution, the dimension is reduced. Then the PixelUnshuffle layer is used to increase the channel dimension to 4 times. At the same time, the size of the feature map is reduced to half of the original size. The feature map I is saved. 13 ; Execute the second step, which is to convert the feature map I 13 After the normalization layer, the pixels are normalized. Based on MDTA, the convolution with kernel_size=1 is used to achieve dimension increase and the group convolution with kernel_size=3, stride=1, and padding=1 is used to divide the features into three parts. Q, V, and K are obtained for self-attention calculation. The convolution with kernel_size=1 is used to change the number of channels and save the feature map I. 14 , the feature map I 14 After the normalization layer, the pixels are normalized, and the hidden layer feature dimension is set equal to the input dimension multiplied by the expansion factor. Based on GDFN, the convolution of kernel_size=1, out_channel=hidden_features×2 is used to achieve dimensionality increase and the group convolution of kernel_size=3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1 is used to divide the features into two parts. The GELU activation function is used for gating, and the convolution of kernel_size=1 is used. The second step is repeated three times again, and the final feature map I is saved. 15 ; The feature map I 15 After kernel_size=3×3 convolution, the channel dimension is increased to twice the original one, and then PixelUnshuffle is used to reduce the number of channels to 1 / 4 of the original one. At the same time, the size of the feature map is expanded to twice the original one, and the feature map I is saved. 16 .

6. A pavement crack image detection method based on MDTA and GDFN as claimed in claim 5, It is characterized in that The feature maps obtained at different processing stages are connected, and the feature maps obtained after the connection operation are normalized, dimensioned, and reduced, specifically including: The feature map I 12 and feature map I 15 Perform the connection operation to obtain the feature map I 17 ; Execute the third step, the third step is to convert the feature map I 17 After the normalization layer, the pixels are normalized. Based on MDTA, the convolution with kernel_size=1 is used to achieve dimension increase and the group convolution with kernel_size=3, stride=1, and padding=1 is used to divide the features into three parts. Q, V, and K are obtained for self-attention calculation. The convolution with kernel_size=1 is used to change the number of channels and save the feature map I. 18 , the feature map I 18 After the normalization layer, the pixels are normalized, and the hidden layer feature dimension is set equal to the input dimension multiplied by the expansion factor. Based on GDFN, the convolution of kernel_size=1, out_channel=hidden_features×2 is used to achieve dimensionality increase and the group convolution of kernel_size=3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1 is used to divide the features into two parts. The GELU activation function is used for gating, and the convolution of kernel_size=1 is used to achieve dimensionality reduction. The number of channels is changed, and the feature map is saved. The third step is repeated twice to save the final feature map I 19 ; The feature map I 19 After convolution with kernel_size=3, the channel dimension is increased to twice the original one, and then PixelUnshuffle is used to reduce the number of channels to 1 / 4 of the original one. At the same time, the size of the feature map is expanded to twice the original one, and the feature map I is saved. 20 ; The feature map I 8 and feature map I 20 Perform the connection operation to obtain the feature map I 21 ; Execute the fourth step, the fourth step is to convert the feature map I 21 After the normalization layer, the pixels are normalized. Based on MDTA, the convolution with kernel_size=1 is used to achieve dimension increase and the group convolution with kernel_size=3, stride=1, and padding=1 is used to divide the features into three parts. Q, V, and K are obtained for self-attention calculation. The convolution with kernel_size=1 is used to change the number of channels and save the feature map I. 22 , the feature map I 22 After the normalization layer, the pixels are normalized, and the hidden layer feature dimension is set equal to the input dimension multiplied by the expansion factor. Based on GDFN, the convolution of kernel_size=1, out_channel=hidden_features×2 is used to achieve dimensionality increase and the group convolution of kernel_size=3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1 is used to divide the features into two parts. The GELU activation function is used for gating, and the convolution of kernel_size=1 is used to achieve dimensionality reduction. The number of channels is changed, the feature map is saved, and the fourth step is repeated once again to save the final feature map I 23 ; The feature map I 23 After convolution with kernel_size=3, the channel dimension is increased to twice the original one, and then PixelUnshuffle is used to reduce the number of channels to 1 / 4 of the original one. At the same time, the size of the feature map is expanded to twice the original one, and the feature map I is saved. 24 ; The feature map I 3 and feature map I 9 Perform the connection operation to obtain the feature map I 25 ; The feature map I 25 After the normalization layer, the pixels are normalized. Based on MDTA, the convolution with kernel_size=1 is used to achieve dimension increase and the group convolution with kernel_size=3, stride=1, and padding=1 is used to divide the features into three parts. Q, V, and K are obtained for self-attention calculation. The convolution with kernel_size=1 is used to change the number of channels and save the feature map I. 26 , the feature map I 26 After the normalization layer, the pixels are normalized, and the hidden layer feature dimension is set equal to the input dimension multiplied by the expansion factor. Based on GDFN, the convolution of kernel_size=1, out_channel=hidden_features×2 is used to achieve dimensionality increase and the group convolution of kernel_size=3, in_channel=hidden_features×2, out_channel=hidden_features×2, stride=1, padding=1 is used to divide the features into two parts. The GELU activation function is used for gating, and the convolution of kernel_size=1 is used to achieve dimensionality reduction. The number of channels is changed and the final feature map I is saved. 27 .

7. A pavement crack image detection method based on MDTA and GDFN as claimed in claim 6, It is characterized in that The refinement module is used to extract detail features from the feature map and to perform image superposition through convolution, specifically including: The feature map I 27 After the Refinement module is used to extract detail features, image superposition is achieved through convolution with kernel_size=3 to obtain feature map I 28 ; The feature map I 28 After three downsamplings with kernel_size=3, feature map I is obtained. 29 , save the feature maps of each downsampling, respectively feature map I 30 , Feature Map I 31 , Feature Map I 32 ; The feature map I 32 Use slices to split the image into multiple slices, map them to one-dimensional vectors through linear mapping according to the given size, output a vector sequence or two-dimensional matrix, input the vector sequence or two-dimensional matrix into the Transformer layer for 12 iterations, and save the feature map I 33 .

8. A pavement crack image detection method based on MDTA and GDFN as claimed in claim 7, It is characterized in that The feature maps of different resolutions are spliced, and the feature maps are upsampled and downsampled to generate a final result map to extract features of the image to be detected, which specifically includes: Based on each layer of the U-Net decoder and each layer of the CNN, the feature maps of different resolutions are spliced ​​using skip connections, and the feature maps I 30 , Feature Map I 31 , Feature Map I 32 As input; The feature map I 33 After upsampling and feature map I 32 Perform concatenation, convolution with kernel_size=3 and upsampling, and save feature map I 34 ; The feature map I 34 After upsampling and feature map I 31 Perform concatenation, convolution with kernel_size=3 and upsampling, and save feature map I 35 ; The feature map I 35 After upsampling and feature map I 30 Perform concatenation, convolution with kernel_size=3 and upsampling, and save feature map I 36 ; The feature map I 36 After convolution with kernel_size=3, the final result graph is generated to achieve feature extraction of the image to be detected.

9. A pavement crack image detection device based on MDTA and GDFN, It is characterized in that include: An acquisition module is used to acquire a data set of pavement crack images under different pavement conditions, and annotate the data set according to the pavement background and the morphology of the cracks to obtain a training set, a validation set, and a test set; A training module, which is used to pre-process the pavement crack images in the training set and train the pavement crack detection network model based on MDTA and GDFN through the training set; A testing module, which is used to test the trained pavement crack detection network model through a test set and calculate evaluation indicators based on the test results; An execution module, which is used to perform crack detection on the image to be detected using the current pavement crack detection network model based on the calculated evaluation index and when the evaluation index meets the set requirements; The method of using the current pavement crack detection network model to perform crack detection on the image to be detected specifically includes: Given an image I with a spatial resolution of H×W×3, a feature map is obtained by convolution with kernel_size=3, stride=1, and padding=1, and the feature map is normalized, dimensionally increased, and dimensionally reduced; Repeat the normalization, dimension increase and dimension reduction of the feature map many times; The feature maps obtained at different processing stages are connected, and the feature maps obtained after the connection operation are normalized, dimensioned, and reduced; Detailed features are extracted from feature maps based on the Refinement module, and images are superimposed through convolution. The feature maps of different resolutions are spliced, and the feature maps are upsampled and downsampled to generate the final result map, thereby realizing feature extraction of the image to be detected.