A pavement disease identification method based on deep learning

Abstract

The application belongs to the technical field of target detection, and particularly relates to a road surface disease identification method based on deep learning, comprising the following steps: S1, constructing a road surface disease identification model; the road surface disease identification model is improved from a YOLOv7 model, and the improvement comprises: replacing ordinary convolution of a feature extraction part in a backbone network with deformable convolution; and introducing a spatial explicit visual center scheme EVC in an overall network structure; the visual center scheme EVC comprises a lightweight MLP and a learnable visual center mechanism LVC; S2, acquiring a training data set comprising road surface disease images and corresponding label information; S3, training the road surface disease identification model using the training data set; S4, identifying road surface diseases using the trained road surface disease identification model. Using the method, various disease types of the road surface can be accurately identified in a real road surface environment.

Description

Technical Field

[0001] This invention belongs to the field of target detection technology, and in particular relates to a method for identifying road surface defects based on deep learning. Background Technology

[0002] With the accelerating pace of urbanization and the rapid development of transportation, the construction and maintenance of road infrastructure have become crucial for ensuring traffic safety and economic development. However, due to long-term road use, natural environmental factors, and vehicle load, various defects often appear on road surfaces, such as cracks and potholes. Especially with the rapid development of intelligent driving, the comprehensive collection and accurate identification of road surface defects will significantly impact the comfort and safety of autonomous driving. Therefore, precise detection and location of these defects are essential for driving safety and road maintenance, not only improving driving safety but also effectively enhancing the efficiency and quality of road maintenance.

[0003] Currently, there are three main types of methods for detecting road surface defects.

[0004] The first method is to rely on manual observation and recording for screening. This is an early method, and because the subjective judgment of the screening personnel is involved in the detection process, it will affect the detection results. Moreover, when facing a large road network, it is inevitably a time-consuming and labor-intensive project to conduct detection manually.

[0005] The second type is the traditional image processing-based detection method that uses manual extraction of disease features and designs complex rules and processes for disease detection. Although it can reduce the problems of strong subjectivity and high labor costs caused by manual detection, its ability to identify diseases of different sizes and shapes is limited, and its recognition effect is not high in complex backgrounds.

[0006] The last approach utilizes image feature representations that can be automatically learned. This method is better adapted to various complex scenes and objects, and can be trained end-to-end on large-scale datasets, improving recognition accuracy and generalization ability, making it the current mainstream approach. However, most current methods focus only on the single task of identifying pavement cracks, neglecting other types of pavement defects; secondly, existing research often only targets road surface images from overhead views, lacking more realistic road environment information, which limits the model's generalization ability in real-world scenarios.

[0007] Therefore, how to accurately identify various types of road surface defects in real road environments, thereby improving driving comfort and the timeliness of road maintenance, has become an urgent problem to be solved. Summary of the Invention

[0008] To address the shortcomings of the existing technologies, this invention provides a deep learning-based method for identifying road surface defects, which can accurately identify various types of road surface defects in real-world road environments, thereby improving driving comfort and the timeliness of road maintenance.

[0009] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0010] A deep learning-based method for identifying road surface defects includes the following steps:

[0011] S1. Construct a pavement defect identification model; the pavement defect identification model is an improvement of the YOLOv7 model, the improvement includes: replacing the ordinary convolution in the feature extraction part of the backbone network with deformable convolution; and introducing the spatial explicit visual center scheme EVC into the overall network structure;

[0012] The Vision Center (EVC) scheme includes a Lightweight MLP and a Learnable Vision Center (LVC) mechanism. The EVC process includes: capturing the global long-range dependencies of top-level features through the Lightweight MLP as global information of the detected object; extracting local information features through the LVC; and then concatenating the long-range dependencies captured by the Lightweight MLP and the local information features learned by the LVC along the feature dimension as the output of the EVC vision center.

[0013] S2. Obtain a training dataset that includes images of road surface defects and corresponding annotation information;

[0014] S3. Train the road surface defect identification model using the training dataset;

[0015] S4. Use the trained pavement defect identification model to identify pavement defects;

[0016] The order of S1 and S2 can be interchanged.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] In this method, the pavement distress identification model is an improvement upon the YOLOv7 model. Considering the significant differences in pavement distress shapes and the potential for distresses of varying scales at different locations, standard convolutional kernels typically sample at fixed positions on the feature map during feature extraction. This approach suffers from poor adaptability to unknown variations and weak generalization ability. Therefore, this method introduces deformable convolution during feature extraction. The model learns how to adjust the sampling position of the convolutional kernel under different conditions, allowing it to better approximate the shape and size of the distress compared to traditional convolution, thus extracting input features more effectively.

[0019] In addition, existing feature pyramid structures, including the SPPCSPC structure in YOLOv7, primarily focus on information transfer between different levels, neglecting the internal patterns of features within a layer. However, the texture or shape features of some targets are mainly reflected in the intra-layer feature representation. Therefore, when dealing with targets with significant scale variations, these methods may not be able to fully cover all scale ranges. The road surface defect recognition model constructed in this method solves the above problem by introducing a spatially explicit visual center scheme (EVC) into the overall network structure, ensuring complete coverage of all scale ranges when dealing with targets with significant scale variations.

[0020] Through the above optimizations, the road surface defect identification model constructed by this method can not only detect and identify a variety of defect types, but also has high stability in complex environments and can adapt to various complex detection environments.

[0021] In summary, this method can accurately identify various types of road surface defects in real-world road environments, thereby improving driving comfort and the timeliness of road maintenance.

[0022] Preferably, in S1, when replacing the ordinary convolution in the feature extraction part of the backbone network with deformable convolution, deformable convolution is introduced before the ELAN module of YOLOv7.

[0023] This setup adds an offset to the sampling position in standard convolution operations. Through parallel convolution, the model learns to determine the displacement of the sampling position on the feature map during convolution operations. During training, the model adjusts the sampling position of the original feature map along the x and y axes using optimization algorithms such as backpropagation and gradient descent. Therefore, the model can learn how to adjust the sampling position of the convolution kernel under different conditions. Compared with traditional convolution, it can better approximate the shape and size of the lesion, thus extracting the input features more effectively.

[0024] Preferably, when introducing deformable convolutions before the ELAN module of YOLOv7, only the convolution with a stride of 1 before the ELAN module is replaced.

[0025] Considering that this convolution increases the number of model parameters and computational complexity, this method only replaces the convolution with a stride of 1 before replacing the ELAN module, while retaining the convolution with a stride of 2. This is because convolutions with a stride of 1 are typically used for dense feature extraction on feature maps, which helps capture the details and local features of the target, while convolutions with a stride of 2 are typically used for downsampling feature maps to increase the receptive field, but this leads to a reduction in the size of the feature map and a loss of some detailed information.

[0026] Preferably, in S1, the lightweight MLP consists of two residual modules; the residual module includes a depthwise separable convolution module and a channel MLP module; the input of the depthwise separable convolution module is a feature map that has undergone group normalization, and the output of the depthwise separable convolution module is the input of the channel MLP module.

[0027] This setup allows depthwise separable convolutional modules to improve feature representation while reducing computational costs. Furthermore, channel-based MLP modules, compared to spatial MLP modules, not only effectively reduce computational complexity but also meet the requirements of general vision tasks.

[0028] Preferably, the residual module further includes a channel scaling module and a DropPath module, used to perform channel scaling and DropPath operations on the output of the channel MLP module.

[0029] This setting can improve the generalization performance and robustness of features.

[0030] Preferably, in S1, the LVC is an encoder with a dictionary, consisting of a codebook B = {b1, b2, ..., b...} n} and a set of visual center scaling factors S = {s1, s2, ..., s} n The LVC process comprises:

[0031] First, a set of convolutional layers is used to process the input features. Encode it and input it into codebook B;

[0032] Then, by using the visual center scaling factor S to make and b k By mapping the corresponding positional information, we can obtain the information of the entire image regarding the k-th codeword in codebook B. Among them, b k Let k be the learnable codeword in codebook B;

[0033] Then, the feature information at different scales is integrated and synthesized using the following formula;

[0034] ;

[0035] Then, using the following formula, a fully connected layer and a 1×1 convolutional layer are used to predict prominent key class features, after which the features are activated by the sigmoid function and compared with the input features. Multiplying along the channel dimension improves the network's expressive power;

[0036] ;

[0037] Finally, the local information features learned by LVC are obtained using the following formula:

[0038] .

[0039] This setup ensures the accuracy and effectiveness of the extracted local information features.

[0040] Preferably, the encoded features are scaled using a visual center scaling factor S. and b k When mapping the corresponding position information, the information of the entire image about the k-th codeword is calculated using the following formula. :

[0041] );

[0042] In the formula, For the i-th pixel, Let S be the k-th learnable scaling factor in the visual center scaling factor S, and P be the total number of pixels.

[0043] This setup, by using the square of the Euclidean distance, can find the codeword that best matches the input features, thus mapping the input features into the coding space. Then, multiplying by a scaling factor can adjust the distance metric, ensuring that the calculation of the coding weights can reflect the degree of feature matching well under different conditions. The weights can accurately represent the distribution of input features in the codeword space; the distributed weights are then compared with... Multiplication is used to weight the input features, causing the model to focus more on those features that better match the encoded codewords, i.e., the more important features. This ensures that the obtained... The accuracy and effectiveness.

[0044] Preferably, in S3, the pavement defect identification model is trained based on an improved loss function RLOSS; the improved loss function RLOSS is:

[0045] ;

[0046] ;

[0047] In the formula, d is the total number of samples; y i The true value of the i-th sample is encoded using one-hot encoding; This represents the model's prediction for the i-th sample; This indicates that the predicted value is scaled to the range of 0-1 after being processed by sigmoid; the IOU value is used to evaluate the degree of matching between the predicted box and the ground truth box.

[0048] This configuration reduces the loss contribution of easily classified samples (1 − Iou / Iou) while increasing the loss proportion of difficult-to-classify samples, thus addressing the issue of low accuracy due to a small number of positive samples. The LOSS value is negatively correlated with IOU; a higher IOU indicates a more easily distinguishable sample, with 1 − Iou / Iou approaching 0, indicating a smaller contribution to the loss. Conversely, a lower IOU value results in a larger contribution to the loss. By focusing on these samples, the model can better learn and adapt to complex road surface distress scenarios, improving the algorithm's generalization ability.

[0049] Preferably, in S2, the acquired training dataset is further augmented; the augmentation includes random cropping and rotation.

[0050] In this way, by enhancing the training dataset, the generalization ability of the trained road surface defect identification model can be improved. Attached Figure Description

[0051] To make the objectives, technical solutions, and advantages of the invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0052] Figure 1 This is a flowchart of the method;

[0053] Figure 2 This is a schematic diagram of the pavement defect identification model constructed in this invention.

[0054] Figure 3 This is a schematic diagram of deformable convolution in the embodiment;

[0055] Figure 4 This is a schematic diagram of the visual center scheme EVC in the embodiment;

[0056] Figure 5 To verify the sample instances in the dataset used in the experiment. Detailed Implementation

[0057] The following detailed explanation illustrates the specific implementation methods:

[0058] Example:

[0059] like Figure 1 As shown in the figure, this embodiment discloses a pavement defect identification method based on deep learning, including the following steps:

[0060] S1. Construct a pavement defect identification model. For example... Figure 2As shown, the road surface defect identification model is an improvement on the YOLOv7 model. The improvements include: replacing the ordinary convolution in the feature extraction part of the backbone network with deformable convolution; and introducing a spatially explicit visual center scheme EVC into the overall network structure.

[0061] In practice, when replacing the ordinary convolutions in the feature extraction part of the backbone network with deformable convolutions, deformable convolutions are introduced before the ELAN module in YOLOv7, and only the convolutions with a stride of 1 before the ELAN module are replaced. Deformable convolutions are as follows: Figure 3 As shown.

[0062] Conventional convolution operations mainly consist of two steps:

[0063] 1) Sample the input feature map x using a regular grid R;

[0064] 2) Use convolution kernel w to perform weighted operations on the sampling points.

[0065] R defines the size and expansion of the receptive field, as shown in the following formula, which defines a convolutional kernel of size 3×3 with an expansion rate of 1:

[0066] .

[0067] For each location on the output feature map The center position of the convolution kernel can be calculated using the following formula: y( ):

[0068] ;

[0069] in: It is an enumeration of the positions listed in R, representing the offset relative to the center position; w represents the learnable weights.

[0070] In deformable convolution operations, an offset {Δ} is added through a regular grid R. |n= 1,2, , N}, and introduce a weight Δ for each sampling point. So, in the same position... The value of y(p0) becomes as shown in the following equation:

[0071] .

[0072] Considering the significant differences in the shape of road surface defects, with different locations potentially corresponding to defects of varying scales, standard convolutional kernels typically sample at fixed positions on the feature map during feature extraction. This approach suffers from poor adaptability to unknown variations and weak generalization ability. Therefore, this method introduces deformable convolution during feature extraction. This convolution adds two offsets to the sampling positions in the standard convolution operation, learned through parallel convolution. This determines the displacement of the sampling positions on the feature map during the convolution operation. During training, the model adjusts these offsets along the x and y axes using optimization algorithms such as backpropagation and gradient descent. Thus, the model learns how to adjust the sampling positions of the convolution kernels under different conditions, better approximating the shape and size of the defects compared to traditional convolution, thereby extracting input features more effectively. On the other hand, considering that this convolution increases the number of model parameters and computational complexity, this method only replaces the convolution with a stride of 1 before the ELAN module, retaining the convolution with a stride of 2. Because convolutions with a stride of 1 are typically used for dense feature extraction on feature maps, which helps capture the details and local features of the target, while convolutions with a stride of 2 are typically used for downsampling feature maps to increase the receptive field, but this results in a reduction in the size of the feature map and a loss of some detailed information.

[0073] like Figure 4 As shown, the Vision Centering (EVC) scheme includes a Lightweight MLP and a Learnable Vision Centering (LVC) mechanism. The working process of EVC includes: capturing the global long-range dependencies of top-level features through the Lightweight MLP as global information of the detected object; extracting local information features through the LVC; and then concatenating the long-range dependencies captured by the Lightweight MLP and the local information features learned by the LVC along the feature dimension as the output of the EVC vision center.

[0074] In specific implementation, the lightweight MLP consists of two residual modules: a depthwise separable convolutional module and a channel MLP module. The input to the depthwise separable convolutional module is a feature map that has undergone group normalization, and the output of the depthwise separable convolutional module is the input to the channel MLP module. This configuration allows the depthwise separable convolutional module to improve feature representation while reducing computational cost. On the other hand, the channel MLP module, compared to the spatial MLP module, not only effectively reduces computational complexity but also meets the requirements of general vision tasks. The residual modules also include a channel scaling module and a DropPath module, used to perform channel scaling and DropPath operations on the output of the channel MLP module. This improves the generalization performance and robustness of the features.

[0075] The LVC is an encoder with a dictionary, consisting of a codebook B = {b1, b2, ..., b}.n} and a set of visual center scaling factors S = {s1, s2, ..., s} n The LVC process comprises:

[0076] First, a set of convolutional layers is used to process the input features. Encode it and input it into codebook B;

[0077] Then, by using the visual center scaling factor S to make and b k Mapping the corresponding positional information, the information of the entire image about the k-th codeword is calculated using the following formula. Among them, b k Let k be the learnable codeword in codebook B;

[0078] );

[0079] In the formula, For the i-th pixel, Let S be the k-th learnable scaling factor in the visual center scaling factor S, and P be the total number of pixels.

[0080] This setup, by using the square of the Euclidean distance, can find the codeword that best matches the input features, thus mapping the input features into the coding space. Then, multiplying by a scaling factor can adjust the distance metric, ensuring that the calculation of the coding weights can reflect the degree of feature matching well under different conditions. The weights can accurately represent the distribution of input features in the codeword space; the distributed weights are then compared with... Multiplication is used to weight the input features, causing the model to focus more on those features that better match the encoded codewords, i.e., the more important features. This ensures that the obtained... The accuracy and effectiveness.

[0081] Then, the feature information at different scales is integrated and synthesized using the following formula;

[0082] ;

[0083] Then, using the following formula, a fully connected layer and a 1×1 convolutional layer are used to predict prominent key class features, after which the features are activated by the sigmoid function and compared with the input features. Multiplying along the channel dimension improves the network's expressive power;

[0084] ;

[0085] Finally, the local information features learned by LVC are obtained using the following formula:

[0086] .

[0087] The long-range dependencies captured by the lightweight MLP are concatenated with the local information features learned by the visual center mechanism along the feature dimension to obtain the output of the EVC visual center:

[0088] ;

[0089] In the formula, Long-range dependencies captured by a lightweight MLP.

[0090] S2. Obtain a training dataset that includes images of road surface defects and corresponding annotation information.

[0091] In practice, the acquired training dataset is further augmented; this augmentation includes random cropping and rotation. By augmenting the training dataset, the generalization ability of the trained road surface defect identification model can be improved.

[0092] S3. Train the road surface defect identification model using the training dataset.

[0093] In specific implementation, the road surface defect identification model is trained based on an improved loss function RLOSS; the improved loss function RLOSS is:

[0094] ;

[0095] ;

[0096] In the formula, d is the total number of samples; y i The true value of the i-th sample is encoded using one-hot encoding; (This represents the model's prediction for the i-th sample). This means that the predicted value is scaled to the range of 0-1 after being processed by sigmoid, which gives the probability value that the sample belongs to a certain category; the IOU value is used to evaluate the degree of matching between the predicted box and the ground truth box.

[0097] This configuration reduces the loss contribution of easily classified samples (1 − Iou / Iou) while increasing the loss proportion of difficult-to-classify samples, thus addressing the issue of low accuracy due to a small number of positive samples. The LOSS value is negatively correlated with IOU; a higher IOU indicates a more easily distinguishable sample, with 1 − Iou / Iou approaching 0, indicating a smaller contribution to the loss. Conversely, a lower IOU value results in a larger contribution to the loss. By focusing on these samples, the model can better learn and adapt to complex road surface distress scenarios, improving the algorithm's generalization ability.

[0098] S4. Use the trained road surface defect identification model to identify road surface defects.

[0099] The order of S1 and S2 can be interchanged.

[0100] Compared with the prior art, the present invention has the following beneficial effects:

[0101] In this method, the pavement distress identification model is an improvement upon the YOLOv7 model. Considering the significant differences in pavement distress shapes and the potential for distresses of varying scales at different locations, standard convolutional kernels typically sample at fixed positions on the feature map during feature extraction. This approach suffers from poor adaptability to unknown variations and weak generalization ability. Therefore, this method introduces deformable convolution during feature extraction. The model learns how to adjust the sampling position of the convolutional kernel under different conditions, better approximating the shape and size of the distress compared to traditional convolution, thus extracting input features more effectively. Furthermore, existing feature pyramid structures, including the SPPCSPC structure in YOLOv7, primarily focus on information transfer between different levels, neglecting the internal patterns of features within a layer. However, the texture or shape features of certain targets are mainly reflected in the intra-layer feature representation. Therefore, when dealing with targets with significant scale variations, these methods may not fully cover all scale ranges. The pavement defect identification model constructed by this method solves the aforementioned problems by introducing a spatially explicit visual center scheme (EVC) into the overall network structure, ensuring complete coverage of all scale ranges when dealing with targets with significant scale variations. Through the above optimizations, the pavement defect identification model constructed by this method can not only detect and identify multiple defect types, but also exhibits high stability in complex environments and can adapt to various complex detection environments.

[0102] To help those skilled in the art better understand the effectiveness of this method, the following verification experiments are used to illustrate the results.

[0103] Dataset

[0104] The dataset used in the validation experiments is the road distress dataset from the Global Road Distress Detection Challenge organized by the IEEE Big Data International Conference Cup. This dataset contains 47,420 road images from six countries, covering a variety of pavement distress types. This study focuses on four common road distress types: longitudinal cracks (D00), transverse cracks (D10), alligator cracks (D20), and potholes (D40).

[0105] Considering the differences in shooting equipment and road quality across countries and regions, this dataset covers highway and asphalt pavements, as well as diverse background environments such as grasslands, snow cover, water stains, different weather conditions, lighting levels, and shaded environments. The combined effect of these factors presents a significant challenge to road damage detection, but it also makes the dataset more realistic and practical, possessing important research and application value. Figure 5 These are sample instances in the dataset.

[0106] Because the dataset contains a large number of unlabeled images and some images of other types, it was preprocessed first. 23,756 images were finally selected and randomly divided into training, validation and test sets in a ratio of 7:1:2 for subsequent experiments.

[0107] Evaluation indicators

[0108] This study uses precision (P), recall (R), mean average precision (MAP), and F1-score as the main evaluation metrics to comprehensively assess the performance and effectiveness of the pavement distress detection algorithm proposed in this paper.

[0109] ablation experiment

[0110] To investigate the impact of the proposed optimizations on the algorithm's experimental results, seven ablation experiments were conducted on the dataset, and the results are shown in Table 1. Deformable convolution, the EVC visual center scheme, and the RLOSS classification loss function are the proposed improvements; × indicates that the model does not include this improvement, and √ indicates that the model includes it.

[0111] Table 1

[0112]

[0113] Table 2 shows that, using the original YOLOv7 model as the baseline, its detection accuracy was 70.9%, recall was 66%, F1 score was 68.36, and mean precision (MAP) was 70.68%. In the second, third, and fourth sets of experiments, deformable convolution was introduced to enhance the model's adaptability to different road surface defects of different shapes and scales. The EVC visual center mechanism was introduced to solve the problem of corner information loss, and the RLOSS loss function was proposed to solve the problem of imbalance between positive and negative samples in the dataset. All of these measures significantly improved the performance, especially in the last set of comprehensive experiments, where the F1 score increased to 72.1% and the MAP score reached 76.8.

[0114] Table 2

[0115]

[0116] Comparative experiment

[0117] Comparative experiments of different loss functions

[0118] This experiment aims to verify the effectiveness of the newly proposed classification loss function RLOSS in addressing the imbalance between positive and negative samples. RLOSS was compared with other mainstream object detection classification loss functions, including YOLOv7's default binary cross-entropy loss (BCE), focal loss, and GHM-Loss. The comparison results are shown in Table 3. It is clear that RLOSS performs excellently across all performance metrics. Compared to other loss functions, RLOSS exhibits significant advantages, effectively solving the imbalance problem and providing a solid foundation for improving model performance.

[0119] Table 3

[0120]

[0121] Comparative experiments of different models

[0122] To further verify the effectiveness of the present invention, the model trained by the present invention was compared with the current mainstream algorithm model under the same training configuration and dataset. The specific experimental results are shown in Table 4. It can be seen that the model of the present invention is superior to other models in all indicators.

[0123] Table 4

[0124]

[0125] Self-built dataset test comparison experiment

[0126] While the results on large datasets demonstrate the potential and effectiveness of the model presented in this invention, its generalizability and accuracy still require further validation for applications in specific regions or scenarios. Therefore, a custom dataset containing 134 road surface damage images was constructed, primarily covering rural and mountainous road sections in City C. The weights trained on the RDD2022 dataset were directly used for testing, and the experimental results are shown in Table 5. It can be observed that the improved model still exhibits superior performance on the custom dataset.

[0127] Table 5

[0128]

[0129] Overall, the proposed model demonstrates superior performance in multiple aspects, not only detecting various types of road surface defects but also exhibiting good stability in complex environments. It can improve driving safety and comfort in the field of autonomous driving.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit the technical solutions. Those skilled in the art should understand that any modifications or equivalent substitutions to the technical solutions of the present invention without departing from the spirit and scope of the present invention should be covered within the scope of the claims of the present invention.

Claims

1. A method for identifying road surface defects based on deep learning, characterized in that, Includes the following steps: S1. Construct a pavement defect identification model; the pavement defect identification model is an improvement of the YOLOv7 model, the improvement includes: replacing the ordinary convolution in the feature extraction part of the backbone network with deformable convolution; and introducing the spatial explicit visual center scheme EVC into the overall network structure; The Vision Center (EVC) scheme includes a Lightweight MLP and a Learnable Vision Center (LVC) mechanism. The EVC process includes: capturing the global long-range dependencies of top-level features through the Lightweight MLP as global information of the detected object; extracting local information features through the LVC; and then concatenating the long-range dependencies captured by the Lightweight MLP and the local information features learned by the LVC along the feature dimension as the output of the EVC vision center. S2. Obtain a training dataset that includes images of road surface defects and corresponding annotation information; S3. Train the road surface defect identification model using the training dataset; S4. Use the trained pavement defect identification model to identify pavement defects; S1 and S2 can be interchanged in order; In S1, when replacing the ordinary convolution in the feature extraction part of the backbone network with deformable convolution, deformable convolution is introduced before the ELAN module of YOLOv7. In S3, a road surface defect identification model is trained based on an improved loss function RLOSS; the improved loss function RLOSS is: ； ； In the formula, d is the total number of samples; y i The true value of the i-th sample is encoded using one-hot encoding; This represents the model's prediction for the i-th sample; This indicates that the predicted value is scaled to the range of 0-1 after being processed by sigmoid; the IOU value is used to evaluate the degree of matching between the predicted box and the ground truth box.

2. The pavement distress identification method based on deep learning as described in claim 1, characterized in that: When introducing deformable convolutions before the ELAN module in YOLOv7, only the convolution with a stride of 1 before the ELAN module is replaced.

3. The pavement distress identification method based on deep learning as described in claim 1, characterized in that: In S1, the lightweight MLP consists of two residual modules; the residual module includes a depthwise separable convolution module and a channel MLP module; the input of the depthwise separable convolution module is a feature map that has undergone group normalization, and the output of the depthwise separable convolution module is the input of the channel MLP module.

4. The pavement distress identification method based on deep learning as described in claim 3, characterized in that: The residual module also includes a channel scaling module and a DropPath module, which are used to perform channel scaling and DropPath operations on the output of the channel MLP module.

5. The pavement distress identification method based on deep learning as described in claim 1, characterized in that: In S1, the LVC is an encoder with a dictionary, consisting of a codebook B = {b1, b2, ..., b...} n } and a set of visual center scaling factors S = {s1, s2, ..., s} n }composition; The LVC processing procedure includes: First, a set of convolutional layers is used to process the input features. Encode it and input it into codebook B; Then, by using the visual center scaling factor S to make and b k By mapping the corresponding positional information, we can obtain the information of the entire image regarding the k-th codeword in codebook B. Among them, b k Let k be the learnable codeword in codebook B; Then, the feature information at different scales is integrated and synthesized using the following formula; ； Then, using the following formula, a fully connected layer and a 1×1 convolutional layer are used to predict prominent key class features, after which the features are activated by the sigmoid function and compared with the input features. Multiplying along the channel dimension improves the network's expressive power; ; Finally, the local information features learned by LVC are obtained using the following formula: 。 6. The pavement distress identification method based on deep learning as described in claim 5, characterized in that: The encoded features are then processed using the visual center scaling factor S. and b k When mapping the corresponding position information, the information of the entire image about the k-th codeword is calculated using the following formula. : ）； In the formula, For the i-th pixel, Let S be the k-th learnable scaling factor in the visual center scaling factor S, and P be the total number of pixels.

7. The pavement distress identification method based on deep learning as described in claim 1, characterized in that: In S2, the acquired training dataset is further augmented; the augmentation includes random cropping and rotation.

Images