A road surface defect detection method based on MSE-DETR

Abstract

The application provides a road surface defect detection method based on MSE-DETR, aiming at the problems of edge detail information loss, insufficient feature extraction and the like in road surface defect detection, and utilizing an efficient state space fusion module ESSF, a multi-scale feedforward enhancement network MFEN and a multi-pooling feature modulation module MPFM, wherein the efficient state space fusion module ESSF is combined with C2f to form C2f-ESSF as a backbone network, feature interaction is enhanced, and redundant parameters and calculation amount are reduced; the multi-scale feedforward enhancement network MFEN is added to an encoding layer, the global and local modeling capabilities of the model are strengthened, the contrast perception between defects and backgrounds is enhanced, and thus outstanding defect features are obtained in a complex environment; and the multi-pooling feature modulation module MPFM differentiates and aggregates features in a multi-scale manner, effectively enhances the representation capability of the model for defects of different sizes and different textures, and retains the edge details of defects such as cracks and potholes.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and deep learning technology, and in particular relates to a road surface defect detection method based on MSE-DETR. Background Technology

[0002] With China's rapid economic development and steady urbanization, the total length of highways has continued to grow. By the end of 2024, the national highway mileage reached 5.4904 million kilometers, an increase of 53,500 kilometers from the end of the previous year. Roads, as a crucial component of transportation infrastructure, bear the responsibility of freight transport and residents' daily travel, and are a vital link in ensuring the normal operation of cities. However, due to increased traffic loads, climate change, and natural aging, surface defects such as cracks and potholes have appeared on road surfaces. If these defects are not addressed promptly, they will become more severe, not only affecting residents' travel experience but also increasing the likelihood of traffic accidents, causing injuries, fatalities, and property damage. Therefore, accurate detection of road surface defects helps ensure road safety, extend road lifespan, improve road repair efficiency, and is of great significance for promoting the development of intelligent transportation.

[0003] Currently, although road defect detection technology has improved detection performance, feature extraction is still insufficient when faced with complex road conditions such as large variations in defect size, irregular potholes, and background interference. This results in low detection accuracy and a tendency for false detections and missed detections. Summary of the Invention

[0004] To address the shortcomings and deficiencies of existing technologies, a road surface defect detection method based on MSE-DETR is provided. This method can solve the problems of insufficient feature extraction, low detection accuracy, and false detections and missed detections that exist in existing road surface defect detection methods when faced with complex road surface conditions such as large variations in defect size, irregular potholes, and background interference.

[0005] To achieve the objectives of this invention, a road surface defect detection method based on MSE-DETR is provided, comprising the following steps:

[0006] S1: Select road surface defect images from the global road damage detection challenge dataset RDD2022 and annotate the defects to form a dataset. The dataset contains five different types of road defects: longitudinal cracks, transverse cracks, grid cracks, potholes, and road repair.

[0007] S2: The Efficient State-Space Fusion (ESSF) module is introduced into C2f to form the C2f-ESSF module, which constructs a new backbone network. The road defect images in the dataset are input into the backbone network. They first go through a convolution, and then through four stages. Each stage consists of a 3×3 convolution and the C2f-ESSF module. The convolution is used for spatial downsampling and channel number adjustment to gradually extract multi-scale features. C2f-ESSF fuses global and local information and finally outputs the enhanced feature representation, thereby improving the network's information perception ability, enhancing long-range modeling ability, and reducing computational complexity.

[0008] S3: The enhanced feature representation obtained in S2 is enhanced by using the Multiscale Feedforward Enhancement Network (MFEN) to strengthen the local geometric details of the defect, thereby obtaining features with edge information and enhancing the model's perception of low-contrast defects.

[0009] S4: The Multi-Pooling Feature Modulation (MPFM) module is used to enhance the multi-scale defect perception of the features with edge information obtained in S3, thereby reducing the loss of details and improving detection accuracy and robustness.

[0010] The beneficial effects of this invention are:

[0011] Compared with existing technologies, this invention provides a road surface defect detection method based on MSE-DETR, addressing issues such as loss of edge detail information and insufficient feature extraction in road surface defect detection. It utilizes an efficient state-space fusion module (ESSF), a multi-scale feedforward enhancement network (MFEN), and a multi-pooling feature modulation module (MPFM). The ESSF is combined with C2f to form C2f-ESSF as the backbone network, enhancing feature interaction and reducing redundant parameters and computational load. The MFEN is added to the encoding layer, strengthening the model's global and local modeling capabilities and enhancing the contrast perception between defects and the background, thus obtaining prominent defect features in complex environments. The MPFM performs differentiated feature aggregation at multiple scales, effectively enhancing the model's representation of defects of different sizes and textures, while preserving edge details of defects such as cracks and potholes.

[0012] Verification shows that the road surface defect detection method provided by this invention significantly improves the accuracy of road surface defect detection compared to existing technologies, achieving a detection accuracy of 88.8%. Simultaneously, optimized computational efficiency and real-time performance meet the requirements, enabling the method to be deployed even under conditions of limited storage and computation. Attached Figure Description

[0013] Figure 1 This is a flowchart of the present invention;

[0014] Figure 2 This is an example diagram of the dataset used in S1 of the present invention;

[0015] Figure 3 This is a general framework diagram of the MSE-DETR method of the present invention;

[0016] Figure 4 This is a schematic diagram of the C2f-ESSF and ESSF modules in the method of the present invention;

[0017] Figure 5 This is a schematic diagram of the HSM-SSD module in the method of the present invention;

[0018] Figure 6 This is a schematic diagram of the MFEN module in the method of the present invention;

[0019] Figure 7 This is a schematic diagram of the MSMHSA module in the method of the present invention;

[0020] Figure 8 This is a schematic diagram of the FEN module in the method of the present invention;

[0021] Figure 9 This is a schematic diagram of the MPFM module in the method of the present invention;

[0022] Figure 10 This is a schematic diagram of spatial attention in the method of the present invention;

[0023] Figure 11 This is a schematic diagram of the PSConv module in the method of the present invention;

[0024] Figure 12 This is a diagram showing the detection results of the method of the present invention. Detailed Implementation

[0025] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings:

[0026] according to Figure 1 As shown, this invention provides a road surface defect detection method based on MSE-DETR, comprising the following steps:

[0027] S1: Select pavement defect images from the global road damage detection challenge dataset RDD2022 and annotate them to form a dataset, such as... Figure 2 As shown, the dataset contains five different types of road defects: longitudinal cracks, transverse cracks, grid cracks, potholes, and road repair.

[0028] The road surface defect images selected from the global road damage detection challenge dataset RDD2022 include various scenes such as shadows, occlusions, and soil coverage, resulting in a total of 4,878 images. These 4,878 images include 2,477 images taken by a motorcycle-mounted camera and 2,401 images taken by a drone-mounted camera, to ensure the balance of the road surface defect images.

[0029] S2: As Figure 3 , Figure 4 As shown, the Efficient State-Space Fusion (ESSF) module is introduced into C2f to form the C2f-ESSF module, constructing a new backbone network. Road surface defect images from the dataset are input into the backbone network, first undergoing a convolution, then four stages. Each stage consists of a 3×3 convolution and a C2f-ESSF module. The convolution is used for spatial downsampling and channel number adjustment, progressively extracting multi-scale features. C2f-ESSF fuses global and local information, ultimately outputting an enhanced feature representation, thereby improving the network's information perception capability, enhancing long-range modeling ability, and reducing computational complexity. Specifically:

[0030] (1) The efficient state-space fusion module ESSF consists of sequentially stacked state-space dual layers HSM-SSD (Hidden State Mixer-based State Space Duality) and a feed-forward network FFN; such as Figure 5 As shown, the state space dual layer HSM-SSD transfers computationally intensive operations from the original token space to the compressed hidden state space, thereby reducing computational complexity while maintaining global modeling capabilities; the feedforward network FFN uses two consecutive 1×1 convolutions to enhance the interaction between channels; a 3×3 depthwise convolution is embedded between HSM-SSD and FFN, and each sub-module uses residual connections to promote gradient flow and training stability; the feedforward network FFN is followed by an SE attention mechanism to calibrate the original feature map and dynamically adjust the influence of each channel.

[0031] The working process of the state space dual layer HSM-SSD can be summarized as follows:

[0032] Step 1: Given the input sequence, generate the result through a linear transformation. , and The weights are then obtained through a discretization step. , The details are as follows:

[0033] ;

[0034] ;

[0035] in, Indicates the input feature map, Indicates the learning parameters;

[0036] Step two, using and We perform weighted aggregation on the input to obtain the compressed global hidden state. This compresses the original sequence into a compact latent representation. The details are as follows:

[0037] ;

[0038] ;

[0039] ;

[0040] in, This indicates element-wise multiplication;

[0041] Step 3: Perform gating and linear projection operations within the compressed hidden state space to achieve information exchange between channels, utilizing... Project the updated hidden state back into the original space to obtain the final output. The details are as follows:

[0042] ;

[0043] ;

[0044] in, This represents the Sigmoid activation function.

[0045] (2) C2f is used to enhance feature reuse and gradient flow, including: CBS module and bottleneck module Bottleneck; wherein, the CBS module is used for preliminary feature extraction, and the CBS module includes: convolutional layer Conv, batch normalization layer BN and activation function SiLU; the bottleneck module Bottleneck is composed of two CBS modules and is used for feature extraction and residual connection.

[0046] (3) The C2f-ESSF module replaces Bottleneck in C2f with the ESSF module to enhance long-range modeling capabilities.

[0047] (4) The four stages refer to: each stage consists of 3×3 convolution and C2f-ESSF module. Each stage starts with 3×3 convolution, which is used for spatial downsampling and channel adjustment to extract multi-scale features. The extracted multi-scale features are then used for state modeling through C2f-ESSF to integrate global and local information and enrich the details.

[0048] S3: The enhanced feature representation obtained in S2 is reinforced using a multiscale feedforward enhancement network (MFEN) to enhance the local geometric details of the defects, resulting in features with edge information and improving the model's perception of low-contrast defects. Wherein:

[0049] like Figure 6 As shown, the Multi-Scale Multi-Head Self-Attention Network (MFEN) includes a Multi-Scale Multi-Head Self-Attention Network (MSMHSA) and a Feedforward Enhancement Network (FEN). The MSMHSA processes the original features in the input feature map to obtain enhanced features, which are used to enhance the model's perception ability. The enhanced features and the original features are then input together into the Feedforward Enhancement Network (FEN) to inject spatial information, improve the discriminative ability of the enhanced features, and supplement edge texture details, so that the model can obtain higher recognition accuracy under low contrast conditions.

[0050] ①For example Figure 7 As shown, the specific steps of Multi-Scale Multi-Head Attention (MSMHSA) in processing the original features in the input feature map to obtain the enhanced features are as follows:

[0051] Step 1: Given input features Through 1 One convolution is performed, reducing its channel count by a factor of 4, and then fed into three parallel dilated depthwise convolutions, thus dividing it into three distinct branches; the dilated depthwise convolution kernel size in each branch is 3. 3. The void ratios were set to 3, 5, and 7 respectively to capture the local context of different receptive fields and extract multi-scale information, as detailed below:

[0052] ;

[0053] Where, represents the features after dilated depth convolution;

[0054] Step 2: After dilated depthwise convolution, each branch is then processed through an independent 1-bit convolution. 1. Convolution restores the number of channels to the original dimensions and sums and merges the outputs of the three branches, as follows:

[0055] ;

[0056] ;

[0057] in, This represents the features output from the three branches. Indicates the summation result;

[0058] Step 3: The fused features are processed by 1 1. Convolution and Original Input Multiplying each element together, we obtain the output result as follows:

[0059] ;

[0060] in, This indicates element-wise multiplication. Indicates the output result;

[0061] Step 4: Feed the output results into the AdaptivePool layer for processing:

[0062] ;

[0063] in, As a multi-scale feature, it preserves rich multi-scale contextual information of the input, serving as input for multi-head self-attention.

[0064] ②Querying Multi-Scale Multi-Head Attention (MSMHSA) Taken from original features , and key Sum This comes from multi-scale features. :

[0065] ;

[0066] in, , These represent the linear transformation matrices used to generate the query, key, and value tensors, respectively.

[0067] Then , , The features are then fed into the multi-head attention module for computation, as follows:

[0068] ;

[0069] in, yes The channel dimension; Softmax is used to calculate the weights.

[0070] ③ For example Figure 8 As shown, the specific steps of the feedforward enhancement network (FEN) to inject spatial information, improve the discriminative ability of the enhanced features, and supplement edge texture details are as follows:

[0071] Step 1: Convert the feature map processed by the MSMHSA module Through a 1 1. Convolution is divided into branches , and Different features are processed separately, while the initial features are... The receptive field is expanded using average pooling (AvgPool), and two Conv-LN-ReLU blocks are used to capture a wider range of spatial information, as detailed below:

[0072] ;

[0073] ;

[0074] Here, CLR represents two Conv-LN-ReLU blocks, LN represents layer normalization, and ReLU represents the activation function. This represents the features after average pooling; This represents the features after ReLU;

[0075] Step 2: Use Upsample, 3 3 depthwise convolution and 1 The efficient up-convolution block (EUCB), composed of 1 convolutional layers, upsamples the features and combines them with... splicing and fusion, then through a 1 1 convolution and 3 3. Depthwise convolution generates branching results The details are as follows:

[0076] ;

[0077] ;

[0078] Where UP stands for Upsample and BN stands for Batch Normalization. Indicates the characteristics after EUCB;

[0079] Step 3: Expand the receptive field using a convolution with a dilation rate of 2 to obtain the branching results. Asymmetric convolution (AsymConv) is used to reduce the number of parameters and better capture branching results in the image. The details are as follows:

[0080] ;

[0081] ;

[0082] in, Indicates dilated convolution;

[0083] Step 4: Branch Results Parallel branches are generated through two convolutions: one branch uses depthwise convolution with residual concatenation to backflush gradients, employing GELU as the activation function to control information flow; it is then element-wise multiplied with the other branch, and the product is further processed by 1... 1. Convolution refines features and fuses channel information;

[0084] Step 5: Combine the results of the three branches , , The features are then concatenated and output to further enhance the model's spatial awareness.

[0085] S4: The Multi-Pooling Feature Modulation (MPFM) module is used to enhance the multi-scale defect perception of the features with edge information obtained in S3, reducing detail loss and improving detection accuracy and robustness. Wherein:

[0086] like Figure 9 As shown, the Multi-Pool Feature Modulation Module (MPFM) is sequentially configured with channel attention, channel shuffling operation, spatial attention mechanism, and feature modulation module;

[0087] ①The specific steps of the channel attention are as follows:

[0088] Step 1: Process the input feature map Perform dimensional replacement, from Convert to This places the channel dimension last; the process is as follows:

[0089] ;

[0090] Where Permute represents dimension permutation. Represents the features after dimension permutation;

[0091] Step 2: Apply the perceptron MLP to the dimensionality-permutated features Nonlinear transformation, feature enhancement, and correlation modeling between channels are performed: first, the number of channels is reduced, then nonlinearity is introduced using the ReLU activation function, and then the channel dimension is restored to its original size, so that the model can learn the relationship between channels and unique feature information while preserving spatial information.

[0092] ;

[0093] Step 3: Restore the feature map to its original dimension through inverse permutation operation, and obtain the final channel attention weights using the Sigmoid activation function. The process is as follows:

[0094] ;

[0095] in, This represents the Sigmoid activation function;

[0096] Step 4: Input Features With channel attention weights Element-wise multiplication yields the enhanced output feature map. :

[0097] ;

[0098] in, This indicates element-wise multiplication;

[0099] ②For example Figure 10 As shown, the specific steps of the channel shuffling operation are as follows:

[0100] Step 1: Enhance the feature map Divided into 4 groups, each group contains One channel;

[0101] Step 2: The grouped feature maps undergo a transpose operation to shuffle the channel order;

[0102] Step 3: Restore the shuffled feature map to its original shape:

[0103] ;

[0104] in, This indicates a channel shuffling operation. This represents a feature map after shredding.

[0105] ③ The specific steps of the spatial attention mechanism are as follows:

[0106] Step 1: Transfer the feature map The signal is fed into the spatial attention, where three types of pooling are fused in the spatial dimension: Power Average Pooling (PAP), Detail-Preserving Pooling (DPP), and Local Importance Pooling.

[0107] ;

[0108] ;

[0109] ;

[0110] Step 2: Perform bilinear interpolation upsampling on the pooling results to restore the output of each pooling branch to its original spatial size, facilitating subsequent weighted operations. The output result is... , , Simultaneously, the feature map is input into the Gumbel-Softmax router to generate weights, which are then weighted and fused together with the three pooling branches, enabling the model to dynamically adjust the contribution level of each pooling branch; the process is as follows:

[0111] ;

[0112] ;

[0113] ;

[0114] in, Represents the weight vector; This represents the weighted feature map;

[0115] Step 3: The fused features are obtained through a 7 A 7×7 convolutional layer reduces the number of channels to one-quarter of the original and performs a non-linear transformation on the feature map using batch normalization (BN) and the ReLU activation function. Next, a second 7×7 convolutional layer restores the number of channels to the original dimension. After batch normalization to stabilize the training process, spatial attention weights are generated using the Sigmoid activation function.

[0116] ;

[0117] in, This represents the Sigmoid activation function;

[0118] Step 4: Multiply the shuffled feature map element-wise with the weights to obtain the output feature map:

[0119] ;

[0120] ④ The specific steps of the feature modulation module are as follows:

[0121] Step 1: Process the output feature map obtained by spatial attention through two parallel branches. One branch serves as the local information branch, capturing details and local spatial features in the enhanced feature map. The other branch serves as the global information branch, responsible for extracting global context and long-range dependencies.

[0122] Step Two: As Figure 11 As shown, the local information branch first passes through PSConv, which uses asymmetric padding to construct horizontal and vertical convolution kernels, allowing them to be applied to different parts of the image and expanding the network's perception range for minute defects. Then, the features are added to the input and convolution output through residual concatenation. PSConv, with its windmill-shaped receptive field and fewer parameters, enhances the model's attention to subtle features, thereby better preserving the detailed information of small targets.

[0123] Step 3: The global information branch transforms the input features through a convolution to model global semantic information;

[0124] Step 4: The local information branch is activated by the GELU function and then multiplied element-wise with the global feature to achieve the fusion of local and global information. The fused feature is then added to the spatial attention output feature Z through residual concatenation to ensure that no information is lost.

[0125] Step 5: Connect the final output map to the initial input features via residual connections. Adding them together preserves the original input information during feature transformation, ensuring effective gradient propagation and promoting model training stability and performance improvement.

[0126] The method proposed in this invention is developed based on Python 3.9. The experimental environment is configured with Ubuntu 20.04 as the operating system and PyTorch as the deep learning framework. For hardware, an Intel® Xeon® Platinum 8380 CPU and an NVIDIA L40 GPU with 48GB of video memory are used. In a specific implementation dataset, a selected dataset is used. The method is compared with representative object detection algorithms on this dataset to verify the effectiveness of the proposed model.

[0127] Table 1

[0128] Faster R-CNN 78.4 136.7 330.0 33 YOLOv8s 85.6 11.13 25.4 145 YOLOv10s 85.0 7.25 21.4 182 YOLOv12s 85.7 9.23 21.2 132 Mamba-YOLO-T 86.5 5.98 13.6 98 RT-DETR-R18 86.3 19.88 57.0 143 MSE-DETR 88.8 19.38 51.3 159

[0129] The MSE-DETR method proposed in this invention outperforms existing mainstream models in several key performance indicators. Table 1 compares the speed and performance of the proposed method with mainstream algorithms. As shown in Table 1, the mAP50 of the proposed MSE-DETR method reaches 88.8%, a 2.5% improvement over the benchmark model RT-DETR-R18. The YOLO series has advantages in parameter quantity and computational complexity, but their detection accuracy is lower than that of MSE-DETR. Mamba-YOLO-T maintains low computational efficiency while exhibiting good detection performance, but its mAP50 is 2.3% lower than that of MSE-DETR, indicating a significant gap. Faster R-CNN has much lower detection accuracy than this method, and its parameter quantity and computational complexity are higher than MSE-DETR, resulting in higher computational resource consumption. Therefore, the proposed MSE-DETR method demonstrates good performance in pavement defect detection, achieving a good balance between performance and efficiency.

[0130] To further verify the advantages of MSE-DETR in visualization, the detection results of different models on the same image are shown in the following figure. Figure 12 .pass Figure 12 As can be seen, MSE-DETR performed best in the task, outperforming other models in detection results due to its superior performance. It exhibited no missed detections or redundant bounding boxes and demonstrated high confidence. RT-DETR, Mamba-YOLO, and YOLOv12s, however, displayed redundant bounding boxes and missed detections in complex environments with dense defects, pedestrians, and gravel, indicating their difficulty in accurately distinguishing subtle features. These results demonstrate the effectiveness of MSE-DETR in improving detection accuracy, exhibiting lower false positive and false negative rates compared to other models, and achieving stable and efficient road surface defect detection.

[0131] The above embodiments are not limited to the technical solutions of the embodiments themselves, and the embodiments can be combined with each other to form new embodiments. The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of the technical solutions of the present invention.

Claims

1. A road surface defect detection method based on MSE-DETR, characterized in that: Includes the following steps: S1: Select road surface defect images from the global road damage detection challenge dataset RDD2022 and annotate the defects to form a dataset. The dataset contains five different types of road defects: longitudinal cracks, transverse cracks, grid cracks, potholes, and road repair. S2: The Efficient State-Space Fusion (ESSF) module is introduced into C2f to form the C2f-ESSF module, which constructs a new backbone network. The road defect images in the dataset are input into the backbone network. They first go through a convolution, and then through four stages. Each stage consists of a 3×3 convolution and the C2f-ESSF module. The convolution is used for spatial downsampling and channel number adjustment to gradually extract multi-scale features. C2f-ESSF fuses global and local information and finally outputs the enhanced feature representation, thereby improving the network's information perception ability, enhancing long-range modeling ability, and reducing computational complexity. S3: The enhanced feature representation obtained in S2 is enhanced by using a multiscale feedforward enhancement network (MFEN) to strengthen the local geometric details of the defect, thereby obtaining features with edge information and enhancing the model's perception of low-contrast defects. S4: The Multi-Pooling Feature Modulation (MPFM) module is used to enhance the multi-scale defect perception of the features with edge information obtained in S3, thereby reducing the loss of details and improving detection accuracy and robustness.

2. The road surface defect detection method based on MSE-DETR according to claim 1, characterized in that: The S1 image selection from the global road damage detection challenge dataset RDD2022 includes various scenes such as shadows, occlusions, and soil coverage, resulting in a total of 4878 images. These 4878 images include 2477 images taken by a motorcycle-mounted camera and 2401 images taken by a drone-mounted camera, to ensure the balance of the road damage images.

3. The road surface defect detection method based on MSE-DETR according to claim 2, characterized in that: The efficient state-space fusion module ESSF in S2 consists of sequentially stacked Hidden State Mixer-based State Space Duality (HSM-SSD) layers and a Feed-Forward Network (FFN). The HSM-SSD layer transfers computationally intensive operations from the original token space to the compressed hidden state space, thereby reducing computational complexity while maintaining global modeling capabilities. The FFN employs two consecutive 1×1 convolutions to enhance the interaction between channels. A 3×3 depthwise convolution is embedded between the HSM-SSD and FFN, and each submodule uses residual connections to promote gradient flow and training stability. The FFN is followed by an SE attention mechanism to calibrate the original feature map and dynamically adjust the influence of each channel.

4. The road surface defect detection method based on MSE-DETR according to claim 3, characterized in that: The working process of the state space dual layer HSM-SSD can be summarized as follows: Step 1: Given the input sequence, generate the result through a linear transformation. , and The weights are then obtained through a discretization step. , The details are as follows: ； ； in, Indicates the input feature map, Indicates the learning parameters; Step two, using and We perform weighted aggregation on the input to obtain the compressed global hidden state. This compresses the original sequence into a compact latent representation. The details are as follows: ； ； ； in, This indicates element-wise multiplication; Step 3: Perform gating and linear projection operations within the compressed hidden state space to achieve information exchange between channels, utilizing... Project the updated hidden state back into the original space to obtain the final output. The details are as follows: ； ； in, This represents the Sigmoid activation function.

5. The road surface defect detection method based on MSE-DETR according to claim 4, characterized in that: The C2f in S2 is used to enhance feature reuse and gradient flow, including: a CBS module and a bottleneck module Bottleneck; wherein, the CBS module is used for preliminary feature extraction, and the CBS module includes: a convolutional layer Conv, a batch normalization layer BN, and an activation function SiLU; the bottleneck module Bottleneck is composed of two CBS modules and is used for feature extraction and residual connection. The C2f-ESSF module in S2 replaces Bottleneck in C2f with the ESSF module to enhance long-range modeling capabilities. The four stages in S2 refer to: each stage consists of a 3×3 convolution and a C2f-ESSF module. Each stage starts with a 3×3 convolution for spatial downsampling and channel adjustment to extract multi-scale features. The extracted multi-scale features are then processed by C2f-ESSF for state modeling, integrating global and local information to enrich the details.

6. The road surface defect detection method based on MSE-DETR according to claim 5, characterized in that: The multi-scale feedforward enhancement network MFEN in S3 includes a multi-scale multi-head self-attention network MSMHSA and a feedforward enhancement network FEN. The multi-scale multi-head self-attention network MSMHSA processes the original features in the input feature map to obtain enhanced features, which are used to enhance the model's perception ability. The enhanced features are then input together with the original features into the feedforward enhancement network (FEN) to inject spatial information, improve the discriminative ability of the enhanced features and supplement edge texture details, so that the model can achieve high recognition accuracy in low contrast conditions.

7. The road surface defect detection method based on MSE-DETR according to claim 6, characterized in that: The specific steps of the Multi-Scale Multi-Head Attention (MSMHSA) method to process the original features in the input feature map to obtain the enhanced features are as follows: Step 1: Given input features Through 1 One convolution is performed, reducing its channel count by a factor of 4, and then fed into three parallel dilated depthwise convolutions, thus dividing it into three distinct branches; the dilated depthwise convolution kernel size in each branch is 3.

3. The void ratios were set to 3, 5, and 7 respectively to capture the local context of different receptive fields and extract multi-scale information, as detailed below: ； Where, represents the features after dilated depth convolution; Step 2: After dilated depthwise convolution, each branch is then processed through an independent 1-bit convolution.

1. Convolution restores the number of channels to the original dimensions and sums and merges the outputs of the three branches, as follows: ； ； in, This represents the features output from the three branches. Indicates the summation result; Step 3: The fused features are processed by 1 1. Convolution and Original Input Multiplying each element together, we obtain the output result as follows: ； in, This indicates element-wise multiplication. Indicates the output result; Step 4: Feed the output results into the AdaptivePool layer for processing: ； in, As a multi-scale feature, it preserves rich multi-scale contextual information of the input, serving as input for multi-head self-attention.

8. The road surface defect detection method based on MSE-DETR according to claim 7, characterized in that: The query of the multi-scale multi-head attention MSMHSA Taken from original features , and key Sum This comes from multi-scale features. : ； in, , These represent the linear transformation matrices used to generate the query, key, and value tensors, respectively. Then , , The features are then fed into the multi-head attention module for computation, as follows: ； in, yes The channel dimension; Softmax is used to calculate the weights.

9. A road surface defect detection method based on MSE-DETR according to claim 8, characterized in that: The specific steps of the feedforward enhancement network (FEN) injecting spatial information to improve the discriminative ability of the enhanced features and supplement edge texture details are as follows: Step 1: Convert the feature map processed by the MSMHSA module Through a 1 1. Convolution is divided into branches , and Different features are processed separately, while the initial features are... The receptive field is expanded using average pooling (AvgPool), and two Conv-LN-ReLU blocks are used to capture a wider range of spatial information, as detailed below: ； ； Here, CLR represents two Conv-LN-ReLU blocks, LN represents layer normalization, and ReLU represents the activation function. This represents the features after average pooling; This represents the features after ReLU; Step 2: Use Upsample, 3 3 depthwise convolution and 1 The efficient up-convolution block (EUCB), composed of 1 convolutional layers, upsamples the features and combines them with... splicing and fusion, then through a 1 1 convolution and 3 3. Depthwise convolution generates branching results The details are as follows: ； ； Where UP stands for Upsample and BN stands for Batch Normalization. Indicates the characteristics after EUCB; Step 3: Expand the receptive field using a convolution with a dilation rate of 2 to obtain the branching results. Asymmetric convolution (AsymConv) is used to reduce the number of parameters and better capture branching results in the image. The details are as follows: ； ； in, Indicates dilated convolution; Step 4: Branch Results Parallel branches are generated through two convolutions: one branch uses depthwise convolution with residual concatenation to backflush gradients, employing GELU as the activation function to control information flow; it is then element-wise multiplied with the other branch, and the product is further processed by 1...

1. Convolution refines features and fuses channel information; Step 5: Combine the results of the three branches , , The features are then concatenated and output to further enhance the model's spatial awareness.

10. A road surface defect detection method based on MSE-DETR according to claim 9, characterized in that: The multi-pooling feature modulation module (MPFM) in S4 is sequentially configured with channel attention, channel shuffling operation, spatial attention mechanism and feature modulation module; ①The specific steps of the channel attention are as follows: Step 1: Process the input feature map Perform dimensional replacement, from Convert to This places the channel dimension last; the process is as follows: ； Where Permute represents dimension permutation. Represents the features after dimension permutation; Step 2: Apply the perceptron MLP to the dimensionality-permutated features Nonlinear transformation, feature enhancement, and correlation modeling between channels are performed: first, the number of channels is reduced, then nonlinearity is introduced using the ReLU activation function, and then the channel dimension is restored to its original size, so that the model can learn the relationship between channels and unique feature information while preserving spatial information. ； Step 3: Restore the feature map to its original dimension through inverse permutation operation, and obtain the final channel attention weights using the Sigmoid activation function. The process is as follows: ； in, This represents the Sigmoid activation function; Step 4: Input Features With channel attention weights Element-wise multiplication yields the enhanced output feature map. : ； in, This indicates element-wise multiplication; ②The specific steps of the channel shuffling operation are as follows: Step 1: Enhance the feature map Divided into 4 groups, each group contains One channel; Step 2: The grouped feature maps undergo a transpose operation to shuffle the channel order; Step 3: Restore the shuffled feature map to its original shape: ； in, This indicates a channel shuffling operation. This represents a feature map after shredding. ③ The specific steps of the spatial attention mechanism are as follows: Step 1: Transfer the feature map The signal is fed into the spatial attention, where three types of pooling are fused in the spatial dimension: Power Average Pooling (PAP), Detail-Preserving Pooling (DPP), and Local Importance Pooling. ； ； ； Step 2: Perform bilinear interpolation upsampling on the pooling results to restore the output of each pooling branch to its original spatial size, facilitating subsequent weighted operations. The output result is... , , Simultaneously, the feature map is input into the Gumbel-Softmax router to generate weights, which are then weighted and fused together with the three pooling branches, enabling the model to dynamically adjust the contribution level of each pooling branch; the process is as follows: ； ； ； in, Represents the weight vector; This represents the weighted feature map; Step 3: The fused features are obtained through a 7 A 7×7 convolutional layer reduces the number of channels to one-quarter of the original and performs a non-linear transformation on the feature map using batch normalization (BN) and the ReLU activation function. Next, a second 7×7 convolutional layer restores the number of channels to the original dimension. After batch normalization to stabilize the training process, spatial attention weights are generated using the Sigmoid activation function. ； in, This represents the Sigmoid activation function; Step 4: Multiply the shuffled feature map element-wise with the weights to obtain the output feature map: ； ④ The specific steps of the feature modulation module are as follows: Step 1: Process the output feature map obtained by spatial attention through two parallel branches. One branch serves as the local information branch, capturing details and local spatial features in the enhanced feature map. The other branch serves as the global information branch, responsible for extracting global context and long-range dependencies. Step 2: The local information branch first passes through PSConv, which uses asymmetric padding to construct horizontal and vertical convolution kernels, allowing them to be applied to different parts of the image and expanding the network's perception range for minute defects. Then, the features are added to the input and convolution output through residual concatenation. PSConv, with its windmill-shaped receptive field and fewer parameters, enhances the model's attention to subtle features, thus better preserving the detailed information of small targets. Step 3: The global information branch transforms the input features through a convolution to model global semantic information; Step 4: The local information branch is activated by the GELU function and then multiplied element-wise with the global feature to achieve the fusion of local and global information. The fused feature is then added to the spatial attention output feature Z through residual concatenation to ensure that no information is lost. Step 5: Connect the final output map to the initial input features via residual connections. Adding them together preserves the original input information during feature transformation, ensuring effective gradient propagation and promoting model training stability and performance improvement.

Images