Lightweight road defect detection method based on improved RT-DETR

Abstract

The improved RT-DETR-based lightweight road defect detection method belongs to the technical field of target detection, and aims to solve the problem that the existing method is difficult to meet the dual requirements of high-precision capture of small targets and slender defects and lightweight easy deployment. The method comprises the following steps: dividing a road defect image dataset into a training set, a verification set and a test set; constructing an improved RT-DETR model, replacing a scale-in feature interaction module with a polarity perception frequency modulation PAFM module, replacing an original backbone with a CSP-SSD lightweight backbone network, and introducing a shift convolution feature extraction SCFE module to expand a receptive field; inputting a road image after training to output defect categories and position information. Through testing, on an RDD2022 dataset, mAP50 reaches 78.0%, mAP50:95 reaches 50.2%, which is improved by 2.1 and 2.6 percentage points compared with a baseline, the parameter quantity is reduced by 27.1%, and the calculation amount is reduced by 17.2%.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a lightweight road defect detection method based on an improved RT-DETR. Background Technology

[0002] Under the combined effects of repeated vehicle loads and natural environmental erosion, road surfaces gradually develop various defects such as cracks and potholes. If these defects are not detected and addressed in a timely manner, they can quickly expand into structural damage, significantly increasing road maintenance costs and even threatening driving safety.

[0003] Deep learning-based road defect detection methods, with their strong feature self-learning capabilities, have become the mainstream technology. These methods are mainly divided into two categories: those based on Convolutional Neural Networks (CNNs) and those based on Transformers. CNN-based methods (such as the YOLO series) extract features through the local receptive fields of convolutional kernels, offering advantages in real-time performance, but they struggle to effectively model the long-range spatial dependencies of defects such as thin cracks. Transformer-based methods model global dependencies through self-attention mechanisms, offering advantages in detection accuracy. The RT-DETR (Real-Time Detection Transformer) achieves real-time detection through a simplified design. To fully leverage its advantages in road defect detection, road defect detection methods based on improvements to the RT-DETR structure have been continuously proposed in recent years.

[0004] However, RT-DETR still has the following shortcomings in road defect detection scenarios: First, the computational overhead of Softmax self-attention in its attention-based Intra-scale Feature Interaction (AIFI) module is relatively large; Second, although existing linear attention reduces the computational complexity to O(N), approximating Softmax with a non-negative kernel function will lose negative interaction information, resulting in a smoother attention distribution and reducing the model's ability to distinguish small defects; Third, its backbone network (such as ResNet) has a large number of parameters and a limited receptive field; Fourth, the large number of model parameters and computational cost is not conducive to deployment on vehicle-mounted or drone-based devices.

[0005] To address the aforementioned shortcomings of RT-DETR, existing technologies have proposed various improvement schemes based on RT-DETR, but these problems remain difficult to solve.

[0006] For example, CN119784759A discloses a lightweight road damage detection method based on RT-DETR improvement. The backbone network of RT-DETR is replaced by a multi-scale edge information enhancement module, and the hybrid encoder is replaced by an AIFI and a multi-branch dilated convolutional pyramid network module to enhance multi-scale and edge information. However, this scheme still retains AIFI based on Softmax self-attention to process deep features, and the problem of high attention computational overhead is not solved. The multi-branch dilated convolutional pyramid, transposed convolution upsampling and multi-path dilated convolution introduced by the scheme are densely stacked, which further increases the number of parameters and computational cost, which contradicts its lightweight requirements. Moreover, its edge enhancement is mainly aimed at local edge texture, and the long-range spatial dependence of defects such as thin cracks still relies on high-overhead self-attention modeling.

[0007] For example, CN121837726A discloses a road defect detection method under complex weather conditions based on improved RT-DETR. It replaces the multi-head self-attention in AIFI with an intra-scale feature interaction module based on statistical feature self-attention, and designs a frequency-domain modulated multi-scale cross-fusion network. It also enhances the detection robustness under complex weather conditions by combining multiple meteorological data. However, the statistical feature self-attention used in this scheme is still a variant of the attention mechanism. It requires multiple sets of low-rank projection matrices and calculates membership degree and second-order statistics, resulting in high structural complexity. Its improvement focuses on robustness under complex weather conditions, without systematically compressing the number of model parameters and computational cost. Furthermore, statistical feature self-attention focuses on global structure characterization and has limited ability to focus on the local response of small defects.

[0008] For example, CN119251225B discloses a method for detecting road surface cracks based on an improved RT-DETR. It constructs a high-low frequency attention module consisting of a high-frequency attention head and a low-frequency attention head to replace AIFI, and replaces the multi-scale feature fusion module with an iterative feature fusion module to improve the crack feature extraction capability in complex road environments. However, the high-low frequency attention module of this scheme is still based on Softmax multi-head dot product attention. Each attention head needs to perform dot product attention calculation, and the problem of large attention computation overhead still exists. Its detection target is mainly for medium-scale and large-scale cracks, and its ability to distinguish small defects is insufficient. Moreover, it does not systematically optimize the overall parameter quantity and computational quantity of the model.

[0009] For example, CN119887641A discloses a lightweight highway pavement defect detection method based on improved RT-DETR. It reconstructs the ResNet18 backbone network using the lightweight module RGCSPELAN, introduces the efficient additive attention module EAA to improve the multi-head self-attention of AIFI, and uses the feature pyramid CGRFPN as a cross-scale feature fusion module to reduce computational overhead. However, the EAA additive attention in this scheme generates a global context by weighting the key elements through a global query, resulting in a smooth attention distribution. It is not capable of discriminating weak response features such as small defects. Its lightweighting mainly relies on convolutional structure reconstruction and additive attention, and it does not obtain linear complexity attention computation while preserving the interaction information of positive and negative features. Its CGRFPN focuses on axial context modeling in the horizontal and vertical directions, which has limitations in modeling the long-range dependency of slender cracks distributed in arbitrary directions.

[0010] In summary, existing road defect detection methods cannot simultaneously meet the dual requirements of high-precision capture of small targets and slender defects, as well as lightweight and easy deployment. In particular, there is a lack of an improved scheme that can reduce the computational complexity of attention to linearity while preserving the interaction between positive and negative features, and systematically compress the number of model parameters and computational load. Summary of the Invention

[0011] The technical problem this invention aims to solve is to provide a lightweight road defect detection method based on an improved RT-DETR, addressing the challenge of existing road defect detection methods simultaneously meeting the dual requirements of high-precision capture of small targets and slender defects, as well as lightweight design and easy deployment. Furthermore, it also needs to address specific issues of RT-DETR, such as the high computational cost of the intra-scale feature interaction module, the loss of negative interaction in linear attention, the large number of parameters in the backbone network with a limited receptive field, and the overall model's unfavorable suitability for edge deployment.

[0012] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: This paper presents a lightweight road defect detection method based on an improved RT-DETR. The method first acquires an image dataset containing road defects and divides it proportionally into training, validation, and test sets. Then, an improved RT-DETR model is constructed, which is improved in three dimensions: In the hybrid encoder, the original intra-scale feature interaction module AIFI is replaced with a polarity-aware frequency modulation (PAFM) module, enabling attention computation to retain the positive and negative feature interactions between query keys while performing frequency-domain adaptive enhancement of features through a frequency modulation feedforward network; The original RT-DETR backbone network is replaced with a lightweight CSP-SSD backbone network, combining the state-space duality mechanism based on a hidden-state mixer with a cross-stage feature fusion architecture; In the multi-scale feature fusion stage, a shift-conv feature extraction (SCFE) module is introduced, which expands the receptive field without introducing additional parameters. Subsequently, the improved RT-DETR model is trained using the training and validation sets. Finally, the road image to be detected is input into the trained model to output the category and location information of the road defect.

[0013] In one implementation, the PAFM module includes a polarity-aware linear attention unit and a frequency modulation feedforward network unit. The polarity-aware linear attention unit element-wise decomposes the query vector and key vector into positive and negative components, calculating interactions between components with the same sign and interactions between components with opposite signs separately, thus fully preserving positive and negative feature interactions. Furthermore, this unit segments the value vector along the channel dimension to process the value vectors corresponding to the two types of interactions separately. The calculation results of the two types of interactions are concatenated and then scaled using a learnable polarity-aware coefficient matrix. To simulate the sharpness characteristics of Softmax attention, this unit also introduces a channel-level learnable exponent. The exponentiation function transforms the feature values ​​of the query vector or key vector, where the exponent... The range of values ​​is , The scaling factor α is greater than 0, and each channel can independently learn the optimal exponent. Preferably, the scaling factor α is 4. The frequency modulation feedforward network unit embeds a frequency domain modulation branch after the standard feedforward neural network. A block-based fast Fourier transform is used to convert the output features of the standard feedforward neural network to the frequency domain. Learnable weights are then used to multiply element-wise in the frequency domain to adaptively suppress or enhance each frequency component. Finally, an inverse fast Fourier transform is used to inversely transform the features back to the spatial domain and reshape them to obtain the frequency-modulated features.

[0014] In another implementation, the CSP-SSD lightweight backbone network constructs the EfficientViM basic module with the HSM-SSD module as its core. This basic module integrates lightweight convolution and state space duality mechanisms. The lightweight convolution is used to capture local texture and edge details, while the HSM-SSD module is used to model long-range spatial dependencies. Combining the cross-stage feature reuse concept of CSPNet (CrossStage Partial Network), multiple sets of EfficientViM basic modules are stacked. Each stage performs spatial downsampling and channel dimensionality expansion step by step, forming a multi-scale hierarchical feature system. Specifically, the HSM-SSD module first generates the global hidden state through weighted linear combination, then the hidden state mixer performs channel mixing and nonlinear transformation on the global hidden state, and finally maps it back to the original feature space through the projection matrix to obtain the output features.

[0015] In another implementation, the SCFE module uses shift convolution to expand the receptive field of the input features: the channels of the input features are divided into five groups, and left, right, up, down, and identity transformations are performed respectively. The features of each channel are shifted by one pixel along the corresponding direction, and then cross-channel fusion is achieved through 1×1 point convolution. The SCFE module further adopts a dual-branch structure, where the identity residual branch adjusts the channel dimension through 1×1 convolution to preserve the local detail information of the original features; the feature extraction branch consists of at least one feature extraction unit, each of which includes a first shift convolution, an activation function, and a second shift convolution. The first shift convolution expands the channel dimension, and after the activation function introduces nonlinearity, the second shift convolution compresses the channel dimension back to the original dimension; the outputs of the two branches are added together to obtain the output features of the SCFE module.

[0016] The lightweight road defect detection method based on improved RT-DETR provided by this invention has the following beneficial effects: 1. This invention systematically improves the RT-DETR model from three core dimensions: feature interaction quality, backbone lightweight efficiency, and long-range receptive field coverage, achieving a synergistic improvement in detection accuracy and model lightweighting. It simultaneously meets the dual requirements of high-precision capture of small targets and slender defects in road defect detection, as well as lightweight design and easy deployment. Testing shows that the complete model integrating PAFM, CSP-SSD, and SCFE improves accuracy by 2.0 percentage points, recall by 1.7 percentage points, mAP50 by 2.1 percentage points, and mAP50:95 by 2.6 percentage points compared to the baseline RT-DETR.

[0017] 2. This invention decomposes query key features into positive and negative components using polarity-aware linear attention, separately calculating the interaction between signs and opposite signs, thus fully preserving the interaction information between positive and negative features. It can recover the negative interaction information lost by traditional linear attention due to non-negative kernel approximation, avoiding feature information loss and thereby improving the model's ability to detect minor defects. Testing on the RDD2022 dataset shows that, without increasing computational overhead, introducing the PAFM module alone improves accuracy by 2.1 percentage points, mAP50 by 1.1 percentage points, and mAP50:95 by 1.5 percentage points.

[0018] 3. This invention segments the value vector along the channel dimension and processes the value vectors corresponding to the two types of interactions separately. Then, it scales the concatenation result using a learnable polarity-aware coefficient matrix. This allows for accurate reconstruction of the dependency relationship between the query and the key without introducing additional parameter overhead, balancing the lightweight nature of attention computation with the accuracy of feature representation.

[0019] 4. This invention simulates the sharpness characteristics of Softmax attention by introducing a power function with a channel-level learnable exponent; it can overcome the problem of overly uniform linear attention distribution, and while maintaining linear complexity, enables each channel to adaptively focus on small target features, providing a foundation for focusing on small targets in subsequent detection.

[0020] 5. This invention embeds a frequency domain modulation branch after the standard feedforward neural network through a frequency modulation feedforward network, and uses block-based fast Fourier transform to convert features to the frequency domain, and performs element-wise multiplication in the frequency domain through learnable weights; it can enhance the model's ability to capture high-frequency details such as fine cracks and pothole edges, while suppressing interference such as road surface texture and lighting changes.

[0021] 6. This invention replaces the original backbone network with a lightweight CSP-SSD backbone network, embeds a state-space dual mechanism based on a hidden-state mixer into the cross-stage feature fusion architecture, and reduces parameter redundancy by incorporating the idea of ​​cross-stage feature reuse. This significantly reduces the number of model parameters and computational cost, while enhancing the preservation of long-range spatial dependencies and fine-grained features. Testing showed that adding the CSP-SSD module to the PAFM module reduced the number of model parameters from 20.1M to 14.6M, a reduction of approximately 27.4%, and the computational cost from 57.2 GFLOPS to 48.2 GFLOPS, while simultaneously increasing the recall rate to 70.2% and the F1 score to 76.0%.

[0022] 7. This invention utilizes the compression-enhancement-reconstruction structure of the HSM-SSD module to generate a global hidden state through a weighted linear combination. After channel mixing and nonlinear transformation by the hidden state mixer, the state is mapped back to the original feature space; this can achieve O(ND) 2Global context modeling is achieved with a computational complexity of O(LND), compared to the traditional self-attention mechanism's O(LND) complexity. 2 D) The computational complexity is greatly reduced, which is beneficial for real-time detection.

[0023] 8. This invention uses the SCFE (Separated Convolutional Feature Extraction) module to divide the input feature channels into five groups, shifting them by one pixel along the left, right, up, down, and constant directions, and then performing 1×1 point convolution to achieve cross-channel fusion. This can expand the receptive field without additional parameter overhead, fully capture the multi-directional spatial correlation of long-range distributed defects such as thin cracks and irregular pits, and improve their semantic perception and positioning accuracy.

[0024] 9. This invention uses the dual-branch structure of the SCFE module to preserve the local details of the original features through the identity residual branch, and to enrich the feature expression through the feature extraction branch through the channel expansion-nonlinear activation-channel compression structure; it can take into account both local details and long-range correlations, and avoid feature degradation while expanding the receptive field.

[0025] 10. This invention divides the road defect image dataset into training, validation, and test sets, making model training, parameter tuning, and performance evaluation independent of each other; it can ensure the credibility of model evaluation results and improve the model's generalization ability on road defect detection tasks.

[0026] 11. This invention adopts a lightweight design overall. Through the synergistic effect of the CSP-SSD lightweight backbone network and parameter-free overhead shifted convolution, it reduces the number of model parameters and computational cost while maintaining high detection accuracy; this is beneficial for deployment on edge devices such as vehicles or drones. Testing shows that the complete model reduces the number of parameters by 27.1% and the computational cost by 17.2% compared to the baseline RT-DETR.

[0027] 12. This invention controls the channel-level learnable index. The range of values ​​is set to This allows the model to achieve optimal overall performance. Testing showed that when... The model accuracy reached 83.6%, with mAP75 and mAP50:95 reaching 53.1% and 49.1% respectively, both the highest in the group. The F1 score was 75.4%, demonstrating a positioning accuracy advantage at a high intersection-union threshold, which meets the needs of precise road defect detection. Attached Figure Description

[0028] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is a structural diagram of the improved RT-DETR model of the present invention; Figure 2This is a structural diagram of the polarity sensing frequency modulation (PAFM) module in this invention; Figure 3 This is a structural diagram of the polarity-sensing linear attention unit in this invention; Figure 4 This is a structural diagram of the frequency modulation feedforward network unit in this invention; Figure 5 This is a structural diagram of the HSM-SSD module in this invention; Figure 6 This is a structural diagram of the EfficientViM basic module in this invention; Figure 7 This is a structural diagram of the shifted convolution in this invention; Figure 8 This is a structural diagram of the SCFE module in this invention; Figure 9 This is a schematic diagram showing the visualization and comparison of the heatmap of the model proposed in this invention; Figure 10 This is a comparison chart of the detection results of the present invention and the baseline RT-DETR on the test set; Figure 11 for Figure 10 First row of images (a) Enlarged view of RT-DETR; Figure 12 for Figure 10 Second row of images (b) - Enlarged view of Ours. Detailed Implementation

[0029] The technical solutions of the present invention will be further described below with reference to the embodiments and accompanying drawings: Example 1 This embodiment provides a lightweight road defect detection method based on an improved RT-DETR (Real-Time Detection Transformer). Systematic improvements are made in three core dimensions: feature interaction quality, backbone lightweight efficiency, and long-range receptive field coverage, achieving a synergistic improvement in accuracy and speed. The specific steps are as follows: Step 1: Dataset Preparation An image dataset containing road defects was obtained and divided into training, validation, and test sets. Preferably, 9175 road images from China and the United States were selected from the RDD2022 (Road Damage Dataset 2022) global road damage detection challenge dataset, covering five types of road defects: longitudinal cracks (D00), transverse cracks (D10), network cracks (D20), potholes (D40), and repair marks. The dataset was randomly divided into training, validation, and test sets in an 8:1:1 ratio.

[0030] Step 2: Construct an improved RT-DETR model like Figure 1 As shown, this invention improves RT-DETR from three core dimensions.

[0031] Step 2.1: Design a Polar Sensing Frequency Modulation (PAFM) module like Figure 2 As shown, a Polarity-Aware Frequency Modulation (PAFM) module is designed to replace the Attention-based Intra-scale Feature Interaction (AIFI) module in the hybrid encoder of the RT-DETR model. The PAFM module includes a polarity-aware linear attention unit and a frequency modulation feedforward network unit.

[0032] Note: Figure 2 In this context, Pola stands for Polarity-Aware Linear Attention; FMFFN stands for Frequency-Modulation Feed-Forward Network; Encoder represents the encoder; and Add&Norm represents residual addition and layer normalization.

[0033] Polarity-sensing linear attention unit Pola structure as follows Figure 3 As shown. To reduce complexity while maintaining feature discriminative power, Pola divides all components into like-sign and opposite-sign components based on the polarity of the query key pairs. Specifically, the query vector... With key vector The elements are decomposed into positive and negative components, and the decomposition form is as follows: (1); in, and They are respectively The positive and negative parts; and They are respectively The positive and negative parts. Substitute these decompositions into... and The inner product of these two products yields: (2); The first two terms capture the similarity of components with the same symbol, while the latter two terms represent the interaction of components with opposite symbols. Traditional linear attention (such as ReLU-based mapping) directly discards negative components, resulting in the loss of a large amount of feature information. To preserve complete interaction information, polarity-aware attention separately calculates the interactions of components with the same and opposite symbols, and the attention weights can be approximated as: (2); In the formula, , , They are respectively The vector transpose; This represents the Softmax kernel function; This indicates the exponentiation operation of the dot product of the query keys; This represents the feature map of the kernel function.

[0034] To avoid introducing additional parameters, the value vector is adjusted along the channel dimension. The interaction is segmented and processed separately for two types. The output features are concatenated and then scaled using a symbolic perception matrix for accurate reconstruction. and Dependencies: (3); In the formula, This indicates that calculations with the same and opposite signs are connected; For the first The positive component of the location query. For the first The negative component of the location query; For the first The positive components of each position key For the first The negative component of each position key. For the first The positive components of each position key For the first The negative component of each position key; ⊙ represents element-wise multiplication; A vector of values ​​with the same sign. A vector of inverse sign values; For a matrix of learnable coefficients with the same sign, It is the learnable coefficient matrix with inverse sign.

[0035] Although polarity-aware linear attention recovers positive and negative interaction information, it still faces the problem of overly uniform attention distribution. To simulate the sharpness characteristics of Softmax, a channel-level learnable exponent is introduced. power function: (4); (5); in, Scaling factor ; This means applying the Sigmoid function individually to each learnable parameter; These are learnable parameters; This represents an element-wise exponential function mapping; Indicates the first Take each dimension Power of 1.

[0036] The final feature mapping is: (6); (7); in, express or ; This indicates a modified linear unit.

[0037] This design allows each channel to learn the optimal exponent independently, effectively simulating the sharpness characteristics of Softmax while maintaining linear complexity, thus providing a foundation for focusing on small targets in subsequent detection.

[0038] Note: Figure 3 In this context, Matmul stands for Matrix Multiplication, and Concat stands for Concatenation.

[0039] The structure of the Frequency-Modulation Feed-Forward Network (FMFFN) is as follows: Figure 4 As shown. To further enhance the model's ability to capture high-frequency details of defects, this paper embeds a frequency domain modulation branch after the standard FFN (Feed-Forward Network). Specifically, a block-based Fast Fourier Transform is used to transform the features obtained by the standard FFN into... Transform to the frequency domain. Next, introduce a learnable weight. By performing element-wise multiplication in the frequency domain, all frequency components are adaptively suppressed or enhanced to obtain the frequency-modulated characteristics. Subsequently, the inverse fast Fourier transform was used to... The inverse transformation is performed back to the spatial domain, and the refined features are obtained by reshaping. The complete process can be formalized as follows: (8); (9); (10); in, The activation function for the Gaussian error linear unit; and These represent block partitioning and block merging operations, respectively. and These represent the Fast Fourier Transform and the Inverse Fast Fourier Transform, respectively. For input features; It is a 1×1 convolution.

[0040] The dual-stage structure of "channel mixing + frequency domain modulation" can significantly enhance the model's ability to perceive minute cracks and pothole edges, while suppressing interference from road surface texture and lighting changes.

[0041] Note: Figure 4 In this context, Conv stands for Convolution, GELU stands for Gaussian Error Linear Unit, Bolck Partitioning represents block partitioning, Bolck Merging represents block merging, FFT stands for Fast Fourier Transform, and IFFT stands for Inverse Fast Fourier Transform.

[0042] Step 2.2: Construct a lightweight CSP-SSD backbone network A lightweight CSP-SSD (Cross Stage Partial State Space Duality) backbone network is constructed to replace the original backbone network of the RT-DETR model. The core component of this network is the HSM-SSD (HSM-SSDHidden State Mixer-based State Space Duality) module, whose structure is as follows: Figure 5 As shown. Specifically, given input features, HSM-SSD first generates the global hidden state through a weighted linear combination. Through this compression operation, the original required O(LD) 2 The complexity of linear projection is reduced to O(ND). 2 This significantly reduces the computational burden. Based on this, HSM (Hidden State Mixer) further processes the compressed hidden states... Channel mixing and nonlinear transformation are performed to enhance feature representation capabilities. This three-stage "compression-enhancement-reconstruction" structure enables HSM-SSD to achieve O(ND) resolution. 2 Global context modeling is achieved within a time complexity of LND+, compared to the traditional self-attention mechanism O(LND). 2 D) has a significant advantage in terms of complexity.

[0043] Note: Figure 5 In this context, Lin is short for Linear, Linear Layer is a linear layer; MatMul is Matrix Multiplication; Discretization is discretization; and DWConv is Depth-wise Convolution.

[0044] like Figure 6 As shown, the core of the EfficientViM (Efficient Vision Mamba) basic module is constructed using HSM-SSD, integrating lightweight convolution and state space duality: lightweight convolution focuses on capturing local texture and edge details, while HSM-SSD is responsible for modeling long-range spatial dependencies. Furthermore, combining the cross-stage feature reuse concept of CSPNet (Cross Stage Partial Network), multiple sets of EfficientViM modules are stacked to build the CSP-SSD backbone network. Each stage progressively completes spatial downsampling and channel dimensionality expansion, forming a multi-scale hierarchical feature system.

[0045] Note: Figure 6 In this context, Stem Layer represents the backbone layer; EfficientViM Block represents efficient vision; Mamba represents the base block; Patch Merging represents the patch merging layer; LayerNorm stands for Layer Normalization; and Stage 1 / Stage 2 / Stage3 represent stages 1 / 2 / 3.

[0046] Step 2.3: Introduce the Shift-Convolution Feature Extraction (SCFE) module. A Shift-Conv Feature Extraction (SCFE) module is introduced to expand the receptive field during the multi-scale feature fusion stage of the RT-DETR model. The shift-conv structure is as follows: Figure 7 As shown. Shift convolution can be viewed as a special type of depthwise convolution, whose core functionality is to expand the receptive field through channel-level shifting without introducing additional parameters. For the input feature tensor... ,in For space dimensions, Input channel number. Shift operation output. The calculation is as follows: (11); in, , For heavy-weight centralized indexes; Indicates the output feature map at position ,aisle Eigenvalues ​​at; This indicates that the shifted convolution kernel is at the offset. ,aisle Weight at each location; This indicates that the input feature map is at the location ,aisle Eigenvalues ​​at; This represents the row offset within the convolution kernel; This represents the column offset within the convolution kernel.

[0047] shift nucleus Defined as: (12); Each channel For each unique shift direction, only that position in the shift kernel is 1, while the rest are 0, thus shifting the channel feature by one pixel along the specified direction. To fully capture the multi-directional spatial correlation of long-range defects, five basic shift directions—left, right, up, down, and identity—are used. The shift operation only rearranges the features within a channel. To achieve cross-channel information fusion, a point convolution needs to be applied after the shift, calculated as follows: (13); in, This indicates the spatial location of the shifted intermediate feature map. ,aisle Eigenvalues ​​at; Indicates the parameters of the point convolution kernel; For point convolution kernels, To output the number of channels, this operation fuses the multi-directionally shifted features by weighted summation of the channel dimensions, generating a feature representation that combines local details with long-range correlations.

[0048] Note: Figure 7 In Chinese, Shift stands for Shift Operation, which is a shift operation.

[0049] like Figure 8 As shown, the SCFE module adopts a dual-branch structure: the identity residual branch adjusts the channel dimension through 1×1 convolution, preserving the local detail information of the original features; the feature extraction branch consists of... The system consists of stacked feature extraction units, each composed of two shift-convolutional layers (Shift-Conv) and a ReLU activation function, forming a feature learning structure of "channel expansion - non-linear activation - channel compression". Specifically, the first shift-convolution expands the channels. times ( Let be the coefficient of thermal expansion, and take . The feature expression is enriched through multi-directional shifting and cross-channel fusion; after introducing nonlinearity through the ReLU activation function, the second shift convolution compresses the number of channels back to the original dimension.

[0050] Note: Figure 8 In this context, Shift-Conv stands for Shift Convolution, and ReLU stands for Rectified Linear Unit, the ReLU activation function.

[0051] Step 3: Model Training The improved RT-DETR model constructed in Step 2 was trained using the training and validation sets from Step 1. All experiments were conducted on the following hardware and software platform: Ubuntu 18.04.6 LTS (Linux 5.4.0-150-generic) operating system, Intel Core i9-10900 X CPU (Central Processing Unit), and NVIDIA GeForce RTX 3080 GPU (Graphics Processing Unit). The deep learning framework used was PyTorch 2.0.1, implemented using Python 3.9 and CUDA 11.7 (Compute Unified Device Architecture 11.7). Experimental parameters were set as follows: input image resolution 640×640, training epochs 150, batch size 4, AdamW optimizer, and initial learning rate 0.0001.

[0052] Step 4: Model Testing and Evaluation The road image to be detected is input into the improved RT-DETR model trained in step 3, which outputs the category and location information of road defects. To evaluate the performance of the improved model, accuracy, recall, F1 score, mean detection precision (mAP) (including mAP50, mAP75, and mAP50:95), model parameters, and computational cost (GFLOPS) are used as evaluation metrics.

[0053] To verify the effectiveness of the various improvements to RT-DETR, ablation experiments were conducted on the road defect dataset used under the same experimental environment and training hyperparameters.

[0054] To determine the most suitable method for road defect detection tasks Values ​​were set in this experiment while keeping other hyperparameters constant. Conduct the experiment.

[0055] For the task of road defect detection, a comparative experiment was conducted using the current mainstream target detection models.

[0056] like Figure 9 and Figure 10 As shown, Figure 9 This image presents a visual comparison of the heatmaps of the proposed detection model (b) and RT-DETR (a) on five sets of road images. In the heatmap, redder colors indicate higher model attention to that area. Figure 9 It can be seen that the model proposed in this invention produces a more concentrated and accurate thermal response in small target areas, effectively reducing false detections in background interference areas, indicating that the improved module significantly enhances the model's ability to perceive road defects.

[0057] Figure 10 The image shows a comparison of the detection results of the proposed detection model (b) and RT-DETR (a) on six sets of road images and on a test set. The top side shows the detection results of the original RT-DETR (a), and the bottom side shows the detection effect of the improved detection model (b) of this invention. As can be seen from the detection results shown in the figure, for the detection of small targets in complex scenes, the improved algorithm can effectively reduce missed detections and improve the detection accuracy of small targets; for long-range distributed defects such as thin cracks, the improved algorithm can achieve more complete detection; for multi-class defect detection, the improved algorithm can effectively detect small targets and also improve the accuracy of targets at other scales.

[0058] Note: Figure 9 In the image, Original is the original road image; (a) RT-DETR is the baseline model heatmap; (b) Ours is the model heatmap proposed in this invention; the colors of the heatmap from blue to red indicate that the model attention level is from low to high.

[0059] Note: Figure 10 In the middle, the top side shows (a) the detection results of RT-DETR on the test set; the bottom side shows the detection results of our invention (b) Ours on the test set; Repair means restoration; Figure 11 for Figure 10First row of images (a) is a magnified view of RT-DETR. Figure 11 The first row of pictures are as follows: Figure 10 Top (a) RT-DETR, from left to right: first, second, and third images; second row of images: Figure 10 The top image shows (a) the fourth, fifth, and sixth images from left to right of RT-DETR; similarly, Figure 12 for Figure 10 Second row of images (b) - Enlarged view of Ours. Figure 12 The first row of pictures are as follows: Figure 10 The bottom (b) section shows the first, second, and third images from left to right. The second row of images are as follows: Figure 10 The bottom shows (b) Ours, the fourth, fifth, and sixth images from left to right.

[0060] Example 2 In another preferred embodiment, based on Embodiment 1, this embodiment provides a lightweight road defect detection method based on an improved RT-DETR (Real-Time Detection Transformer). The algorithm is implemented using Python and PyTorch frameworks. Experimental data comes from the RDD2022 (Road Damage Dataset 2022) dataset, selecting 9175 road images from China and the United States, divided into training, validation, and test sets in an 8:1:1 ratio. Experimental hardware: Ubuntu 18.04, Intel Core i9-10900X CPU (Central Processing Unit), NVIDIA GeForce RTX 3080 GPU (Graphics Processing Unit).

[0061] Step 11: Dataset Preparation The dataset contains five categories: longitudinal cracks (D00), transverse cracks (D10), network cracks (D20), pits (D40), and repair.

[0062] Step 2: Construct an improved RT-DETR model like Figure 1 As shown, this invention designs a lightweight road defect detection method based on an improved RT-DETR. The specific improvements include three aspects: 1. In the intra-scale feature interaction stage of the hybrid encoder, the original AIFI (Attention-based Intra-scale Feature Interaction) module is replaced with PAFM (Polarity-Aware Frequency Modulation) module. The positive and negative feature interactions are preserved through polarity-aware linear attention, and frequency domain adaptive modulation is combined to enhance the high-frequency details of defects, thereby improving the ability to distinguish small defects in complex backgrounds under linear complexity. 2. The original ResNet (Residual Network) backbone structure is replaced with a lightweight CSP-SSD (Cross Stage Partial State Space Duality) backbone, which deeply integrates the state space duality mechanism of the hidden state mixer with the cross-stage feature fusion architecture. This significantly reduces the number of parameters and computation while enhancing the ability to preserve long-range spatial dependencies and fine-grained features. 3. In the multi-scale feature fusion stage, the SCFE (Shift-Conv Feature Extraction) module is introduced to achieve an equivalent large receptive field with shift convolution without additional parameter overhead, which significantly improves the semantic perception and localization accuracy of long-range distributed defects such as thin cracks and irregular pits.

[0063] Step 2.1, PAFM module implementation PAFM module structure as follows Figure 2 As shown. The structure of the polarity-sensing linear attention unit (Pola) is as follows. Figure 3 As shown. In the specific implementation, the query vector With key vector Decomposed into positive and negative components: (1); The same-symbol attention and anti-symbol attention are calculated separately, concatenated, and then scaled using a symbol perception matrix. Channel-level learnable index. The power function is: (5); in, From learnable parameters and scaling factor Decide: (4); The structure of a frequency-modulation feed-forward network (FMFFN) is as follows: Figure 4 As shown, a frequency domain modulation branch is embedded after the standard FFN (Feed-Forward Network). Specifically, a block-based Fast Fourier Transform is used to transform the features obtained from the standard FFN. Transform to the frequency domain. Next, introduce a learnable weight. By performing element-wise multiplication in the frequency domain, all frequency components are adaptively suppressed or enhanced to obtain the frequency-modulated characteristics. Subsequently, the inverse fast Fourier transform was used to... The inverse transformation is performed back to the spatial domain, and the refined features are obtained by reshaping. .

[0064] Step 2.2, Implementation of CSP-SSD Backbone Network The HSM-SSD (HSM-SSDHidden State Mixer-based State Space Duality) module structure is as follows: Figure 5 As shown. Given input features, HSM-SSD first generates a global hidden state through a weighted linear combination. Then, the hidden state mixer processes the compressed hidden state. Channel mixing and nonlinear transformation are performed. Finally, the projection matrix is ​​used. Mapping back to the original feature space yields the output sequence. .

[0065] The EfficientViM (Efficient Vision Mamba) basic module is built around HSM-SSD, with the following structure: Figure 6 As shown, stacked EfficientViM modules form the CSP-SSD backbone, with each stage progressively completing spatial downsampling and channel dimensionality expansion to form a multi-scale hierarchical feature system.

[0066] Step 2.3, SCFE module implementation Shifted convolutional structures, such as Figure 7 As shown. Input feature F divides the channels into 5 groups, and performs left shift, right shift, up shift, down shift, and identity transformation respectively. After shifting, cross-channel fusion is achieved through 1×1 point convolution.

[0067] The SCFE (Shift-Conv Feature Extraction) module structure is as follows: Figure 8As shown. The input features are divided into two paths by a 1×1 convolution: the identity residual branch adjusts the channels through a 1×1 convolution; the feature extraction branch is... The feature extraction unit consists of stacked feature extraction units, each of which is Shift-Conv (channels expanded by 4 times) - ReLU-Shift-Conv (channels compressed back to the original dimension). The two outputs are added together to obtain the final feature.

[0068] Step 3: Model Training and Evaluation In polarity-aware linear attention, the scaling factor Control the learnability index The range of values This directly affects the sharpness of attention distribution. To determine the most suitable [method / mechanism] for road defect detection tasks... In this experiment, while keeping other hyperparameters constant, the values ​​were set separately. The experiment was conducted, and the results are shown in Table 1.

[0069] Table 1 Scaling Factors Impact on performance

[0070] Note: In Table 1, P is the accuracy; R is the average recall; F1-Score is the F1 score; mAP is the mean detection precision (IoU thresholds are 0.5, 0.75, and 0.5~0.9, respectively).

[0071] Experiments show that when The model achieved the best overall performance, with an accuracy of 83.6%, mAP75 and mAP50:95 of 53.1% and 49.1% respectively, both the highest in the group; the F1-Score was 75.4%, only slightly lower than 75.6% for α=7, achieving the optimal balance between accuracy and recall. Its superior positioning accuracy at high IoU thresholds perfectly aligns with the needs for precise road defect detection. Therefore, subsequent experiments all adopted... .

[0072] To verify the effectiveness of each improvement strategy, we designed an ablation experiment, and the results are shown in Table 2.

[0073] Table 2 Ablation Experiment

[0074] Note: In Table 2, P is accuracy; R is average recall; F1-Score is F1 score; mAP is average detection precision (IoU thresholds are 0.5 and 0.5-0.9 respectively); Param / M is the number of model parameters (unit: million); GFLOPS / G is the number of floating-point operations per billion.

[0075] As shown in Table 2, the experimental data reveals that introducing the PAFM module alone, without increasing computational overhead, improves accuracy by 2.1 percentage points, mAP50 by 1.1 percentage points, and mAP50:95 by 1.5 percentage points. Adding the CSP-SSD module significantly reduces the number of parameters from 20.1M to 14.6M, a reduction of approximately 27.4%, and the computational cost from 57.2 GFLOPS to 48.2 GFLOPS, while simultaneously increasing recall to 70.2% and F1-Score to 76.0%. Finally, the complete model integrating PAFM, CSP-SSD, and SCFE (i.e., the method of this invention) achieves optimal detection performance, with mAP50 reaching 78.0%, mAP50:95 increasing to 50.2%, and F1-Score at 76.3%. Compared to the baseline RT-DETR, the model parameters were reduced by 27.1%, the computational cost was reduced by 17.2%, the accuracy was improved by 2.0 percentage points, the recall was improved by 1.7 percentage points, the mAP50 was improved by 2.1 percentage points, and the mAP50:95 was improved by 2.6 percentage points.

[0076] For the road defect detection task, a comparative experiment was conducted using the current mainstream target detection model. The experimental results are shown in Table 3.

[0077] Table 3 Comparative Experiments

[0078] Note: In Table 3, P is accuracy; R is average recall; F1-Score is F1 score; mAP is average detection precision (IoU thresholds are 0.5 and 0.5-0.9 respectively); Param / M is the number of model parameters (unit: million); GFLOPS / G is the number of floating-point operations per billion.

[0079] Experimental results clearly demonstrate that the proposed model has significant advantages in high recall, stable localization, and lightweight design, and its overall performance surpasses that of baselines and most mainstream methods.

[0080] In a preferred embodiment, the PAFM module described in step 2.1 includes a polarity-aware linear attention unit and a frequency modulation feedforward network unit; the polarity-aware linear attention unit will process the query vector... With key vector The elements are decomposed into positive and negative components, and the interactions between components with the same sign and the interactions between components with opposite signs are calculated separately to preserve the positive and negative feature interactions. The above settings enable the attention calculation to retain the complete positive and negative feature interactions while maintaining linear complexity, avoiding the loss of defect feature information due to the discarding of negative components, thereby improving the ability to distinguish minor defects.

[0081] In a preferred embodiment, the polarity-aware linear attention unit pairs the value vector along the channel dimension. The algorithm segments the data and processes the value vectors corresponding to the same-symbol interaction and the opposite-symbol interaction separately. The calculation results of the two types of interactions are concatenated and then scaled using a learnable polarity-aware coefficient matrix. This approach accurately reconstructs the dependency relationship between the query and the key without introducing additional parameter overhead, balancing the lightweight nature of attention computation with the accuracy of feature representation.

[0082] In a preferred embodiment, the polarity-aware linear attention unit further incorporates a channel-level learnable index. The power function transforms the feature values ​​of the query vector or key vector to simulate the sharpness characteristics of Softmax attention; where, , The scaling factor is greater than 0; in the power function, each channel has an independent learnable exponent parameter; the above settings enable each channel to adaptively simulate the sharpness characteristics of Softmax attention while maintaining linear complexity, overcoming the problem of overly uniform attention distribution, thereby enhancing the focus on small target features; preferably, the scaling factor... The value of is 4. Testing showed that when The model accuracy reached 83.6%, and the mAP50:95 reached 49.1%, both of which were the highest in the group. The F1 score was 75.4%.

[0083] In a preferred embodiment, the frequency modulation feedforward network unit embeds a frequency domain modulation branch after the standard feedforward neural network. The frequency domain modulation branch uses a block-based fast Fourier transform to convert the output features of the standard feedforward neural network to the frequency domain. Through learnable weights, it performs element-wise multiplication in the frequency domain to adaptively suppress or enhance frequency components. Then, it uses an inverse fast Fourier transform to inversely transform the features back to the spatial domain and reshape them to obtain the frequency-modulated features. The above settings enhance the model's ability to capture high-frequency details such as fine cracks and pothole edges, while suppressing interference from road surface textures and lighting changes.

[0084] In the preferred embodiment, the CSP-SSD lightweight backbone network described in step 2.2 constructs the EfficientViM basic module with the HSM-SSD module as its core. The EfficientViM basic module integrates lightweight convolution and state space duality mechanisms. Lightweight convolution is used to capture local texture and edge details, while the HSM-SSD module is used to model long-range spatial dependencies. Combined with CSPNet (Cross Stage Partial... The network (cross-stage local network) utilizes the concept of cross-stage feature reuse, stacking multiple sets of EfficientViM basic modules. Each stage progressively completes spatial downsampling and channel dimensionality expansion, forming a multi-scale hierarchical feature system. This setup significantly reduces the number of model parameters and computational cost while enhancing the preservation of long-range spatial dependencies and fine-grained features. Testing showed that after introducing the aforementioned polarity-aware frequency modulation module and adding the CSP-SSD lightweight backbone network, the number of model parameters decreased from 20.1M to 14.6M, a reduction of approximately 27.4%, and the computational cost decreased from 57.2 GFLOPS to 48.2 GFLOPS. Simultaneously, the recall rate increased to 70.2%, and the F1 score increased to 76.0%.

[0085] In a preferred embodiment, the HSM-SSD module generates a global hidden state through a weighted linear combination; the hidden state mixer performs channel mixing and nonlinear transformation on the global hidden state; the transformed hidden state is mapped back to the original feature space via a projection matrix to obtain the output features; the above settings are configured with O(ND) 2 Global context modeling is achieved with a computational complexity of O(LND), compared to the traditional self-attention mechanism's O(LND) complexity. 2 D) Reduced computational complexity reduces the computational burden and facilitates real-time detection.

[0086] In the preferred embodiment, the SCFE module described in step 2.3 uses shift convolution to expand the receptive field of the input features: the channels of the input features are divided into five groups, and left shift, right shift, up shift, down shift, and identity transformation are performed respectively. The features of each channel are shifted by one pixel along the corresponding direction. After shifting, cross-channel fusion is achieved through 1×1 point convolution. The above settings expand the receptive field in a way without additional parameter overhead, fully capturing the multi-directional spatial association of long-range distributed defects such as thin cracks and irregular pits, and improving their semantic perception and localization accuracy.

[0087] In the preferred embodiment, the SCFE module has a dual-branch structure: the identity residual branch adjusts the channel dimension through a 1×1 convolution, preserving the local details of the original features; the feature extraction branch consists of at least one feature extraction unit, each of which includes a first shift convolution, an activation function, and a second shift convolution connected in sequence. The first shift convolution expands the channel dimension, and after the activation function introduces nonlinearity, the second shift convolution compresses the channel dimension back to its original dimension; the outputs of the identity residual branch and the feature extraction branch are added together to obtain the output features of the SCFE module. This configuration allows the network to preserve the local details of the original features while expanding the receptive field, balancing local details with long-range correlations, and preventing feature degradation during deep propagation.

[0088] In summary, this invention proposes a lightweight road defect detection method based on an improved RT-DETR. By implementing a three-dimensional synergistic improvement of the RT-DETR model through a polarity-aware frequency modulation (PAFM) module, a lightweight CSP-SSD backbone network, and a shifted convolutional feature extraction (SCFE) module, this method overcomes the technical problem that existing road defect detection methods cannot simultaneously meet the dual requirements of high-precision capture of small targets and slender defects, as well as lightweight and easy deployment. This achieves a synergistic improvement in detection accuracy and model lightweighting.

[0089] At the feature interaction level, this invention establishes a new balance between Softmax self-attention and linear attention: query key features are element-wise decomposed into positive and negative components, and same-sign and anti-sign interactions are calculated separately. This prevents the loss of negative value interactions due to the non-negative kernel approximation in linear complexity attention calculations, overcoming the dilemma of smoothed attention distribution and decreased discrimination power for minor defects. Furthermore, a channel-level learnable exponent simulates the sharpness characteristics of Softmax, enabling each channel to adaptively focus on minor targets. A frequency modulation feedforward network then adaptively enhances each frequency component in the frequency domain, strengthening the perception of high-frequency details such as fine cracks and pit edges. Testing shows that introducing only the PAFM module improves accuracy by 2.1 percentage points and mAP50 by 1.1 percentage points without increasing computational overhead.

[0090] At the backbone network level, this invention embeds the state-space duality mechanism of the hidden state mixer into a cross-stage feature fusion architecture, using the HSM-SSD module at O(ND) 2 Global context modeling is completed within the computational complexity of LND (+LND), and parameter redundancy is reduced by cross-stage feature reuse, changing the situation where conventional backbone networks struggle to balance receptive field coverage and model lightweighting. Tests show that after adding CSP-SSD, the number of model parameters decreased from 20.1M to 14.6M, and the computational cost decreased from 57.2GFLOPS to 48.2GFLOPS, while the recall rate increased to 70.2% and the F1 score increased to 76.0%.

[0091] At the receptive field coverage level, this invention expands the receptive field without additional parameter overhead by using the SCFE (Switched Convolutional Feature Extraction) module: the input feature channels are divided into five groups, each shifted by one pixel along the left, right, up, down and constant directions, and then cross-channel fusion is completed by 1×1 point convolution; the dual-branch structure preserves the original local details while expanding the receptive field, so that defects such as long cracks and irregular pits distributed in multiple directions can be fully perceived and accurately located.

[0092] At the overall level, the three improvements work synergistically to achieve the optimal balance between detection accuracy and lightweight design: Testing showed that the complete model reduced the number of RT-DETR parameters by 27.1% and computational cost by 17.2% compared to the baseline, while improving accuracy, recall, mAP50, and mAP50:95 by 2.0, 1.7, 2.1, and 2.6 percentage points, respectively, with mAP50 reaching 78.0% and mAP50:95 reaching 50.2%; when the scaling factor... When the value is 4, the accuracy reaches 83.6%, and the mAP75 and mAP50:95 reach 53.1% and 49.1% respectively, both of which are the highest in the group. The F1 score is 75.4%, which shows the advantage of positioning accuracy under the high crossover ratio threshold, which meets the actual needs of precise detection of road defects and end-side deployment.

Claims

1. A lightweight road defect detection method based on improved RT-DETR, characterized in that, Includes the following steps: Step 1: Obtain the road defect image dataset and divide the road defect image dataset into training set, validation set and test set; Step 2: Construct an improved RT-DETR model; Step 3: Train the improved RT-DETR model using the training and validation sets to obtain the trained model; Step 4: Input the road image to be detected into the trained model, and output the category and location information of the road defects.

2. The lightweight road defect detection method based on improved RT-DETR according to claim 1, characterized in that, Step 2 specifically includes the following steps: Step 2.1: Replace the intra-scale feature interaction module AIFI in the hybrid encoder of the RT-DETR model with the polarity-aware frequency modulation (PAFM) module. The PAFM module retains the positive and negative feature interactions between query key features through polarity-aware linear attention and combines it with a frequency modulation feedforward network to achieve frequency domain adaptive enhancement. Step 2.2: Replace the original backbone network of the RT-DETR model with the CSP-SSD lightweight backbone network. The CSP-SSD lightweight backbone network embeds the state space duality mechanism based on the hidden state mixer into the cross-stage feature fusion architecture. Step 2.3: In the multi-scale feature fusion stage of the RT-DETR model, a shifted convolution feature extraction (SCFE) module is introduced. The SCFE module uses shifted convolution to expand the receptive field.

3. The lightweight road defect detection method based on improved RT-DETR according to claim 2, characterized in that: The PAFM module described in step 2.1 includes a polarity-aware linear attention unit and a frequency modulation feedforward network unit; The polarity-aware linear attention unit decomposes the query vector and key vector into positive and negative components element by element, and calculates the interaction between components with the same sign and the interaction between components with opposite signs, in order to preserve the interaction between positive and negative features.

4. The lightweight road defect detection method based on improved RT-DETR according to claim 3, characterized in that: The polarity-aware linear attention unit segments the value vector along the channel dimension, processes the value vectors corresponding to same-symbol interactions and opposite-symbol interactions respectively, concatenates the calculation results of the two types of interactions, and then scales them using a learnable polarity-aware coefficient matrix.

5. The lightweight road defect detection method based on improved RT-DETR according to claim 4, characterized in that: The polarity-aware linear attention unit also introduces a channel-level learnable index. The power function transforms the feature values ​​of the query vector or key vector to simulate the sharpness characteristics of Softmax attention; where, , It is a scaling factor greater than 0; in the power function, each channel has an independent learnable exponent parameter.

6. The lightweight road defect detection method based on improved RT-DETR according to claim 3, characterized in that: The frequency modulation feedforward network unit embeds a frequency domain modulation branch after the standard feedforward neural network; The frequency domain modulation branch uses block-based fast Fourier transform to convert the output features of the standard feedforward neural network to the frequency domain. Through learnable weights, element-wise multiplication is performed in the frequency domain to adaptively suppress or enhance frequency components. Then, the features are transformed back to the spatial domain and reshaped by inverse fast Fourier transform to obtain the frequency-modulated features.

7. The lightweight road defect detection method based on improved RT-DETR according to claim 2, characterized in that: The CSP-SSD lightweight backbone network described in step 2.2 uses the HSM-SSD module as the core to construct the EfficientViM basic module. The EfficientViM basic module integrates lightweight convolution and state space duality mechanism. The lightweight convolution is used to capture local texture and edge details, and the HSM-SSD module is used to model long-range spatial dependencies. Combining the CSPNet cross-stage feature reuse idea, multiple sets of EfficientViM basic modules are stacked, and each stage completes spatial downsampling and channel dimension expansion step by step to form a multi-scale hierarchical feature system.

8. The lightweight road defect detection method based on improved RT-DETR according to claim 7, characterized in that: The HSM-SSD module generates a global hidden state through a weighted linear combination; the hidden state mixer performs channel mixing and nonlinear transformation on the global hidden state; the transformed hidden state is mapped back to the original feature space through a projection matrix to obtain the output features.

9. The lightweight road defect detection method based on improved RT-DETR according to claim 2, characterized in that: The SCFE module described in step 2.3 uses shift convolution to expand the receptive field of the input features: the channels of the input features are divided into five groups, and left shift, right shift, up shift, down shift and identity transformation are performed respectively. The features of each channel are shifted by one pixel along the corresponding direction. After shifting, cross-channel fusion is achieved through 1×1 point convolution.

10. The lightweight road defect detection method based on improved RT-DETR according to claim 9, characterized in that: The SCFE module has a dual-branch structure: the identity residual branch adjusts the channel dimension through a 1×1 convolution, preserving the local details of the original features; the feature extraction branch consists of at least one feature extraction unit, each of which includes a first shift convolution, an activation function, and a second shift convolution connected in sequence. The first shift convolution expands the channel dimension, and after the activation function introduces nonlinearity, the second shift convolution compresses the channel dimension back to its original dimension; the outputs of the identity residual branch and the feature extraction branch are added together to obtain the output features of the SCFE module.

Images