Road garbage detection method combining RT-DETR and state space

Abstract

The invention discloses a road garbage detection method in combination with Real-Time Desection Transform (RT-DETR) and a state space, and the method comprises the following steps: firstly, obtaining an image through a crawler, carrying out the marking of the image, and constructing a garbage detection data set of a diversified urban road scene; secondly, designing a brand-new feature pyramid structure (FFS) and an internal scale feature enhancement module (IFEM) based on a state space, and establishing a road garbage detection model combining the RT-DETR and the state space; finally, a self-built road garbage data set is used for training the model, an experiment is carried out on a COCO2017 data set, the experiment proves that the algorithm provided by the invention obtains good balance between real-time performance and accuracy, and the performance of the model in a complex environment, especially the detection precision under small target recognition and shielding conditions, is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning target detection, and specifically proposes a road garbage detection method combining RT-DETR and state space. Background Art

[0002] With the rapid advancement of global urbanization, the problem of urban road garbage is becoming increasingly serious, directly affecting the quality of the environment and people's health. Traditional road cleaning and garbage collection methods often have limitations such as high labor costs, low efficiency, and insufficient coverage. There is an urgent need to rely on automated detection and cleaning systems to achieve more efficient management.

[0003] In recent years, computer vision and deep learning technologies have made significant breakthroughs in the fields of image recognition and target detection, such as the YOLO series and DETR, which provide practical solutions for the automation of road garbage detection. Among them, RT-DETR (Real-Time DEtection TRansformer) has performed well in real-time target detection by introducing the Transformer architecture. It adopts an innovative encoding-decoding structure, combined with an efficient encoder and IoU-aware query selection mechanism, abandons the traditional convolution design and NMS post-processing steps, giving it significant advantages in detection accuracy and speed. However, due to the diverse types and complex shapes of road garbage, the model still needs to be further optimized to improve performance in small target detection and complex scene recognition.

[0004] In recent years, Mamba based on the state space model (SSM) has performed well in processing long-range dependencies and has superior linear time complexity, providing a new idea for solving image problems. Therefore, the present invention proposes an improved road garbage detection algorithm based on RT-DETR and the state space model. The algorithm combines the feature pyramid structure, the state space-based internal scale feature enhancement strategy and the efficient detection capability of RT-DETR. On the basis of maintaining real-time performance, it significantly improves the accuracy and robustness of garbage detection, enhances the target detection accuracy in complex scenes, and performs well in occlusion and small target detection. Summary of the invention

[0005] Purpose of the invention: In view of the shortcomings of the above-mentioned prior art, a road garbage detection method combining RT-DETR and state space is proposed. The algorithm proposed in the present invention achieves a good balance between real-time and accuracy, and significantly improves the performance of the model in complex environments, especially the detection accuracy in small target recognition and occlusion.

[0006] The technical solution adopted by the present invention is a road garbage detection method combining Real-Time Detection transformer (RT-DETR) and state space, comprising the following steps:

[0007] S1: Acquire road garbage images;

[0008] S2: Annotate and augment images;

[0009] S3: Construct a road garbage detection model combining RT-DETR and state space;

[0010] S4: Training a road trash detection model combining RT-DETR and state space;

[0011] S5: Use the trained model to detect road litter in the field and conduct experiments on the COCO2017 dataset.

[0012] Furthermore, in step S1, road garbage images are collected by self-collection and web crawlers, imitating the perspective of an unmanned sweeper to collect large, medium and small targets under different backgrounds, lighting and weather conditions; for classes with less data volume, Stable Diffusion AI generation is used for collection, and a total of more than 60,000 diverse road garbage images are collected.

[0013] Furthermore, in step S2, after completing the image collection, the data is cleaned and screened to remove invalid or low-quality samples to ensure data diversity; the road garbage images are annotated using the VoTT tool to generate bounding boxes and labels; data enhancement is performed through methods such as rotation, flipping, cropping and splicing to increase sample diversity; the image data is divided into training set, validation set and test set in a ratio of 7:2:1, and organized into the standard COCO format, and finally the data set is exported to support model training and testing.

[0014] Furthermore, in step S3, the road garbage detection model combining RT-DETR and state space is composed of a backbone network, a feature pyramid structure (FFS), a deep fusion encoder (DFE), and a transformer decoder with an auxiliary prediction head:

[0015] The model first normalizes the input image and then feeds it into the backbone network to extract the features of the last three stages (S3, S4, S5). These features are fed into the innovatively designed feature pyramid structure (FFS) to generate multi-scale features for encoder input. In the deep fusion encoder (DFE), the intra-scale feature enhancement module (IFEM) and the cross-scale feature fusion module (CCFM) jointly transform the multi-scale features into image feature sequences. Subsequently, an IoU-aware query selection mechanism is used to select a fixed number of image features as the initial object query for the decoder. The decoder generates a prediction box and its corresponding confidence score through iterative optimization and combined with the auxiliary prediction head.

[0016] The innovative feature pyramid structure (FFS) first fuses the features extracted by the backbone network, and then uses the feature fusion module (FFM) and feature enhancement module (FEM) to extract more representative multi-scale features; finally, through the cascade feature fusion module (SFAM), these multi-level features are integrated into multi-scale features as the input of the subsequent deep fusion encoder (DFE):

[0017] The feature fusion module FFMv1 first fuses the last three stage features (S3, S4, S5) extracted by the backbone network; then it compresses the number of channels through 1x1 convolution and uses upsampling to adjust the size of the feature map; finally, after the splicing operation, the basic feature layer (S3', S4', S5') is obtained to provide input for the feature enhancement module (FEM); the feature enhancement module (FEM) adopts a lightweight U-Net architecture. After completing the upsampling and downsampling operations, it performs element-by-element addition on the feature maps of the corresponding sizes to obtain the feature layer (a3, a4, a5); in addition, a 1x1 convolution layer is introduced in the module to enhance the learning ability of the model. The smoothness of the features is maintained; it is worth noting that the feature fusion module FEMv1 contains an additional upsampling operation compared to the feature fusion module FEMv2, because its input comes from the S4' feature layer; the a3 feature output by the feature fusion module FEMv1 is then passed as input to the feature fusion module FFMv2; the feature fusion module FFMv2 fuses the S3' in the basic feature layer and the output a3 of the feature enhancement module (FEM) in the previous layer. Since the two have the same size, they are directly concatenated after compressing the number of channels through 1x1 convolution to obtain *S3, and the feature enhancement module FEMv2 is used to process the *S3 output feature layer (b3, b4, b5);

[0018] Next, the scale feature fusion module (SFFM) aggregates the previous multi-level multi-scale features (a3, a4, a5) and (b3, b4, b5) to build a multi-level feature pyramid; the scale feature fusion module (SFFM) first splices feature maps of the same scale in the channel dimension, and then, drawing on the concept of SENet (Squeeze-and-Excitation Networks), introduces the channel attention mechanism; by calculating the relationship between channels, the scale feature fusion module (SFFM) assigns different weights to each channel and focuses on features that contribute significantly to the task; finally, the channels are fused according to these weights, and the results (C3, C4, C5) are used as the input of the subsequent deep fusion encoder (DFE); through this channel-level attention mechanism, the model not only improves the accuracy of feature selection and effectively reduces the interference of redundant information, but also enhances the representation ability of useful features, thereby better supporting the performance of the model in the task;

[0019] The deep fusion encoder (DFE) includes an intrascale feature enhancement module (IFEM) and a cross-scale feature fusion module (CCFM); after the feature pyramid structure is operated, more representative multi-scale features (C3, C4, C5) are generated; by inputting C5 into the intrascale feature enhancement module (IFEM), the feature information is enhanced, thereby generating a new high-level feature layer F5; using (C3, C4, F5) to perform cross-scale feature fusion in the cross-scale feature fusion module (CCFM) to generate an image feature sequence as the input of the decoder;

[0020] The intrascale feature enhancement module (IFEM) is composed of four identical layers of VSSBlock; the initial input feature layer C5 is first subjected to layer normalization (LN) to standardize the input features and ensure the stability of the values; then, the input feature layer enters the spatial perception two-dimensional convolution block (SS2D Block), in which not only the spatial features are extracted, but also the perception of spatial information is enhanced, and the local structure and detail features in the image can be identified; the output of the spatial perception two-dimensional convolution block (SS2D Block) is superimposed with the initial input through a residual connection, effectively retaining the original feature information, and improving the performance of the subsequent model through the enhanced spatial features; the superimposed features are then further processed through a normalization layer to alleviate the gradient vanishing or exploding problems that may occur during the training process, and deep feature transformation and enhancement are performed through a feedforward network (FFN); finally, after the residual connection processing, the output feature layer obtained incorporates rich spatial information and context information; through this series of operations, the intrascale feature enhancement module (IFEM) can not only enhance the spatial expression ability of the image, but also better retain the original information, ensuring that the model fully understands the image details and context. This feature layer design has a significant effect on improving small target detection and occlusion processing in complex scenes;

[0021] In the processing flow of the spatially aware two-dimensional convolution block (SS2D Block), the input is first transformed linearly through a linear layer (Linear), and then features are further extracted through depthwise separable convolution (DWConv) and SiLU activation function; next, drawing on VMamba, an SS2D layer is added to enhance the understanding of the spatial structure and realize the extraction of spatial dependencies; the output of the SS2D layer is transformed through layer normalization (LN) and another linear layer (Linear) to provide optimized feature representation for subsequent modules.

[0022] Furthermore, in step S4: the experiment of training the road garbage detection model combining RT-DETR and state space was carried out on four Tesla T4s using the PyTorch framework; the Epoch was set to 70, the global weight decay value was 0.0001, the batch size was set to 16, the number of data loading subprocesses was 4, the global learning rate was set to 0.0001, and the AdamW optimizer hyperparameters were set to [0.9, 0.999]; after the training was completed, the trained model weights and training logs were saved for subsequent use and analysis.

[0023] Beneficial effects: This paper proposes a road garbage detection model that combines RT-DETR with state space, designs a new feature pyramid structure, and efficiently integrates multi-scale feature information, so that the model can more accurately identify targets in complex scenes. At the same time, an internal scale feature enhancement module based on state space is proposed, which significantly improves the accuracy of the model in occlusion and small target detection, and enhances the overall performance and robustness. In addition, the ablation experiment on the COCO2017 dataset further verifies the superiority and generalization ability of this method. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 The figure is a flow chart of a road garbage detection method combining RT-DETR and state space of the present invention.

[0025] Figure 2 Schematic diagram of different road garbage samples.

[0026] Figure 3 Schematic diagram of the overall structure of the road garbage detection model that combines RT-DETR and state space.

[0027] Figure 4 Schematic diagram of the innovative feature pyramid structure (FFS).

[0028] Figure 5 Schematic diagram of the structure of the feature fusion module (FFM).

[0029] Figure 6 Schematic diagram of the structure of the feature enhancement module (FEM).

[0030] Figure 7 Schematic diagram of the structure of the scale feature fusion module (SFFM).

[0031] Figure 8 Schematic diagram of the structure of the deep fusion encoder (DFE).

[0032] Fig. 9 Schematic diagram of the structure of the intrascale feature enhancement module (IFEM).

[0033] Fig.10 It is a schematic diagram for visualizing the experimental results of the present invention. DETAILED DESCRIPTION

[0034] The present invention will be further explained below in conjunction with the accompanying drawings.

[0035] The present invention proposes a road garbage detection method combining RT-DETR and state space. The specific implementation case is only used to illustrate the implementation of the present invention and does not limit its scope. The implementation steps of the present invention are described in detail below with reference to the accompanying drawings. Figure 1As shown, specifically including the following:

[0036] S1: Acquire road garbage images;

[0037] The present invention mainly collects road garbage images by self-collection and web crawlers, imitating the perspective of an unmanned sweeper to collect large, medium and small targets under different backgrounds, lighting, weather and other conditions; for classes with less data volume, the Stable Diffusion AI generation method is used for collection, and a total of more than 60,000 diverse road garbage images are collected.

[0038] S2: Annotate and augment images;

[0039] After completing the image collection, the data is cleaned and screened to remove invalid or low-quality samples to ensure data diversity; the VoTT tool is used to annotate the road garbage images and generate bounding boxes and labels; data enhancement is performed through methods such as rotation, flipping, cropping and splicing to increase sample diversity; the image data is divided into training set, validation set and test set in a ratio of 7:2:1, and organized into the standard COCO format, and finally the data set is exported to support model training and testing. The data set is divided into 12 categories, namely: cigarette butts, batteries, plastic bags, glass, metal, plastic waste, paper cups, garbage bags, waste clothing, paper bags, waste paper and fruit peels, such as Figure 2 shown.

[0040] S3: Construct a road garbage detection model combining RT-DETR and state space;

[0041] The main implementation method of the road garbage detection model combining RT-DETR and state space in step S3 is as follows:

[0042] Figure 3 Schematic diagram of the overall structure of the road garbage detection model that combines RT-DETR with state space. The model consists of a backbone network, a feature pyramid structure (FFS), a deep fusion encoder (DFE), and a transformer decoder with an auxiliary prediction head.

[0043] The model first normalizes the input image and then feeds it into the backbone network to extract the features of the last three stages (S3, S4, S5). These features are fed into an innovatively designed feature pyramid structure (FFS) to generate multi-scale features for encoder input. In the deep fusion encoder (DFE), the intra-scale feature enhancement module (IFEM) and the cross-scale feature fusion module (CCFM) jointly transform the multi-scale features into image feature sequences. Subsequently, an IoU-aware query selection mechanism is used to select a fixed number of image features as the initial object query for the decoder. The decoder generates a prediction box and its corresponding confidence score through iterative optimization and combined with the auxiliary prediction head.

[0044] The structural diagram of the innovative feature pyramid structure (FFS) is as follows Figure 4 As shown in the figure, the features extracted by the backbone network are first fused, and then the feature fusion module (FFM) and feature enhancement module (FEM) are used to extract more representative multi-scale features. Finally, these multi-level features are integrated into multi-scale features through the cascade feature fusion module (SFAM) as the input of the subsequent deep fusion encoder (DFE):

[0045] The feature fusion module FFMv1 first fuses the last three stage features (S3, S4, S5) extracted by the backbone network. Figure 5 As shown on the left. Then, the number of channels is compressed through 1x1 convolution, and upsampling is used to adjust the size of the feature map. Finally, after the splicing operation, the basic feature layer (S3', S4', S5') is obtained to provide input for the FEM module. The feature enhancement module (FEM) adopts a lightweight U-Net architecture, as shown in Figure 6 As shown. After completing the upsampling and downsampling operations, it performs element-by-element addition on the feature maps of the corresponding sizes to obtain the feature layer (a3, a4, a5). In addition, a 1x1 convolution layer is introduced in the module to enhance the learning ability of the model while maintaining the smoothness of the features. It is worth noting that FEMv1 includes an additional upsampling operation compared to FEMv2 because its input comes from the S4' feature layer. The a3 feature output by FEMv1 is then passed as input to FFMv2. FFMv2 fuses S3' in the basic feature layer and the output a3 of the previous FEM module. Since the two have the same size, they are directly concatenated after compressing the number of channels through 1x1 convolution to obtain *S3. The *S3 output feature layer (b3, b4, b5) is processed using the feature enhancement module FEMv2. FEMv2 is as shown Figure 5 Shown right.

[0046] Next, the scale feature fusion module (SFFM) aggregates the previous multi-level multi-scale features (a3, a4, a5) and (b3, b4, b5) to construct a multi-level feature pyramid, such as Figure 7 As shown. SFFM first splices feature maps of the same scale in the channel dimension. Next, it introduces the channel attention mechanism based on the concept of SENet (Squeeze-and-Excitation Networks). By calculating the relationship between channels, the SFFM module assigns different weights to each channel and focuses on features that contribute significantly to the task. Finally, the channels are fused according to these weights, and the results (C3, C4, C5) are used as the input of the subsequent deep fusion encoder (DFE). Through this channel-level attention mechanism, the model not only improves the accuracy of feature selection and effectively reduces the interference of redundant information, but also enhances the representation ability of useful features, thereby better supporting the performance of the model in the task.

[0047] Figure 8 This is a schematic diagram of the structure of the deep fusion encoder (DFE) of the present invention, which is mainly composed of an intrascale feature enhancement module (IFEM) and a cross-scale feature fusion module (CCFM). After the feature pyramid structure is operated, more representative multi-scale features (C3, C4, C5) are generated. By inputting C5 into the IFEM module, the feature information is enhanced, thereby generating a new high-level feature layer F5. Using (C3, C4, F5) to perform cross-scale feature fusion in CCFM, an image feature sequence is generated as the input of the decoder.

[0048] The Intrascale Feature Enhancement Module (IFEM) consists of four identical layers of VSSBlock, such as Fig. 9 As shown in the figure. The initial input feature layer C5 first undergoes layer normalization (LN) to standardize the input features and ensure numerical stability. Subsequently, the input feature layer enters the spatially aware two-dimensional convolutional block (SS2D Block), in which not only spatial features are extracted, but also the perception of spatial information is enhanced, which can identify local structures and detail features in the image. The output of the SS2D Block is superimposed with the initial input through residual connections, effectively retaining the original feature information, while improving the performance of subsequent models through enhanced spatial features. The superimposed features are then further processed through a normalization layer to mitigate the gradient vanishing or exploding problems that may occur during training, and deep feature transformation and enhancement are performed through a feedforward network (FFN). Finally, after the residual connection processing, the output feature layer obtained incorporates rich spatial and contextual information. Through this series of operations, IFEM can not only enhance the spatial expression ability of the image, but also better retain the original information, ensuring that the model fully understands the image details and context. This feature layer design has a significant effect on improving the detection of small targets in complex scenes and the processing of occlusion problems.

[0049] In the processing flow of SS2D Block, the input is first transformed by a linear layer, and then further features are extracted by depthwise separable convolution (DWConv) and SiLU activation function. Next, we refer to VMamba and add an SS2D layer to enhance the understanding of spatial structure and extract spatial dependencies. The output of the SS2D layer is transformed by layer normalization (LN) and another linear layer to provide optimized feature representation for subsequent modules.

[0050] S4: Training a road trash detection model combining RT-DETR and state space.

[0051] The initialization training hyper parameters of the road garbage detection model combining RT-DETR and state space are as follows:

[0052] The experiment of the present invention was conducted on four Tesla T4s using the PyTorch framework. The epoch was set to 70, the global weight decay value was 0.0001, the batch size was set to 16, the number of data loading subprocesses was 4, the global learning rate was set to 0.0001, and the AdamW optimizer hyperparameters were set to [0.9, 0.999]. After the training was completed, the trained model weights and training logs were saved for subsequent use and analysis.

[0053] S5: Use the trained model to detect road litter in the field and conduct experiments on the COCO2017 dataset.

[0054] The experimental results are as follows:

[0055] Table 1 compares this method with other methods on the COCO dataset. The experimental results on the COCO dataset show that our model performs well in target detection performance, achieving an average precision (AP) of 53.9%, which is the same as the commonly used real-time detector Yolov8, but has more advantages in the number of parameters (Params) and computational complexity (GFLOPs). Compared with the original model RT-DETR, the model of the present invention has improved in various detection indicators, and the average precision (AP) has increased from 53.1% to 53.9%. With only a slight increase in the number of parameters (Params) and computational complexity (GFLOPs), the detection accuracy has been successfully improved, fully demonstrating the effectiveness in feature fusion and model optimization. Especially in the detection of small targets, the accuracy has been improved by 1.8 percentage points, indicating that the improvements in feature fusion and enhancement of the model of the present invention have significantly enhanced the detection ability of small targets.

[0056] Table 1 Comparison of this method with other methods on the COCO dataset

[0057]

[0058] The comparison of this method with other methods on the self-made road garbage detection dataset is shown in Table 2. Compared with RT-DETR, the average precision (AP) of the model of the present invention is improved to 45.5%, which is an improvement over RT-DETR's 44.3%. While maintaining a similar number of training rounds (70 rounds), the number of parameters and computational complexity of the model of the present invention only increased slightly, showing good resource utilization efficiency. More importantly, the accuracy of the model of the present invention on small targets (APs) and medium targets (APm) reached 28.5% and 48.9%, respectively, further proving that the model of the present invention not only improves the overall detection performance, but also has a significant improvement in the processing ability of small-scale targets in practical applications. In general, the model of the present invention achieves a good balance between performance and efficiency, can meet the high-precision requirements in practical applications, and can still maintain excellent performance under limited computing resources, which makes it have broader application potential in real scenarios.

[0059] Table 2 Comparison of this method with other methods on the self-made road garbage detection dataset

[0060]

[0061] The experimental results of the present invention are visualized as follows Fig.10 As shown in the figure, the upper part is the detection result of the RT-DETR model, and the lower part is the detection result of the model of the present invention. The comparison shows that the model of the present invention has better detection performance under different lighting conditions, and has been enhanced in occlusion and small target detection. This performance improvement is due to the improvement of the model in feature extraction. First, the model in this paper introduces an innovative feature pyramid structure, which can effectively integrate feature information from different scales, thereby improving the model's perception of multi-scale objects. In addition, the newly designed internal scale feature enhancement module in the model can capture the detailed features and global context of the target object, and enhance the detection performance of similar category targets. Overall, these improvements significantly improve the detection accuracy and robustness of the model, making the model more suitable for target detection tasks in complex environments.

[0062] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A road garbage detection method combining RT-DETR and state space, characterized in that: The following steps are involved: S1: Acquire road garbage images; S2: Annotate and enhance images; S3: Construct a road garbage detection model combining RT-DETR and state space; S4: Training a road trash detection model combining RT-DETR and state space; S5: Use the trained model to detect road litter in the field and conduct experiments on the COCO2017 dataset.

2. A road garbage detection method combining RT-DETR and state space as claimed in claim 1, characterized in that: In step S1, road garbage images are collected by self-collection and web crawlers, and large, medium and small targets under different backgrounds, lighting and weather conditions are collected by imitating the perspective of an unmanned sweeper. For classes with less data volume, Stable DiffusionAI generation is used for collection, and a total of more than 60,000 diverse road garbage images are collected.

3. A road garbage detection method combining RT-DETR and state space as claimed in claim 1, characterized in that: In step S2, after completing the image collection, the data is cleaned and screened to remove invalid or low-quality samples to ensure data diversity; the road garbage images are annotated using the VoTT tool to generate bounding boxes and labels; data enhancement is performed through methods such as rotation, flipping, cropping and splicing to increase sample diversity; the image data is divided into a training set, a validation set and a test set in a ratio of 7:2:1, and organized into a standard COCO format, and finally the data set is exported to support model training and testing.

4. A road garbage detection method combining RT-DETR and state space as claimed in claim 1, characterized in that: In step S3, the road garbage detection model combining RT-DETR and state space is composed of a backbone network, a feature pyramid structure, a deep fusion encoder and a transformer decoder with an auxiliary prediction head: The model first normalizes the input image and then feeds it into the backbone network to extract the features of the last three stages: S3, S4, and S5. These features are fed into the feature pyramid structure to generate multi-scale features for encoder input. In the deep fusion encoder, the intra-scale feature enhancement module and the cross-scale feature fusion module jointly transform the multi-scale features into image feature sequences. Subsequently, an IoU-aware query selection mechanism is used to select a fixed number of image features as the initial object query for the decoder. The decoder generates a prediction box and its corresponding confidence score through iterative optimization and combined with the auxiliary prediction head. The innovative feature pyramid structure first fuses the features extracted by the backbone network, and then uses the feature fusion module and feature enhancement module to extract more representative multi-scale features; finally, through the cascade feature fusion module, these multi-level features are integrated into multi-scale features as the input of the subsequent deep fusion encoder: The feature fusion module FFMv1 first fuses the last three stage features extracted by the backbone network: S3, S4, and S5; then it compresses the number of channels through 1x1 convolution and uses upsampling to adjust the size of the feature map; finally, after the splicing operation, the basic feature layers S3', S4', and S5' are obtained to provide input for the feature enhancement module; the feature enhancement module adopts a lightweight U-Net architecture. After completing the upsampling and downsampling operations, it performs element-by-element addition on the feature maps of the corresponding sizes to obtain the feature layers: a3, a4, and a5; in addition, a 1x1 convolution layer is introduced in the module to enhance the learning ability of the model. The smoothness of the features is maintained; the feature fusion module FEMv1 includes an additional upsampling operation compared to the feature fusion module FEMv2 because its input comes from the S4' feature layer; the feature a3 output by the feature fusion module FEMv1 is then passed as input to the feature fusion module FFMv2; the feature fusion module FFMv2 fuses S3' in the basic feature layer and the output a3 of the feature enhancement module in the previous layer. Since the two have the same size, they are directly concatenated after compressing the number of channels through 1x1 convolution to obtain *S3, and the feature enhancement module FEMv2 is used to process the output feature layer of *S3: b3, b4, b5; Next, the scale feature fusion module aggregates the previous multi-level multi-scale features: a3, a4, a5 and b3, b4, b5 to build a multi-level feature pyramid; the scale feature fusion module first splices feature maps of the same scale in the channel dimension, and then, drawing on the concept of SENet, introduces the channel attention mechanism; by calculating the relationship between channels, the scale feature fusion module assigns different weights to each channel, focusing on features that contribute significantly to the task; finally, the channels are fused according to these weights, and the results: C3, C4, C5 are used as the input of the subsequent deep fusion encoder; The deep fusion encoder includes an inner scale feature enhancement module and a cross-scale feature fusion module; after the feature pyramid structure is operated, more representative multi-scale features are generated: C3, C4, C5; by inputting C5 into the inner scale feature enhancement module, the feature information is enhanced, thereby generating a new high-level feature layer F5; using: C3, C4, F5 to perform cross-scale feature fusion in the cross-scale feature fusion module to generate an image feature sequence as the input of the decoder; The inner scale feature enhancement module is composed of four identical layers of VSSBlock; the initial input feature layer C5 is firstly subjected to layer normalization to standardize the input features and ensure the stability of the values; then, the input feature layer enters the spatial perception two-dimensional convolution block; the output of the spatial perception two-dimensional convolution block is superimposed with the initial input through residual connection, effectively retaining the original feature information, while improving the performance of the subsequent model through enhanced spatial features; the superimposed features are then further processed through the normalization layer to alleviate the gradient vanishing or exploding problems that may occur during the training process, and deep feature transformation and enhancement are performed through the feedforward network; finally, after the residual connection processing, the output feature layer obtained integrates rich spatial information and context information; In the processing flow of the spatially aware two-dimensional convolutional block, the input is first linearly transformed through a linear layer, and then features are further extracted through depthwise separable convolution and SiLU activation function; next, drawing on VMamba, an SS2D layer is added to enhance the understanding of the spatial structure and realize the extraction of spatial dependencies; the output of the SS2D layer is transformed through layer normalization and another linear layer to provide optimized feature representation for subsequent modules.

5. A road garbage detection method combining RT-DETR and state space as claimed in claim 1, characterized in that: In the step S4: the experiment of training the road garbage detection model combining RT-DETR and state space is carried out on four Tesla T4s using the PyTorch framework; the Epoch is set to 70, the global weight decay value is 0.0001, the batch size is set to 16, the number of data loading subprocesses is 4, the global learning rate is set to 0.0001, and the AdamW optimizer hyperparameters are set to [0.9, 0.999]; After the training is completed, the trained model weights and training logs are saved for subsequent use and analysis.

Images