Road trash detection method based on dynamic ranking perception RT-DETR

Abstract

The application discloses a road garbage detection method based on dynamic ranking perception RT-DETR, including the following steps: constructing a road garbage detection dataset and obtaining a COCO standard dataset; image labeling and data enhancement; constructing an improved RT-DETR model based on dynamic ranking perception; training the improved RT-DETR model based on dynamic ranking perception; using the trained improved RT-DETR model based on dynamic ranking perception to detect road garbage, and performing experiments on the self-built dataset and the COCO standard dataset. Experiments prove that the application reduces the interference of high-confidence but inaccurate positioning prediction boxes on the detection results, and suppresses the missing detection and false detection phenomenon of small targets, occluded targets and complex background scenes, thereby improving the accuracy, positioning accuracy and model robustness of road garbage target detection.

Description

Technical Field

[0001] This invention relates to the field of object detection using deep learning, specifically to a road litter detection method based on Dynamic Ranking Awareness RT-DETR. Background Technology

[0002] With the continuous development of computer vision and deep learning technologies, object detection technology has been widely applied in fields such as intelligent transportation, environmental monitoring, and smart cities. Road litter detection, as an important application of intelligent environmental sensing, is of great significance for improving the efficiency of road cleaning management and ensuring road environmental safety.

[0003] Among existing object detection methods, RT-DETR, as a Transformer-based real-time object detection model, has achieved a good balance between detection accuracy and inference speed. However, existing RT-DETR models still suffer from a mismatch between classification confidence scores and bounding box localization quality. That is, some prediction results have high classification confidence but insufficient localization accuracy, thus affecting the ranking quality of detection results and overall detection performance.

[0004] Especially in road litter detection tasks, litter targets are typically small in size, diverse in shape, and easily affected by background interference. Furthermore, they are prone to missed detections and false detections under conditions of occlusion, dense distribution, and complex lighting. Therefore, the detection accuracy and robustness of existing methods still need improvement. To address this, it is necessary to propose a road litter detection method based on dynamic sorting perception RT-DETR to improve the accuracy and positioning precision of road litter detection. Summary of the Invention

[0005] Purpose of the invention: To address the shortcomings of the existing technologies, a road waste detection method based on dynamic sorting perception RT-DETR is proposed.

[0006] The technical solution adopted in this invention is as follows:

[0007] A road litter detection method based on dynamic sorting perception RT-DETR includes the following steps:

[0008] S1: Construct a road litter detection dataset and obtain the COCO standard dataset;

[0009] S2: Image tagging and data augmentation;

[0010] S3: Construct an improved RT-DETR model based on dynamic order awareness;

[0011] S4: Train an improved RT-DETR model based on dynamic order awareness;

[0012] S5: Detect road litter using a trained, improved RT-DETR model based on dynamic sorting awareness, and conduct experiments on a self-built dataset and the COCO standard dataset.

[0013] Preferably, in step S2, the constructed road litter detection dataset undergoes image labeling, data augmentation, and dataset partitioning. Specifically, this includes: First, classifying and annotating the targets in the road litter images to obtain category and spatial location information of the road litter targets; wherein, the road litter targets include at least typical road litter targets such as plastic bags and garbage bags. Then, data augmentation is performed on the labeled image samples. The data augmentation includes one or more of random flipping, random cropping, random scaling, and color perturbation to enhance the model's adaptability to targets of different scales, complex backgrounds, occluded targets, and scenes with varying lighting. Finally, the processed road litter detection data is input into the subsequent model training process and combined with the COCO standard dataset for improving the model's generalization performance verification.

[0014] Preferably, in step S3, the improved RT-DETR model based on dynamic ranking perception is a road waste target detection model that introduces a dynamic ranking perception mechanism, GIoU perception classification loss, and a high-order matching strategy on the basis of RT-DETR; wherein, the dynamic ranking perception mechanism includes a dynamic scoring module based on classification confidence scores and a query ranking module based on dynamic scores; the GIoU perception classification loss is used to improve the consistency between classification results and bounding box localization results, and the high-order matching strategy is used to improve the matching accuracy between predicted boxes and ground truth boxes. The improved RT-DETR model first utilizes the encoder's output feature sequence containing image information and selects a fixed number of high-quality features through an IoU-aware query selection mechanism to initialize object queries in the decoder. Then, a dynamic scoring module based on classification confidence scores is introduced between layers before the penultimate decoder layer to analyze and correct the classification confidence distribution of the current layer's object queries. Subsequently, a query ranking module based on dynamic scoring is introduced after each dynamic scoring module to reorder, reduce, and reconstruct the object queries to enhance high-confidence target prediction. Next, a GIoU-aware classification loss is introduced during the training phase to improve the consistency between classification and localization by combining bounding box localization quality information. Finally, a higher-order matching strategy is introduced during the Hungarian matching process to integrate classification confidence scores, bounding box coordinate distances, and the intersection-union ratio (IoU) between the predicted and ground truth boxes, thereby obtaining better matching results between predicted and ground truth boxes.

[0015] Preferably, in step S3, the specific operation flow of the dynamic scoring module based on classification confidence scores is as follows: First, between the layers before the penultimate decoder, the object query output by the current Transformer decoder is decoupled using a dedicated projection network to obtain a classification query; then, the classification query is input into a multilayer perceptron to generate an independent classification score corresponding one-to-one with the object query; next, the classification score is added to a set of learnable logit bias vectors to dynamically correct the original classification score; subsequently, the corrected score is mapped to a probability interval using a sigmoid function to obtain the final classification confidence score; finally, the classification confidence score is passed to the subsequent query ranking module based on dynamic scoring to provide a basis for prioritizing object queries. By introducing the learnable logit bias vectors, the classification score can be dynamically fine-tuned without changing the original detection structure, thereby improving the ability to distinguish between true positive predictions and noise predictions.

[0016] Preferably, in step S3, the specific operation flow of the query ranking module based on dynamic scoring is as follows: First, the object query output by the current layer Transformer decoder is decoupled to obtain the classification query and the location query; then, the classification confidence score vector is obtained from the previous dynamic scoring module, and the maximum value of the classification confidence score corresponding to each object query is used as the ranking basis; then, the classification query and the location query are sorted in descending order according to the ranking basis, and the sorted classification query and location query are reduced by a progressively decreasing retention ratio, retaining only the top few high-scoring queries to reduce redundant calculations and retain high-quality target information; subsequently, the sorted and reduced classification query is concatenated and fused with a set of randomly initialized and continuously optimized content queries during training, and then mapped back to the original dimension through a fully connected layer to obtain a new classification query; finally, the new classification query and the sorted location query are concatenated by features, and reconstructed into an enhanced object query through linear projection and residual connection, and then input into the subsequent decoding layer for iterative optimization. The query ranking module can continuously incorporate ranking information during the decoding process, making the model pay more attention to high-confidence targets and improving detection efficiency and positioning accuracy while ensuring detection recall.

[0017] Preferably, in step S3, the specific implementation of the GIoU-aware classification loss is as follows: First, during training, the generalized intersection-union (GIoU) ​​between the predicted bounding box and the ground truth bounding box is introduced as a supervision signal; then, the GIoU score is normalized and introduced into the classification loss function; next, the GIoU-aware classification loss is used to simultaneously constrain the target category prediction result and the bounding box localization result, so that the classification head not only learns the semantic category information of the target, but also optimizes the classification result based on the overlap quality between the predicted box and the ground truth box; finally, the GIoU-aware classification loss improves the classification accuracy and localization accuracy of the model under high IoU threshold conditions. The GIoU-aware classification loss can effectively alleviate the problem of mismatch between classification confidence score and bounding box localization quality, and is especially suitable for detection tasks of small targets and targets with large shape changes.

[0018] Preferably, in step S3, the higher-order matching strategy is implemented as follows: First, a matching cost matrix between the predicted bounding box and the ground truth bounding box is constructed during the Hungarian matching process. Then, the matching cost matrix comprehensively considers the classification confidence score, the L1 distance of the bounding box coordinates, and the intersection-union ratio (IU) between the predicted and ground truth bounding boxes. Next, a larger localization accuracy weight parameter is introduced into the matching cost function to enhance the impact of bounding box localization accuracy on the matching process. Subsequently, the optimal matching relationship between the predicted and ground truth bounding boxes is obtained from the cost matrix using the Hungarian algorithm. Finally, by prioritizing queries with higher classification confidence scores and higher IoU scores as positive samples and suppressing negative samples with higher classification confidence scores but lower IoU scores, the model training phase focuses more on high-quality predicted targets. This higher-order matching strategy can further enhance the matching rationality and detection stability of the model in complex scenes, small target scenes, and occlusion scenes.

[0019] Preferably, in step S4, the initialization of training hyperparameters includes: model training is performed in the PyTorch framework, the number of training epochs is set to 72, the global weight decay value is set to 0.0001, the batch size is set to 16, the number of data loading subprocesses num_workers is set to 4, the initial learning rate is set to 0.0001, and the optimizer is the AdamW optimizer; after training, the trained model weights and training logs are saved for subsequent analysis and testing.

[0020] Preferably, in step S5, the improved RT-DETR model based on dynamic ranking perception, after training, is tested on a self-built road litter detection dataset and the COCO standard dataset. Test images are input into the improved RT-DETR model, which outputs corresponding road litter target detection results. One or more evaluation metrics selected from AP, AP50, AP75, APs, APm, and APl are used to quantitatively evaluate the model's detection performance, verifying its effectiveness in road litter detection and its generalization ability in general target detection tasks. Specifically, comparative experiments, ablation experiments, and visualization analysis results on the self-built road litter detection dataset show that the improved model exhibits better detection accuracy and localization stability in small target detection, occluded target detection, and complex background scenes. Generalization experiments on the COCO standard dataset show that the improved model, while maintaining good computational efficiency, can further improve average detection accuracy and detection performance under high IoU thresholds.

[0021] Beneficial effects: This invention effectively alleviates the mismatch between classification confidence scores and bounding box localization quality in existing RT-DETR models, while reducing the interference of high-confidence but inaccurately localized predicted boxes on detection results; by introducing a dynamic scoring module based on classification confidence scores and a query ranking module based on dynamic scoring, the ranking quality of object queries and the retention of high-quality queries are improved; by introducing GIoU-aware classification loss and higher-order matching strategies, the model's perception of bounding box localization quality and matching accuracy are enhanced, thereby improving the detection accuracy, localization accuracy, and model robustness of small targets, occluded targets, and complex scenes in road waste detection tasks. Attached Figure Description

[0022] Figure 1 This is a flowchart of the road waste detection method based on dynamic sorting perception RT-DETR of the present invention.

[0023] Figure 2 This is a sample example of a road litter detection dataset.

[0024] Figure 3 This is a structural diagram of the dynamic sorting perception decoder in the road waste detection method based on dynamic sorting perception RT-DETR of the present invention.

[0025] Figure 4 This is a flowchart of the dynamic scoring module for the classification confidence score of the road waste detection method based on dynamic sorting perception RT-DETR of the present invention.

[0026] Figure 5 This is a flowchart of the query and sorting module for dynamic scoring in the road waste detection method based on dynamic sorting perception RT-DETR of the present invention.

[0027] Figure 6 This diagram illustrates the perceptual classification loss function and higher-order matching cost function method for enhancing positioning accuracy in the road waste detection method based on dynamic sorting perception RT-DETR of this invention.

[0028] Figure 7 This is a visual comparison of the experimental results of this method on a self-made road waste detection dataset.

[0029] Figure 8 The experimental results of this method on the COCO dataset are visualized and compared. Detailed Implementation

[0030] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0031] This invention proposes a road litter detection method based on dynamic sorting perception RT-DETR, the overall process of which is as follows: Figure 1 As shown, the specific steps include:

[0032] S1: Construct a road litter detection dataset and obtain the COCO standard dataset.

[0033] First, images of litter in real-world road scenes were collected to construct a self-built road litter detection dataset. The road litter targets include at least typical lightweight road litter such as plastic bags and garbage bags, and can be expanded to other litter categories such as paper scraps, packaging materials, and beverage bottles as needed for practical applications. Simultaneously, the COCO standard dataset was obtained as an auxiliary validation dataset to evaluate the generalization performance of the improved model. Figure 2 An example sample of the road litter detection dataset is shown.

[0034] S2: Image tagging and data augmentation.

[0035] The road debris images constructed in step S1 are manually labeled to obtain the category label and bounding box coordinates of each target. To enhance the model's adaptability to complex backgrounds, occluded targets, small targets, and scenes with varying scales, data augmentation processing is performed on the labeled images. The data augmentation methods include, but are not limited to, random flipping, random cropping, random scaling, and color perturbation. After augmentation, the data is divided into training, validation, and test sets for subsequent model training and performance evaluation.

[0036] S3: Construct an improved RT-DETR model based on dynamic order awareness.

[0037] Using the RT-DETR model as the basic detection framework, this invention improves its Transformer decoder structure to construct a road litter detection model based on dynamic order awareness. Unlike existing RT-DETR methods that primarily utilize order information in the initial query selection phase, this invention introduces a dynamic order awareness mechanism throughout the entire Transformer decoder process, enabling the model to continuously utilize order information during layer-by-layer decoding. Figure 3 As shown, specifically, the present invention is in the penultimate layer (the... A dynamic scoring module based on classification confidence scores is added between each layer before the decoder layer. Through this module, the model can dynamically adjust the classification confidence scores according to the features of the current layer, thereby more accurately distinguishing true positives from noise predictions. At the same time, after each dynamic scoring module based on classification confidence scores, this invention sequentially embeds a query ranking module based on dynamic scores. This module further optimizes the ranking effect of object queries, making subsequent processing more focused on high-confidence predictions.

[0038] First, the dynamic scoring module based on classification confidence scores will be explained: The dynamic scoring module based on classification confidence scores proposed in this invention innovatively introduces a set of learnable logit bias vectors. And add it to the category confidence score that corresponds independently to each object query. Then, after passing through the Sigmoid function, we obtain the first... Classification confidence score of layer decoder :

[0039]

[0040] in , indicating the first The query vector output by the Transformer Decoder layer, and and The hidden dimension is the total number of categories. The entire processing flow of the dynamic scoring module based on classification confidence scores is as follows: Figure 4 As shown.

[0041] Specifically, each Transformer decoder layer outputs an object query. On the second to last floor ( Between each layer before the decoder layer, the model first uses a dedicated projection network to decouple its features into classification queries. . Then, through an MLP, it generates object queries. One-to-one independent classification scores This step queries each object Preliminary classification information was provided. Next, to introduce ranking-aware score calibration and dynamically adjust scores, these scores are correlated with a set of learnable logit bias vectors. The scores are added together to fine-tune the original scores. Here... This can be viewed as a correction factor, the purpose of which is to dynamically adjust the scores to better reflect the true ranking of each query without changing the overall prediction structure. This step is equivalent to injecting prior information and global optimization signals into each object query.

[0042] After logit correction, the result is then converted into a probability value using the Sigmoid function. The purpose of the Sigmoid function is to map the corrected score to... Within the interval, the numerical range is standardized to obtain the final classification confidence score. Finally, the classification confidence score was calculated. It is then passed to the subsequent query sorting module based on dynamic scoring.

[0043] Secondly, the query ranking module based on dynamic scoring is described: To make fuller use of ranking information, this invention embeds a query ranking module based on dynamic scoring sequentially after the dynamic scoring module based on classification confidence scores. The main function of this module is to reprocess existing queries, regenerating ranked position and content queries, thereby enabling query features to more accurately reflect the ranking requirements in object detection. The entire processing flow is as follows: Figure 5 As shown. In other words, the query ranking module based on dynamic scoring will use the classification confidence scores obtained by the previous dynamic scoring module. The object query is reordered and optimized, so that the final query is not only more accurate in position, but also better reflects the sorting advantage in semantic information expression.

[0044] First, this invention will detail how to construct a content query with dynamic ranking awareness, a step that is crucial for ensuring the effectiveness of query ranking throughout the entire detection process:

[0045]

[0046] Specifically, the present invention first addresses the first layer( Object query output from the Transformer decoder Using a dedicated projection network, its features are decoupled into classification queries. and location query Then, the classification confidence score vector is obtained from the previous dynamic scoring module based on classification confidence scores. .because Each element in is a This invention uses a dimensional vector and employs the maximum value of each category confidence score in this vector as the sorting criterion. Based on the obtained category confidence scores, the classification query is then sorted. and location query Perform a descending sort operation, then use a progressively decreasing retention ratio. The category query and location query were reduced (five query blocks in the diagram were reduced to four query blocks), retaining the previous ones. The queries that are selected will be removed to optimize computational efficiency. This will result in sorted categorized queries. and location query The present invention defines operation symbols. Used according to The elements in the array are arranged in descending order. Sort the corresponding elements in the array and define the operation symbols. Used according to proportion ,right Perform a deletion operation on the elements in the file, keeping the first few elements. of middle element, after deletion of In this implementation, the retention ratio is precisely controlled by the hyperparameter `keep_ratio`. A high query retention rate is maintained in the shallow layers (early stages) of the model to ensure that potential targets (especially small and occluded targets) are not filtered out prematurely, effectively maintaining detection recall. As the network depth increases, the selection criteria are gradually raised, retaining only the most discriminative, high-quality queries in the deeper layers (later stages). The final output layer retains all queries, ensuring the integrity of the prediction results. From a computational efficiency perspective, this scheme achieves an exponential decrease in computational complexity (FLOPs) through a progressive deletion strategy. In deep networks, the system only needs to process a small number of high-value queries, thus concentrating limited computational resources on key detection tasks. Compared to a fixed-ratio deletion scheme, this method significantly improves inference speed while maintaining excellent detection accuracy.

[0047] Next, this invention will perform a sorted and reduced category query. Querying content with a set of features that are randomly initialized along the feature dimensions. Perform Concat concatenation (using symbols) (This indicates a concatenation operation). This set of content can be queried. The entire training process employs an end-to-end approach for continuous optimization, enabling the capture of common semantic information to better adapt to the needs of detection tasks. Subsequently, this invention utilizes fully connected layers ( The concatenated features are then fused back into the original dimensions to obtain a new classification query. In other words, this invention maintains a set of static content embeddings with hierarchy-aware information shared across different samples. This embedding can effectively simulate and capture the most common semantic information distributions, thereby further improving the model's detection performance in complex scenes.

[0048] This study examines categorized queries generated through sorting, reduction, and merging. Searching for sorted positions Feature stitching is performed, and the data is reconstructed into an enhanced object query through linear projection and residual connections. The data is then fed into subsequent decoding layers for iterative optimization. These subsequent layers can utilize this rich feature information for further fusion and processing. This design achieves effective fusion of classification semantics and spatial geometric information, enabling subsequent decoding layers to perform attention calculations based on richer features, thereby significantly improving the model's ability to detect complex scenes.

[0049] The key objective of the structural design of the above two modules is to utilize the classification confidence score ranking information of object queries to dynamically adjust object queries through collaborative optimization, thereby improving the performance of the object detection model. In these two modules, this invention introduces two sets of learnable representations: one is the logit bias vector. The other set is the object query vector. These two sets of vectors refine the object query from different perspectives, enhancing the model's ability to perceive ranking information.

[0050] Specifically, the logit bias vector By adjusting the classification confidence score, the model can assign different weights to different prediction results based on the quality of the classification confidence score; while the classification query vector By fusing semantic information from the queries, the semantic representation is enhanced, further improving the model's ability to identify targets. This dynamic ranking-aware design allows the model to adjust each object query in the decoder according to the ranking of the classification queries, thereby improving the final detection accuracy.

[0051] like Figure 6As shown, the GIoU-aware classification loss used in this invention will be further explained. This invention introduces the GIoU score into the classification loss, which essentially guides the model during training not only to correctly classify the target category but also to optimize the localization results of the bounding boxes. This method differs from traditional binary target-based classification heads; it utilizes GIoU as a supervisory signal, allowing the model to focus more on the accuracy of the bounding boxes while classifying. This design significantly improves the overall detection performance of the model, especially for small targets or targets with significant shape variations.

[0052] The introduction of normalized GIoU scores enables the model to effectively classify and distinguish between different targets, avoiding the class imbalance problem that may exist in traditional methods. During training, the model not only learns the class information of each target, but also optimizes the target detection results based on the accuracy of the bounding boxes, thereby achieving higher classification and localization accuracy. This GIoU-aware classification loss significantly improves the performance of the target detection model, especially at high Intersection over Union (IoU) thresholds, providing more accurate target localization and classification results.

[0053]

[0054] To simultaneously optimize the accuracy of semantic classification and bounding box localization, this invention improves the original classification head structure, adopting a new structure... A supervised GIoU-aware classification head. In this way, the present invention enables the classification head not only to predict the category of the target, but also to adjust the classification loss based on the GIoU score between the predicted bounding box and the ground truth bounding box, thereby improving the accuracy of the detection results.

[0055] It is worth noting that when At this point, the GIoU value is at its maximum, indicating that the predicted bounding box and the ground truth bounding box completely overlap. This will be simplified to the original focus loss function. This refers to the standard classification loss function. Therefore, the method of this invention has greater flexibility and adaptability compared to the traditional focus loss, and can effectively fuse semantic and spatial information.

[0056] To further emphasize the advantage of positioning accuracy, this invention also introduces a magnification factor. To enhance the impact of accurate positioning scores. Specifically, It is the ratio between accurate location scores and inaccurate location scores. It amplifies the impact of accurate location predictions during the optimization process, thus making the model more inclined to select accurate bounding box predictions.

[0057] To further verify the effectiveness of the method of this invention, it was compared with zoom loss. Zoom loss is commonly used to handle class imbalance problems by dynamically adjusting the loss weights of samples from different classes, so that the model pays more attention to samples that are difficult to classify during training. The mathematical formula for zoom loss can be expressed as:

[0058]

[0059] in, This represents the predicted probability of the target category. It is the class balance coefficient. This refers to the hyperparameters that adjust between easy and difficult samples. In contrast, the GIoU-aware classification loss of this invention further improves the model's performance at high IoU thresholds by incorporating the localization information of the bounding boxes.

[0060] In summary, the GIoU perceptual classification loss proposed in this invention can not only improve classification accuracy, but also significantly enhance the accuracy of target box localization, especially under high crossover union ratio (CUB) conditions.

[0061] like Figure 6 As shown, the high-order matching strategy described in this invention will be further explained. In the object detection task, the Hungarian algorithm achieves the optimal allocation of predicted and ground truth boxes by optimizing the comprehensive matching cost. This algorithm weights and fuses three indicators: classification confidence error, bounding box L1 distance, and GIoU geometric similarity, to construct a comprehensive matching cost function. As a classic method in the field of combinatorial optimization, the Hungarian algorithm is characterized by its computational efficiency of polynomial time complexity (typically...). The guarantee of both the target assignment and the global optimum demonstrates unique advantages in Transformer-based detection frameworks such as DETR. This matching mechanism not only effectively solves the uncertainty of target assignment in traditional detection methods, but also avoids complex preprocessing and hyperparameter tuning through end-to-end training, significantly improving the model's practicality and generalization ability.

[0062] Specifically, assuming there is One prediction box and Given a set of ground truth bounding boxes, this invention requires matching the predicted bounding boxes with the ground truth bounding boxes one by one and calculating the cost of the matching:

[0063] First, construct the cost matrix, which is a... A matrix, where each element represents the matching cost between the predicted bounding box and the ground truth bounding box. For example, if using... As a cost metric, each element in the matrix represents the difference between the predicted bounding box and the ground truth bounding box. Value. This invention typically uses... As a price, this The higher the matching value, the lower the cost:

[0064]

[0065] in, Indicates the first One prediction box, Indicates the first A real frame, It is the matching cost between the predicted bounding box and the ground truth bounding box.

[0066] After the cost matrix is ​​generated, the Hungarian algorithm is used for optimal matching. This algorithm ensures that each predicted bounding box matches as closely as possible to a ground truth bounding box by selecting the pair with the minimum cost. After matching is completed, the object detection algorithm calculates the loss function based on the matching results of the Hungarian algorithm. The matching cost is typically constructed by combining multiple loss functions to ensure that the matching between the predicted and ground truth bounding boxes considers not only location accuracy but also category accuracy. Common matching costs usually consist of L1 loss (regression loss) and IoU loss (Intersection over Union loss). Therefore, the matching cost can be defined as:

[0067]

[0068] in: and They represent the first The first prediction box and the first prediction box A real frame. This represents the intersection-union ratio (IUU) between the predicted bounding boxes and the ground truth bounding boxes. This represents the L1 distance between the predicted bounding box and the ground truth bounding box. and These are two weighted hyperparameters used to balance the impact of IoU and L1 loss. This cost function combines location accuracy and classification accuracy, enabling the Hungarian algorithm to maximize... At the same time, it reduces the positional error of the frame.

[0069] However, this invention proposes a novel higher-order matching cost function that can capture a more complex combination of classification confidence scores and IoU scores:

[0070]

[0071] Specifically, this matching cost function considers not only the classification information of the bounding boxes but also the intersection-union ratio (IUU) between the predicted and ground truth bounding boxes. In this design, This indicates the degree of overlap between the predicted bounding box and the actual value.

[0072] To emphasize the importance of positioning accuracy, this invention introduces a large hyperparameter into the higher-order matching cost function. By using this parameter, the present invention can place greater emphasis on the accuracy of bounding box localization, thereby promoting the model to generate more accurate bounding box predictions and reducing interference from inaccurate predictions. This approach helps improve overall detection performance, especially in scenarios with high precision requirements.

[0073] It is worth noting that the high-order matching strategy of this invention mainly plays a role in the middle stage of training. This is because in the early stage of training, model predictions are usually coarse and the localization accuracy is poor. Therefore, introducing high-order matching costs helps to improve the localization quality of the model and promotes the model to converge toward more accurate prediction results.

[0074] By introducing GIoU awareness into the classification confidence score, this invention enables more accurate ranking in the query ranking module based on dynamic scoring. Specifically, this method helps the model better evaluate and rank the prediction results for each query by considering the localization quality of the bounding boxes. Building on this, a higher-order matching strategy effectively suppresses challenging negative samples with high classification confidence scores but low IoU scores by prioritizing queries with both high classification confidence scores and high IoU scores as positive samples. This allows the model to focus more on high-quality prediction results, thereby improving overall performance.

[0075] S4: Train an improved RT-DETR model based on dynamic order awareness.

[0076] The model was trained in the PyTorch framework with 72 training epochs, a global weight decay of 0.0001, a batch size of 16, 4 data loading subprocesses (num_workers), a learning rate of 0.0001, and the AdamW optimizer.

[0077] S5: Detect road litter using a trained, improved RT-DETR model based on dynamic sorting awareness, and conduct experiments on a self-built dataset and the COCO standard dataset.

[0078] like Figure 7 , Figure 8As shown, the improved RT-DETR model based on dynamic ranking awareness, after training, was tested on a self-built road litter detection dataset and the COCO standard dataset. Test images were input into the improved RT-DETR model, which outputs corresponding road litter target detection results. One or more evaluation metrics, including AP, AP50, AP75, APs, APm, and APl, were used to quantitatively evaluate the model's detection performance, verifying its effectiveness in road litter detection and its generalization ability in general target detection tasks. Specifically, comparative experiments, ablation experiments, and visualization analysis results on the self-built road litter detection dataset show that the improved model exhibits better detection accuracy and localization stability in small target detection, occluded target detection, and complex background scenes. Generalization experiments on the COCO standard dataset show that the improved model, while maintaining good computational efficiency, can further improve average detection accuracy and detection performance at high IoU thresholds.

[0079] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A road litter detection method based on dynamic sorting perception RT-DETR, characterized in that, Includes the following steps: S1: Construct a road litter detection dataset and obtain the COCO standard dataset; S2: Image tagging and data augmentation; S3: Construct an improved RT-DETR model based on dynamic order awareness; S4: Train an improved RT-DETR model based on dynamic order awareness; S5: Detect road litter using a trained, improved RT-DETR model based on dynamic sorting awareness, and conduct experiments on a self-built dataset and the COCO standard dataset.

2. The road litter detection method based on dynamic sorting perception RT-DETR according to claim 1, characterized in that, In step S1, the road litter detection dataset is a self-built road litter detection dataset, and the COCO standard dataset is used to verify the generalization performance of the improved RT-DETR model; the detection targets in the road litter detection dataset include at least road litter targets such as plastic bags and garbage bags.

3. The road litter detection method based on dynamic sorting perception RT-DETR according to claim 2, characterized in that, In step S2, the image labeling includes class labeling and bounding box labeling of road litter targets, and the data augmentation includes one or more of random flipping, random cropping, random scaling and color perturbation to improve the model's adaptability to complex scenes, occluded scenes and small target scenes.

4. The road litter detection method based on dynamic sorting perception RT-DETR according to claim 3, characterized in that, In step S3, the improved RT-DETR model based on dynamic ranking awareness uses RT-DETR as the base model and introduces a dynamic ranking awareness mechanism in the Transformer decoder. The dynamic ranking awareness mechanism includes a dynamic scoring module based on classification confidence scores and a query ranking module based on dynamic scores.

5. The road litter detection method based on dynamic sorting perception RT-DETR according to claim 4, characterized in that, The dynamic scoring module based on classification confidence scores is set between each layer before the penultimate decoder layer. It is used to adaptively adjust the reliability assessment of the prediction results according to the classification confidence distribution of the current layer object query, so as to improve the ability to distinguish between positive and negative samples. The dynamic scoring module based on classification confidence scores includes: [The following is a list of parameters, not a direct translation] The object query output by the Transformer decoder is decoupled into a classification query by the projection network, and then an independent classification score corresponding one-to-one with the object query is generated by the multilayer perceptron. The classification score is added to a set of learnable logit bias vectors and then mapped by the Sigmoid function to obtain the final classification confidence score. The classification confidence score is then passed to the subsequent query ranking module based on dynamic scoring.

6. The road litter detection method based on dynamic sorting perception RT-DETR according to claim 5, characterized in that, The query sorting module based on dynamic scoring is set after the dynamic scoring module based on classification confidence score. It is used to re-sort, reduce and reconstruct the object query according to the classification confidence score output by the dynamic scoring module, so that the subsequent decoding process focuses more on high confidence target prediction. The query sorting module includes: decoupling the object query output by the current layer Transformer decoder into a category query and a location query; The category queries and location queries are sorted in descending order based on their category confidence scores. A progressively decreasing retention ratio is used to reduce the number of sorted category queries and location queries, keeping only the top-scoring queries. The sorted and reduced category queries are then concatenated and merged with the content queries to obtain new category queries. These new category queries are then merged with the sorted location queries to reconstruct an enhanced object query, which is then input into the subsequent decoding layer for iterative optimization.

7. The road litter detection method based on dynamic sorting perception RT-DETR according to claim 6, characterized in that, Step S3 also includes GIoU-aware classification loss. The GIoU-aware classification loss introduces GIoU scores into the classification loss and uses the normalized GIoU scores as a supervision signal, so that the model can optimize the bounding box localization results while learning target category information, thereby improving the classification accuracy and localization accuracy of the model under high IoU threshold.

8. The road litter detection method based on dynamic sorting perception RT-DETR according to claim 7, characterized in that, Step S3 also includes a higher-order matching strategy. The higher-order matching strategy constructs a higher-order matching cost function between the predicted bounding box and the ground truth bounding box, comprehensively considering the classification confidence score, the bounding box coordinate distance, and the intersection-union ratio information between the predicted bounding box and the ground truth bounding box. It also enhances the impact of bounding box positioning accuracy on the matching process by increasing the positioning accuracy weight parameter, so that the model prioritizes queries with higher classification confidence scores and higher IoU scores as positive samples, and suppresses negative samples with high classification confidence scores but low IoU scores.

9. The road litter detection method based on dynamic sorting perception RT-DETR according to claim 8, characterized in that, In step S4, the model training is performed in the PyTorch framework, with the number of training epochs set to 72, the global weight decay value set to 0.0001, the batch size set to 16, the number of data loading subprocesses num_workers set to 4, the learning rate set to 0.0001, and the AdamW optimizer used.

Images