Non-motor vehicle target detection method based on multi-scale feature integration and residual enhancement
By improving the multi-scale feature integration and residual enhancement techniques of the RT-DETR model, the problem of low detection accuracy of non-motorized vehicle targets was solved, and the detection accuracy and the ability to identify small targets were improved.
Patent Information
- Application Number
- CN202510988483.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-17
- Publication Date
- 2025-10-31
AI Technical Summary
Existing detection methods are not accurate enough for non-motorized vehicle targets, especially in urban traffic environments where they are easily obscured and come in a variety of types, leading to missed detections and false detections. Detection of small targets is particularly difficult at long distances.
An improved RT-DETR model based on multi-scale feature integration and residual enhancement is adopted. The EEH module improves the feature map detail information, the MST-MEIE module is constructed to extract multi-scale features, the PKC module is used to extract features at different scales, and the DPBF module enhances the model's expressive and generalization capabilities.
It improves the accuracy of non-motorized vehicle target detection, reduces missed detections and false detections, especially the ability to detect small targets under obstructed and long-distance conditions.
Smart Images
Figure CN120877221A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision target detection technology, and in particular to a method for non-motorized vehicle target detection based on multi-scale feature integration and residual enhancement. Background Technology
[0002] Non-motorized vehicle target detection is an important research direction in computer vision and autonomous driving. With the continuous development of urban transportation, non-motorized vehicles (such as bicycles, electric bikes, and tricycles) are playing an increasingly important role in urban traffic. In autonomous vehicles, accurately identifying surrounding non-motorized vehicles is crucial for ensuring safe driving. The dynamic characteristics and diversity of non-motorized vehicles pose challenges to detection algorithms, thus requiring continuous optimization and improvement of detection techniques. Non-motorized vehicles often intertwine with motor vehicles, pedestrians, and other traffic participants in urban traffic, leading to frequent traffic accidents. Effective non-motorized vehicle detection methods can help improve traffic safety and reduce the accident rate.
[0003] In recent years, artificial intelligence has developed rapidly, computer vision has become increasingly popular, and object detection applications based on convolutional neural networks have become more and more widespread. RT-DETR is an innovative real-time object detection model that cleverly combines the Transformer architecture with the DETR (Detection Transformer) method, achieving a balance between performance and efficiency in object detection. This model significantly reduces computational costs by using smaller feature maps and simplifies model parameters by reducing the number of attention heads, thus achieving efficient real-time object detection while maintaining high detection accuracy.
[0004] However, existing detection methods are not ideal for detecting non-motorized vehicle targets. In urban traffic environments, non-motorized vehicles are often obscured by other road users (such as motor vehicles and pedestrians), making it difficult for detection algorithms to accurately identify and locate the targets. Furthermore, there are many types of non-motorized vehicles, including bicycles, electric bikes, and motorcycles, each with different appearances, sizes, and movement patterns. This diversity makes it difficult for target detection algorithms to handle them uniformly, so improving the model's feature extraction capabilities is essential. Simultaneously, non-motorized vehicles may occupy a small area in an image, especially at long distances; detecting small targets is often more challenging than detecting large targets, frequently resulting in missed or false detections. To address these issues, this paper proposes a non-motorized vehicle target detection method based on multi-scale feature integration and residual enhancement, which improves the accuracy of non-motorized vehicle target detection, reduces the probability of missed and false detections, and effectively enhances the detection precision of non-motorized vehicle targets. Summary of the Invention
[0005] Purpose of the Invention: The purpose of this invention is to provide a non-motorized vehicle target detection method based on multi-scale feature integration and residual enhancement. Addressing the low accuracy of non-motorized vehicle target detection caused by false positives and false negatives, this invention effectively improves the detection accuracy of occluded non-motorized vehicle targets, the detection accuracy of small non-motorized vehicle targets at long distances, and enhances the ability to extract target image features.
[0006] Technical Solution: This invention provides a non-motorized vehicle target detection method based on multi-scale feature integration and residual enhancement, including the following steps:
[0007] S1: Collect real-time video of traffic intersections and preprocess it frame by frame to construct a training dataset;
[0008] S2: Divide the dataset obtained in S1 into a training set, a validation set, and a test set. Then, perform data augmentation on the training set, validation set, and test set respectively. The training set is used to train the improved model; the validation set is used to verify the improved model's ability to detect non-motorized vehicle targets; and the test set is used to test the detection effect of the improved model.
[0009] S3: Using RT-DETR as the base model, the MST-MEIE module is constructed using the designed EEH module. The backbone network of the base model is replaced by Conv convolution and the MST-MEIE module to form a new backbone network. Then, the PKC module is designed, and the DPBF module is constructed using this module to replace the RepC3 module in the base model, resulting in the improved RT-DETR model.
[0010] S4: Use the labeled training set and validation set obtained in S2 to train and validate the improved RT-DETR model. Set the training parameters and evaluate them each time. The improved RT-DETR model identifies different non-motorized vehicle targets from the input image. Train the model using the loss function and obtain the optimal model weights after convergence.
[0011] S5: Using the optimal model weights obtained from training in S4, input the test set, which is mainly used to verify the detection performance of the improved model.
[0012] Furthermore, the specific steps of S1 are as follows:
[0013] S11: Collect real-time video of traffic intersections, perform frame-by-frame preprocessing, construct a training dataset, divide it into training set, validation set and test set, use Mosaic data augmentation to scale, rotate, translate and flip the input images to increase data diversity;
[0014] S12: Scale the input image to a standard size of 640*640 pixels and then feed it into the improved RT-DETR model.
[0015] Furthermore, the specific steps of S3 are as follows:
[0016] S31: Construct the MST-MEIE module using the designed EEH module, and replace the backbone network of the base model with Conv convolution and the MST-MEIE module to form a new backbone network; the first, second, fourth, sixth and eighth layers of the new backbone network are composed of Conv convolution, and the third, fifth, seventh and ninth layers are composed of the designed MST-MEIE module; at the same time, replace the RepC3 module in the baseline model with the DPBF module constructed using the designed PKC module.
[0017] Furthermore, the backbone network described in S31 is mainly responsible for feature extraction, including the Conv module and the designed MST-MEIE module.
[0018] The MST-MEIE module is a new module constructed by combining the designed EEH module with adaptive average pooling and upsampling. The EEH module improves the detail information of the input feature map through smoothing operations and edge enhancement techniques. The MST-MEIE module can effectively extract features from different scales of the input image, highlight the edge information of the image, and integrate these multi-scale features together, demonstrating good representation capabilities in image feature extraction and edge enhancement.
[0019] The PKC module effectively extracts features at different scales by using parallel dilated convolutions with different dilation rates. This enables the network model to capture local and contextual information in the input image, improving its ability to detect small targets.
[0020] The DPBF module utilizes a combination of a designed PKC module, batch normalization, and a fully connected feedforward network layer. Batch normalization and the fully connected feedforward network layer improve the stability and generalization ability of the model training. Simultaneously, by connecting these layers through residual connections, the input features and processed features are added together, preserving the original information while introducing new multi-scale information, thereby enhancing the model's expressive power.
[0021] Furthermore, the specific training parameters described in S4 are: input image size imgsz = 640, initial learning rate lr = 0.01, number of training iterations epoch = 400, batch size of batch training dataset = 4, and official pre-trained weights are used for transfer learning and fine-tuning.
[0022] Furthermore, the evaluation metrics mentioned in S4 mainly include: mean precision (mAP), precision (P), and recall (R). Here, mAP represents the weighted average of the average precision (AP) across all categories; P represents the proportion of correctly predicted positive samples to the actual number of positive samples; and R represents the proportion of correctly predicted positive samples to the total number of predicted samples. The specific formulas are as follows: Where APi represents the average precision of the i-th class, K represents the K classes, TP represents true positives, i.e. positive samples predicted by the model as positive, FP represents false positives, i.e. negative samples predicted by the model as positive, and FN represents false negatives, i.e. positive samples predicted by the model as false.
[0023] Compared with existing technologies, the beneficial effects of this invention are as follows:
[0024] (1) To address the problem of low detection accuracy caused by the overlap and occlusion of non-motorized vehicle targets with other traffic targets in the original model, this invention designs the EEH module, which improves the detail information of the input feature map through smoothing operation and edge enhancement technology; at the same time, the MST-MEIE module is constructed, which can effectively extract features from different scales of the input image, highlight the edge information of the image and integrate these multi-scale features together. This module has good representation ability in image feature extraction and edge enhancement, and effectively solves the problem of low detection accuracy caused by target overlap and occlusion.
[0025] (2) Due to the wide variety of non-motorized vehicles, including bicycles, electric vehicles, tricycles, etc., each type has different appearances, sizes, and modes of movement. This diversity makes it difficult for target detection algorithms to handle them uniformly. In view of the fact that the original model does not extract image features sufficiently during the detection process, this invention constructs a DPBF module. This module adds the input features and the processed features by using residual connections, which preserves the original information and introduces new multi-scale information, thereby improving the expressive power of the model. At the same time, by combining batch normalization and fully connected feedforward network layers, the stability and generalization ability of the model training are improved, thereby improving the feature extraction ability of the model.
[0026] (3) In view of the case of small targets in the original model during long-distance detection, the present invention constructs a PKC module. This module can effectively extract features of different scales by using parallel dilated convolutions with different dilation rates. This enables the network model to capture local and contextual information in the input image, improve its ability to detect small targets, and reduce the problem of missed detection of small targets. Attached Figure Description
[0027] Figure 1 This is a flowchart of the present invention;
[0028] Figure 2 The network structure diagram of MPD-DETR;
[0029] Figure 3 This is a structural diagram of the EEH module;
[0030] Figure 4 This is a structural diagram of the MST-MEIE module;
[0031] Figure 5 This is a structural diagram of the PKC module;
[0032] Figure 6 This is a schematic diagram of the DPBD module.
[0033] Figure 7 This is a comparison chart of detection results from some algorithms. Detailed Implementation
[0034] To better understand the present invention, a non-motorized vehicle target detection method based on multi-scale feature integration and residual enhancement will be described in more detail below with reference to the accompanying drawings in the examples of the present invention.
[0035] By Figure 1 As can be seen, the specific steps of this invention are as follows:
[0036] Step 1: Obtaining the dataset and preprocessing it.
[0037] In this invention, real-time video of traffic intersections is collected and frame-by-frame to form a dataset, which includes 4,300 training images, 500 validation images, and 100 test images. The dataset includes three categories: electric vehicles, tricycles, and bicycles. Data augmentation is performed on the dataset by randomly scaling, rotating, translating, and flipping the input images to enhance the diversity of the data.
[0038] Step 2: Construct a network model based on the improved RT-DETR non-motorized vehicle target detection method, such as... Figure 2 As shown.
[0039] (1) Improved Backbone Network: The MST-MEIE module was constructed using the designed EEH module. The backbone network of the base model was replaced with Conv convolution and the MST-MEIE module to form a new backbone network. The designed MST-MEIE module consists of the EEH module, Conv convolution module, RepConv convolution module, and Upsample module. Among them, the EEH module improves the detail information of the input feature map through smoothing operations and edge enhancement techniques; the MST-MEIE module can effectively extract features from different scales of the input image, highlight the edge information of the image, and integrate these multi-scale features together. This module has good representation ability in image feature extraction and edge enhancement.
[0040] The EEH module structure diagram is as follows: Figure 3 As shown, this module improves the detail information of the input feature map through smoothing operations and edge enhancement techniques. First, the module accepts input data, and the input image is passed through an average pooling layer (AvgPool). The average pooling layer is used to reduce the spatial dimension of the data while retaining important feature information. Then, the original input feature map is subtracted from the smoothed feature map to obtain the enhanced high-frequency information. Next, the enhanced edge information is further processed by convolution operations. Finally, the convolutional feature map of the original input feature map is added to the processed edge information to form the enhanced output.
[0041] The structure diagram of the MST-MEIE module is as follows: Figure 4 As shown, this module consists of an EEH module, a Conv convolution module, a RepConv convolution module, and an Upsample module. First, the input image is divided into three paths. The left and right images pass through an Adaptive AvgPool layer, which adjusts the spatial dimension of the input feature map to a fixed size. The output of each Adaptive AvgPool layer passes through two consecutive convolutional layers, primarily used to extract image features. Next, the output of each convolutional layer passes through an upsampled layer to increase the spatial dimension of the feature map. The output of each upsampled layer passes through a specially designed EEH module. The middle image first undergoes a 1×1 convolution, then a Split operation splits the data. One part of the data undergoes a RepConv convolution and two ordinary convolutions for feature extraction, while the other part is directly connected to a concatenation layer. The feature maps from the three paths are concatenated to integrate the feature information from multiple paths. The concatenated feature map is then further processed by a 1×1 convolutional layer to output the final feature map.
[0042] (2) Improve the neck network: Construct the PKC module, which can effectively extract features at different scales by using parallel dilated convolutions with different dilation rates. This enables the network model to capture local and contextual information in the input image and improve its ability to detect small targets. Then, construct the DPBF module and replace the RepC3 module of the neck network in the original RT-DETR network with this module. This module adds the input features and the processed features by using residual connections, which preserves the original information and introduces new multi-scale information, thereby improving the expressive power of the model. At the same time, the stability of model training and the generalization ability are improved by combining batch normalization and fully connected feedforward network layers.
[0043] The PKC module structure diagram is as follows: Figure 5 As shown, the input image is divided into three paths, each of which passes through a dilated convolutional layer with a different dilation rate. Dilated convolution increases the receptive field of the convolutional kernel by inserting holes between the elements of the kernel without participating in the parameter count, which helps to capture a wider range of contextual information. Then, the outputs of the three dilated convolutional layers are concatenated together in the channel dimension to integrate the feature information extracted by different convolutional layers. Finally, the concatenated feature map is output through a standard convolutional layer.
[0044] The DPBF module structure diagram is as follows: Figure 6 As shown, the input image first passes through a 3×3 deep convolutional layer. Deep convolution reduces the number of model parameters and computational cost. The output of the deep convolution is then residually concatenated with the original input image, which helps alleviate the vanishing gradient problem in deep networks and promotes information flow. Next, the PKC module constructed above re-concatenates the output feature map with the output of the previous residual concatenation. The output feature map then passes through a batch normalization (BN) layer to accelerate the training process and improve model stability. It then passes through a feedforward network (FFN) layer for further image feature processing. Finally, this output is residually concatenated with the previous output to form the final output. This module combines residual convolution with dilated convolution and a feedforward network, effectively improving the model's feature extraction and expressive capabilities while ensuring computational efficiency.
[0045] Step 3: Set the training parameters, train the model, evaluate and compare the training results. Set the input image size imgsz = 640, the initial learning rate lr = 0.01, the number of training iterations epoch = 400, the batch size of the batch training dataset = 4, and use the official pre-trained weights for transfer learning and fine-tuning.
[0046] The evaluation metrics are mainly: mean precision (mAP), precision (P), and recall (R). mAP represents the weighted average of the average precision (AP) across all categories. P represents the proportion of correctly predicted positive samples to the actual number of positive samples. R represents the proportion of correctly predicted positive samples to the total number of predicted samples. The specific formulas are shown in formulas (1) to (3).
[0047] (1)
[0048] (2)
[0049] (3)
[0050] Where APi represents the average precision of the i-th class, K represents the K classes, TP represents true positives, i.e. positive samples predicted by the model as positive, FP represents false positives, i.e. negative samples predicted by the model as positive, and FN represents false negatives, i.e. positive samples predicted by the model as false.
[0051] The model training evaluation results are shown in Table 1.
[0052] Table 1
[0053]
[0054] As shown in Table 1, the MPD-DETR model significantly improves upon the YOLO series algorithms and the original RT-DETR model in terms of precision (P), recall (R), and mean average precision (mAP). The higher recall indicates that MPD-DETR can detect more non-motorized vehicle targets and reduce missed detections, especially in scenarios with overlapping or dense targets. At the same time, the improvement in mAP_0.5 and mAP_0.5:0.95 also effectively improves the detection performance of small non-motorized vehicle targets and reduces missed detections.
[0055] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. After reading this invention, those skilled in the art can make various modifications or alterations to the invention, and these equivalent transformations and modifications also fall within the scope defined by the claims. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A non-motorized vehicle target detection method based on multi-scale feature integration and residual enhancement, characterized in that, Includes the following steps: S1: Collect real-time video of traffic intersections and preprocess it frame by frame to construct a training dataset; S2: Divide the dataset obtained in S1 into training set, validation set, and test set, and then perform data augmentation on the training set, validation set, and test set respectively; S3: Using RT-DETR as the base model, the MST-MEIE module is constructed using the designed EEH module. The backbone network of the base model is replaced by Conv convolution and the MST-MEIE module to form a new backbone network. Then, the PKC module is designed, and the DPBF module is constructed using this module to replace the RepC3 module in the base model, resulting in the improved RT-DETR model. S4: Use the labeled training and validation sets obtained in S2 to train and validate the improved RT-DETR model. Set the training parameters and evaluate them each time. The improved RT-DETR model identifies different non-motorized vehicle targets from the input image. Train the model using the loss function and obtain the optimal model weights after convergence.
2. The non-motorized vehicle target detection method based on multi-scale feature integration and residual enhancement according to claim 1, characterized in that, The specific process of S1 is as follows: S11: Collect real-time video of traffic intersections, perform frame-by-frame preprocessing, construct a training dataset, divide it into training set, validation set and test set, use Mosaic data augmentation to scale, rotate, translate and flip the input images to increase data diversity; S12: Scale the input image to a standard size of 640*640 pixels and then feed it into the improved RT-DETR model.
3. The non-motorized vehicle target detection method based on multi-scale feature integration and residual enhancement according to claim 2, characterized in that, The specific process of S3 is as follows: S31: Construct the MST-MEIE module using the designed EEH module, and replace the backbone network of the base model with Conv convolution and the MST-MEIE module to form a new backbone network; the first, second, fourth, sixth and eighth layers of the new backbone network are composed of Conv convolution, and the third, fifth, seventh and ninth layers are composed of the designed MST-MEIE module; at the same time, replace the RepC3 module in the base model with the DPBF module constructed using the designed PKC module.
4. The non-motorized vehicle target detection method based on multi-scale feature integration and residual enhancement according to claim 3, characterized in that, The backbone network described in step S31 is responsible for feature extraction, including the Conv module and the designed MST-MEIE module. The MST-MEIE module is a new module constructed by combining the designed EEH module with adaptive average pooling and upsampling. The EEH module improves the detail information of the input feature map through smoothing operations and edge enhancement techniques. The MST-MEIE module can effectively extract features from different scales of the input image, highlight the edge information of the image, and integrate these multi-scale features together, demonstrating excellent representation capabilities in image feature extraction and edge enhancement.
5. The non-motorized vehicle target detection method based on multi-scale feature integration and residual enhancement according to claim 3, characterized in that, In the PKC module, the input image is divided into three paths, each passing through a dilated convolutional layer with a different dilation rate. The outputs of the three dilated convolutional layers are then concatenated along the channel dimension, integrating the feature information extracted by different convolutional layers. Finally, the concatenated feature map is output through a standard convolutional layer. In the DPBF module, the input image first passes through a 3×3 depthwise convolutional layer. The output of the depthwise convolution is residually concatenated with the original input image. Then, through the PKC module, the output feature map is residually concatenated again with the output of the previous residual concatenation. The output feature map passes through a batch normalization layer, then a feedforward network layer for further image feature processing. Finally, this output is residually concatenated with the previous output to form the final output.
6. The non-motorized vehicle target detection method based on multi-scale feature integration and residual enhancement according to claim 3, characterized in that, The specific training parameters in S4 are as follows: input image size imgsz = 640, initial learning rate lr = 0.01, number of training iterations epoch = 400, batch size of batch training dataset = 4, and official pre-trained weights are used for transfer learning and fine-tuning.
7. The non-motorized vehicle target detection method based on multi-scale feature integration and residual enhancement according to claim 3, characterized in that, The evaluation metrics in S4 are mainly: mean precision (mAP), precision (P), and recall (R). mAP represents the weighted average of the average precision (AP) across all categories; P represents the proportion of correctly predicted positive samples to the actual number of positive samples; and R represents the proportion of correctly predicted positive samples to the total number of predicted samples. The specific formulas are as follows: Where APi represents the average precision of the i-th class, K represents the K classes, TP represents true positives, i.e. positive samples predicted by the model as positive, FP represents false positives, i.e. negative samples predicted by the model as positive, and FN represents false negatives, i.e. positive samples predicted by the model as false.