Image Domain Vehicle Target Detection Method Based on Deep Neural Networks

By improving the YOLOv11s network structure, embedding the SimAM module to enhance small target features, and designing the dual-branch detection head FineGrainedHead and the hollow multi-scale enhancement module DREM, the problems of missed detection of small targets, confusion of similar vehicle models, and identification of engineering vehicles under the perspective of UAVs were solved, achieving efficient vehicle detection and recognition results.

CN122657776BActive Publication Date: 2026-11-17AIR FORCE UNIV PLA
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202611161275.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-08-03
Publication Date
2026-11-17
Estimated Expiration
2046-08-03

AI Technical Summary

Technical Problem

From the perspective of drones, small-sized vehicles have weak features and complex backgrounds, resulting in a high rate of missed detections for small targets. There is a lack of effective fine-grained differentiation between similar-looking models such as sedans and SUVs, leading to low classification accuracy and easy confusion. Engineering vehicles have complex structures, and the features of key components are easily lost during network fusion, resulting in serious confusion with conventional vehicle categories. Existing improvement strategies are mostly aimed at optimizing single problems and lack systematic solutions for scenarios with multiple challenges.

Method used

An improved YOLOv11s network structure was constructed, and a SimAM parameterless attention module was embedded to enhance the response to small target features. A dual-branch fine-grained head, FineGrainedHead, was designed to mine subtle feature differences. A dilated multi-scale enhancement module, DREM, was introduced between the backbone network output and the neck feature fusion path. End-to-end training was performed through a multi-task joint loss function to output the bounding box coordinates, class confidence, and class label of the vehicle target.

Benefits of technology

It significantly reduced the false negative rate of small target vehicles, improved the accuracy of fine-grained vehicle classification, optimized the recognition accuracy of engineering vehicles, and maintained the real-time performance and lightweight performance of detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122657776B_ABST
    Figure CN122657776B_ABST
Patent Text Reader

Abstract

This application relates to a method, apparatus, medium, device, and program for vehicle target detection in the image domain based on deep neural networks. The method includes constructing an improved YOLOv11s network structure, embedding a SimAM parameterless attention module into the C3K2 module of the backbone network to form an improved C3K2_SimAM module to enhance the feature response of small targets; constructing the native detection head into a dual-branch fine-grained detection head, FineGrainedHead, and mining subtle differences between similar vehicle models through local spatial alignment and dynamic adaptive fusion; adding a dedicated feature extraction branch for engineering vehicles and introducing a dilated multi-scale enhancement module (DREM) to strengthen engineering vehicle features. The vehicle image dataset from a UAV perspective is preprocessed, and end-to-end training is performed using a multi-task joint loss function. The completed model outputs detection results. This application effectively solves the problems of missed detection of small targets, confusion between similar vehicle models, and low accuracy in recognizing engineering vehicles.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and computer vision technology, specifically to a method, apparatus, medium, device, and program for image domain vehicle target detection based on deep neural networks. Background Technology

[0002] With the rapid development of Unmanned Aerial Vehicle (UAV) technology and optical imaging systems, UAV-mounted ground observation technology has been widely applied in key areas such as intelligent traffic monitoring, emergency rescue, urban security, and disaster assessment. Achieving high-precision, real-time detection of ground vehicle targets, especially small and engineering vehicles, has become a core prerequisite for intelligent decision-making in these application scenarios. However, under UAV high-altitude imaging conditions, vehicle targets often exhibit characteristics such as small size, weak features, and complex backgrounds, making them highly susceptible to missed detections and false detections. Furthermore, the lack of effective fine-grained differentiation between similar-looking vehicle categories such as sedans and sport utility vehicles (SUVs) leads to low model classification accuracy. In addition, engineering vehicles, due to their complex external structures and the difficulty of feature extraction, further exacerbate the category confusion problem in recognition tasks. Real-world factors such as changes in lighting, inclement weather, and target occlusion also further degrade image quality, making the aforementioned detection challenges even more prominent.

[0003] In recent years, deep learning-based target detection algorithms have made significant progress. Among them, the YOLO (You Only Look Once) series of algorithms has become the mainstream method for vehicle detection tasks due to its balanced performance of speed and accuracy. YOLOv11 further improves the performance of general target detection through structural optimization and enhanced feature extraction capabilities. However, existing research mainly focuses on the detection optimization of regular-sized targets, and its feature extraction and classification capabilities for small-sized vehicles, similar vehicle models, and engineering vehicles from the perspective of UAVs are still insufficient.

[0004] To further improve the overall performance of vehicle detection in complex scenarios, scholars both domestically and internationally have proposed various improvement strategies, such as attention mechanisms, feature pyramid structures, and fine-grained classification networks. However, most of these strategies focus on optimizing single problems and lack systematic solutions for scenarios with multiple challenges. In practical applications of UAVs equipped with optical imaging, small target detection, fine-grained classification, and engineering vehicle recognition often face challenges simultaneously, making it difficult for traditional methods to achieve satisfactory overall detection results. Summary of the Invention

[0005] The main objective of this application is to provide a method, apparatus, medium, device, and program for image-domain vehicle target detection based on deep neural networks. This aims to address the following technical problems encountered in ground vehicle target detection from the perspective of unmanned aerial vehicles (UAVs): under high-altitude imaging conditions, small-sized vehicles have weak features and complex backgrounds, leading to a high rate of missed detections for small targets; there is a lack of effective fine-grained differentiation between similar-looking models such as sedans and SUVs, resulting in low classification accuracy and easy confusion; engineering vehicles have complex structures, and key component features are easily lost during network fusion, leading to serious confusion with conventional vehicle categories; existing improvement strategies mostly optimize single problems and lack systematic solutions for scenarios with multiple challenges, such as small target detection, fine-grained classification, and engineering vehicle recognition.

[0006] To achieve the above objectives, the first aspect of this application provides an image domain vehicle target detection method based on a deep neural network, applied to a ground vehicle detection scenario using an optical imaging system mounted on an unmanned aerial vehicle (UAV), comprising: An improved YOLOv11s network structure is constructed, comprising: embedding a SimAM parameterless attention module into the C3K2 module of the backbone network to form a C3K2_SimAM improved module, used to enhance the response to small target features; constructing the native YOLOv11s network's detector head as a dual-branch fine-grained head, FineGrainedHead, which includes a regular branch and a fine-grained branch connected in parallel. The regular branch adopts a CBS convolutional structure and is used for bounding box regression and target confidence prediction. The fine-grained branch mines subtle feature differences between similar vehicle models through a local spatial alignment module and a dynamic adaptive fusion module. The outputs of the regular branch and the fine-grained branch are concatenated along the channel dimension, and the dynamic adaptive fusion module predicts the dynamic... Weighted fusion is performed using fusion weights to obtain fused features for fine-grained classification. A dedicated feature extraction branch for engineering vehicles is added, and a dilated multi-scale enhancement module (DREM) is introduced between the backbone network output and the neck feature fusion path. The DREM extracts features from the three layers P3, P4, and P5 of the backbone network output in parallel using dilated convolutions with different dilation rates. Specifically, the low-layer P3 features use dilated convolutions with dilation rates r=1 and 2 to enhance texture details, the middle-layer P4 features use dilated convolutions with multiple dilation rates r=2, 4, and 8 to expand the receptive field and capture large-scale component features, and the high-layer P5 features use standard convolutions with dilation rate r=1 to enhance the overall semantic contour. The features extracted with different dilation rates are concatenated and adaptively fused through convolutional layers to obtain enhanced features, which are then fed into the neck feature fusion path and further enhanced through a dedicated feature extraction branch. The vehicle image dataset from the perspective of the UAV is preprocessed, and the improved YOLOv11s network structure is trained end-to-end using a multi-task joint loss function, which is obtained by weighted summation of bounding box regression loss, classification loss and fine-grained feature measurement loss. The optical image to be detected is input into the trained improved YOLOv11s network structure. After feature extraction, feature fusion and detection head processing, the bounding box coordinates, class confidence and class label of the vehicle target are output.

[0007] Optionally, the SimAM parameterless attention module is embedded into the C3K2 module to obtain the C3K2_SimAM improved module. The importance weight of each neuron in the C3K2_SimAM improved module is calculated using an energy function, which is:

[0008] in, This represents the average number of neurons within the current channel. The variance of neurons within the current channel. For learnable bias terms, The neurons within the current channel; The attention weights are obtained based on the energy function: ; The final enhanced features are: ,in, X The image is used as the input backbone network.

[0009] Optionally, the regular branch of the dual-branch fine-grained detection head FineGrainedHead adopts a CBS convolutional structure for bounding box regression and target confidence prediction tasks; the fine-grained branch of the dual-branch fine-grained detection head FineGrainedHead includes a Local Spatial Alignment (LSA) module and a Dynamic Adaptive Fusion (DAF) module; wherein, the LSA module is used to focus on local regions to mine fine-grained local features between different vehicle models; the DAF module is used to concatenate global features and fine-grained local features, predict dynamic fusion weights, and perform weighted fusion.

[0010] Optionally, in the Local Spatial Alignment (LSA) module, the scale parameter is predicted by the network. s Translation parameters t Then, an affine transformation is performed on the input features, the expression of which is:

[0011] In the formula, These are the aligned features; Affine is the spatial alignment transformation operation. These are the transformation parameters, which include the scale parameter. s Translation parameters t; The network structure includes an average pooling layer, a first fully connected layer, a SiLU activation layer, a second fully connected layer, and a Sigmoid activation layer connected in sequence. The input of the average pooling layer and the output of the Sigmoid activation layer are multiplied and then input into a first depthwise separable convolutional layer to output the final affine transformation result.

[0012] Optionally, in the dynamic adaptive fusion module DAF, global features and fine-grained local features are respectively input into an average pooling layer and then concatenated. The concatenation result is then input into a convolutional layer and a sigmoid activation layer in sequence. The output of the sigmoid activation layer and the inputs of the two average pooling layers are multiplied and then input into a second depthwise separable convolutional layer, which outputs the weighted fusion weights.

[0013] Optionally, in the dilated multi-scale enhancement module DREM, for the input P3, P4, and P5 three-layer features, dilated convolutions with different dilation rates are used for parallel feature extraction. Specifically, the lower-layer P3 feature uses a dilation rate of... r Dilated convolutions with dilation rates of r=1 and r=2 enhance texture details; the middle layer P4 features use dilated convolutions with dilation rates of r=2, 4, and 8 to expand the receptive field and capture large-scale part features; the high layer P5 features use standard convolutions with dilation rate of r=1 to enhance the overall semantic contour. Features extracted with different porosity are concatenated and then adaptively fused through convolutional layers to obtain enhanced features, which are then incorporated into the neck of the improved YOLOv11s network structure.

[0014] To achieve the above objectives, a second aspect of this application provides an image-domain vehicle target detection device based on a deep neural network, applied to a ground vehicle detection scenario using an optical imaging system mounted on an unmanned aerial vehicle (UAV), comprising: A network construction module is used to construct an improved YOLOv11s network structure. This improved YOLOv11s network structure includes: embedding a SimAM parameterless attention module into the C3K2 module of the backbone network to form a C3K2_SimAM improved module, used to enhance the response to small target features; constructing the native YOLOv11s network's detector head as a dual-branch fine-grained head, FineGrainedHead. This FineGrainedHead includes parallel regular branches and fine-grained branches. The regular branches employ a CBS convolutional structure and are used for bounding box regression and target confidence prediction. The fine-grained branches mine subtle feature differences between similar vehicle models through a local spatial alignment module and a dynamic adaptive fusion module. The outputs of the regular branches and the fine-grained branches are concatenated along the channel dimension and then processed by the dynamic adaptive fusion module. The predicted dynamic fusion weights are weighted and fused to obtain fused features for fine-grained classification. A dedicated feature extraction branch for engineering vehicles is added, and a dilated multi-scale enhancement module (DREM) is introduced between the backbone network output and the neck feature fusion path. The DREM extracts features from the three layers P3, P4, and P5 of the backbone network output in parallel using dilated convolutions with different dilation rates. Specifically, the low-layer P3 features use dilated convolutions with dilation rates r=1 and 2 to enhance texture details, the middle-layer P4 features use dilated convolutions with multiple dilation rates r=2, 4, and 8 to expand the receptive field and capture large-scale component features, and the high-layer P5 features use standard convolutions with dilation rate r=1 to enhance the overall semantic contour. The features extracted with different dilation rates are concatenated and adaptively fused through convolutional layers to obtain enhanced features, which are then fed into the neck feature fusion path and enhanced through a dedicated feature extraction branch. The training module is used to preprocess the vehicle image dataset from the perspective of the UAV, and to perform end-to-end training on the improved YOLOv11s network structure using transfer learning and a multi-task joint loss function. The multi-task joint loss function is obtained by weighted summation of bounding box regression loss, classification loss and fine-grained feature measurement loss. The inference module is used to input the optical image to be detected into the trained improved YOLOv11s network structure. After feature extraction, feature fusion and detection head processing, it outputs the bounding box coordinates, class confidence and class label of the vehicle target.

[0015] To achieve the above objectives, a third aspect of this application provides an electronic device including a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement the image domain vehicle target detection method based on a deep neural network as provided in the preceding embodiments.

[0016] To achieve the above objectives, a fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image domain vehicle target detection method based on a deep neural network as provided in the preceding embodiments.

[0017] To achieve the above objectives, the fifth aspect of this application provides a computer program product including computer-readable instructions that, when executed by a processor, implement the image domain vehicle target detection method based on a deep neural network as provided in the preceding embodiments.

[0018] This application discloses a method, apparatus, medium, device, and program for image domain vehicle target detection based on deep neural networks. The method includes constructing an improved YOLOv11s network structure. The improved YOLOv11s network structure includes: embedding a SimAM parameterless attention module into the C3K2 module of the backbone network to form a C3K2_SimAM improved module, which is used to enhance the feature response of small targets; and constructing the detection head of the original YOLOv11s network into a dual-branch fine-grained detection head FineGrainedHead. The dual-branch fine-grained detection head FineGrainedHead includes fine-grained branches, which are aligned with the local space model. The block and dynamic adaptive fusion module mines subtle feature differences among similar vehicle models; a dedicated feature extraction branch for engineering vehicles is added, and a dilated multi-scale enhancement module DREM is introduced between the backbone network output and the neck feature fusion path to enhance engineering vehicle features through the dedicated feature extraction branch; the vehicle image dataset from the UAV perspective is preprocessed, and the improved YOLOv11s network structure is trained end-to-end using transfer learning and a multi-task joint loss function; the optical image to be detected is input into the trained improved YOLOv11s network structure, and after feature extraction, feature fusion, and detection head processing, the bounding box coordinates, class confidence, and class label of the vehicle target are output. Compared with the prior art, this application has the following beneficial effects: enhanced small target detection capability: by combining the parameterless attention module SimAM with the C3K2 unit of the backbone network to construct the C3K2_SimAM feature enhancement module, this module can adaptively highlight the feature regions of small-sized vehicles from the UAV perspective without additional parameters and computational overhead, suppress background noise interference, effectively solve the problem of weak features and easy missed detection of small targets, and significantly reduce the missed detection rate of small target vehicles. Improved Fine-Grained Classification Accuracy: To address the issue of confusion between similar-looking vehicles such as sedans and SUVs, a dual-branch fine-grained head, FineGrainedHead, was designed. The conventional branch focuses on bounding box regression and basic feature extraction, while the fine-grained branch, through Local Spatial Alignment (LSA) and Dynamic Adaptive Fusion (DAF) modules, uncovers subtle appearance differences between vehicle models, effectively improving the accuracy of fine-grained vehicle classification and alleviating category confusion. Optimized Engineering Vehicle Recognition: To address the challenges of complex structures and the easy loss of key component features in engineering vehicles, a Hollow Multi-Scale Enhancement (DREM) module was added to the neck feature fusion stage. This path performs hierarchical enhancement of the P3 / P4 / P5 features output from the backbone, specifically enhancing the large-scale structural and detailed features of engineering vehicles. This solves the problems of difficult feature extraction and easy confusion with conventional vehicles, significantly improving the recognition accuracy of engineering vehicles.Speed ​​and accuracy are balanced and optimized: The overall improvement plan is based on the principle of lightweight design, and only introduces lightweight enhancement modules in the backbone, neck and detection head. The inference speed is basically maintained at the original real-time level of YOLOv11s. While improving detection performance, it also takes into account the needs of edge deployment and real-time detection. Attached Figure Description

[0019] Figure 1 A schematic diagram of the overall structure of the improved YOLOv11s network; Figure 2 This is a schematic diagram of the C3K2_SimAM module structure; Figure 3 This is a schematic diagram of the FineGrainedHead structure. Figure 4 A schematic diagram of the LSA module structure for local spatial alignment; Figure 5 This is a schematic diagram of the dynamic adaptive fusion DAF module structure; Figure 6 Schematic diagram of a cavity multi-scale enhancement module DREM; Figure 7 Example image of a dataset sample; Figure 8 This is a schematic diagram of the loss curves for box_loss, cls_loss, and dlf_loss for the training set. Figure 9 A schematic diagram of the loss curves for the validation set: box_loss, cls_loss, and dlf_loss; Figure 10 This is a schematic diagram of the Precision curve; Figure 11 This is a schematic diagram of the recall curve; Figure 12 This is a schematic diagram of the mAP@50 curve; Figure 13 This is a schematic diagram of the mAP@50-95 curve; Figure 14 This is a schematic diagram of the Precision-Confidence Curve; Figure 15 This is a schematic diagram of the Recall-Confidence Curve; Figure 16 This is a schematic diagram of the Precision-Recall curve; Figure 17 This is a schematic diagram of the F1-confidence curve; Figure 18 This is a schematic diagram of the normalized confusion matrix for different vehicle types classified by Faster R-CNN. Figure 19This is a schematic diagram of the normalized confusion matrix of the original YOLOv11s model for different vehicle classifications; Figure 20 A schematic diagram of the normalized confusion matrix for different vehicle classifications using the improved YOLOv11s model; Figure 21 This is a visual representation of the detection results from Faster R-CNN. Figure 22 This is a schematic diagram of the original YOLOv11s detection results; Figure 23 A schematic diagram of the improved YOLOv11s visualization results; Figure 24 This is a flowchart illustrating an embodiment of the method of this application.

[0020] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0022] In practical applications of UAVs equipped with optical imaging, small target detection, fine-grained classification, and engineering vehicle recognition often face simultaneous challenges. Traditional methods such as two-stage detectors like Faster R-CNN, while offering high accuracy, suffer from high computational complexity, making them unsuitable for real-time requirements. Single-stage detectors like YOLOv11s, while fast, still fall short in feature extraction and classification capabilities when dealing with small vehicles, similar vehicle models, and engineering vehicles from the UAV's perspective. Existing improvement strategies primarily optimize for single problems, lacking systematic solutions for scenarios with multiple challenges, resulting in unsatisfactory overall detection performance.

[0023] refer to Figure 1 and Figure 24 The first embodiment of this application provides an image domain vehicle target detection method based on deep neural networks to solve the technical problems mentioned in the background art, such as high false negative rate of small targets, serious confusion in the classification of similar vehicle models, and low recognition accuracy of engineering vehicles. This method can be executed by a processor, which can be set in a terminal or a server. The execution process of the image domain vehicle target detection method based on deep neural networks can be as follows: Step S101: Construct the improved YOLOv11s network structure.

[0024] The improved YOLOv11s network structure includes: embedding a SimAM parameterless attention module into the C3K2 module of the backbone network to form an improved C3K2_SimAM module, used to enhance the response to small target features; constructing the native YOLOv11s network's detection head as a dual-branch fine-grained detection head, FineGrainedHead, which includes parallel regular branches and fine-grained branches. The fine-grained branches mine subtle feature differences between similar vehicle models through a local spatial alignment module and a dynamic adaptive fusion module; and adding a feature extraction component specifically for engineering vehicles. A dilated multi-scale enhancement module (DREM) is introduced between the backbone network output and the neck feature fusion path. The DREM performs parallel feature extraction on the P3, P4, and P5 layers using dilated convolutions with different dilation rates. Specifically, the low-level P3 features use dilated convolutions with dilation rates r=1 and 2, the middle-level P4 features use dilated convolutions with multiple dilation rates r=2, 4, and 8, and the high-level P5 features use standard convolutions with dilation rate r=1. The features extracted with different dilation rates are concatenated, adaptively fused through convolutional layers, and then fed into the neck feature fusion path. A dedicated feature extraction branch is used to enhance the engineering vehicle features.

[0025] Specifically, the processor first normalizes the image size to a standard input size of 640×640, and simultaneously normalizes pixel values, mapping them to between 0 and 1 to suit the model's input requirements. The preprocessed image is input into the improved backbone network, which then sequentially passes through a Focus layer, a C2F layer, and multiple C3K2_SimAM modules for feature extraction. After basic feature extraction, each C3K2_SimAM module enhances the feature map using the SimAM attention module, automatically increasing the feature weights of small target regions and outputting multi-scale feature maps. These scaled feature maps correspond to features downsampled by 8x, 16x, and 32x, respectively. The backbone network feeds the output three-layer multi-scale features into the neck PAN-FPN structure, employing a top-down and bottom-up bidirectional feature fusion approach to fully integrate shallow texture details and deep semantic abstractions, thereby enhancing the network's ability to represent vehicle targets at different scales. Then, in the connection stage between the backbone output and neck feature fusion, this application adds a dedicated feature enhancement branch for engineering vehicles, connecting the P3, P4, and P5 layers of features to the Dermatologically Modulated Emulation (DREM) module. By setting multiple sets of dilated convolutions with different dilation rates to expand the receptive field, the large-scale structural components and shallow detailed textures of engineering vehicles are captured layer by layer, resulting in enhanced multi-scale features. Finally, the processor integrates the enhanced multi-scale features into the neck fusion process, complementing and stitching them with conventional features to achieve dedicated feature enhancement for engineering vehicles such as bulldozers, excavators, and cranes. This can, to some extent, solve the problem of complex structures in engineering vehicles and their susceptibility to confusion with conventional vehicles. The enhanced multi-scale features after neck fusion are input into the dual-branch fine-grained detection head FineGrainedHead, whose conventional branch and fine-grained branch are set in parallel. The conventional branch of FineGrainedHead is responsible for completing target bounding box regression and class confidence prediction. The fine-grained head's fine-grained branch mines subtle feature differences between similar vehicle types such as sedans and SUVs through LSA local spatial alignment and DAF dynamic adaptive fusion modules, and optimizes feature distribution by combining metric learning. Finally, the features output from the two branches are dynamically weighted and adaptively fused to obtain fine-grained classification results. Then, non-maximum suppression (NMS) is used to remove redundant detection boxes, ultimately outputting the accurate location, confidence score, and class label of all vehicle targets in the image, completing the detection and fine-grained recognition of vehicle targets in UAV optical images.

[0026] Step S102: Preprocess the vehicle image dataset from the perspective of the UAV, and use transfer learning and multi-task joint loss function to train the improved YOLOv11s network structure end-to-end.

[0027] In the specific implementation process, refer to Figure 7 This application trains the model on the publicly available dataset from the 2025 China ATR Challenge. The dataset includes civilian vehicles such as sedans and SUVs, as well as engineering vehicles such as bulldozers and excavators, covering various scenarios including complex backgrounds, varying lighting conditions, target occlusion, and densely packed small targets. Preprocessing includes operations such as Mosaic data augmentation, random cropping, and color dithering to improve the model's generalization ability. Then, pre-trained YOLOv11s weights are loaded for fine-tuning.

[0028] The loss function used during training is a multi-task joint loss function. This multi-task joint loss function consists of a weighted sum of bounding box regression loss, classification loss, and fine-grained feature measurement loss. This application employs an end-to-end training approach for the model, which optimizes network parameters and enables the model to simultaneously adapt to small object detection, fine-grained classification, and engineering vehicle recognition tasks.

[0029] refer to Figures 8-13 , Figures 8-13 This is a schematic diagram illustrating the training performance of the multi-task joint loss function. Among them, Figure 8 This is a schematic diagram of the loss curves for box_loss, cls_loss, and dlf_loss on the training set. Figure 9 A schematic diagram of the loss curves for the validation set: box_loss, cls_loss, and dlf_loss; Figure 10 This is a schematic diagram of the Precision curve. Figure 11 This is a schematic diagram of the recall curve. Figure 12 This is a schematic diagram of the mAP@50 curve. Figure 13 This is a schematic diagram of the mAP@50-95 curve.

[0030] Step S103: Input the optical image to be detected into the trained improved YOLOv11s network structure. After feature extraction, feature fusion and detection head processing, the bounding box coordinates, class confidence and class label of the vehicle target are output.

[0031] Finally, after the model training is complete, the processor can input the optical vehicle image of the drone to be detected into the trained and improved YOLOv11s model.

[0032] The inference process of the model is as follows: First, the image is processed by the backbone network to extract multi-scale basic features, in which the embedded C3K2_SimAM module can effectively enhance the feature representation of small target vehicles; then, the features are fed into the neck network, and based on the bidirectional multi-scale feature fusion, the added three-scale dilated multi-scale enhancement module (DREM) is used to enhance the specific features of engineering vehicles; finally, the enhanced multi-scale features are input into the dual-branch fine-grained head, and the output includes the target bounding box coordinates, class confidence and class label, so as to realize the accurate detection and recognition of vehicle targets in optical images.

[0033] refer to Figures 14-23 As shown, where, Figure 14 This is a diagram illustrating the Precision-Confidence Curve. Figure 15 This is a schematic diagram of the Recall-confidence curve. Figure 16 This is a schematic diagram of the Precision-Recall curve. Figure 17 This is a schematic diagram of the F1-confidence curve. Figure 18 This is a schematic diagram of the normalized confusion matrix for different vehicle models classified by Faster R-CNN. Figure 19 This is a schematic diagram of the normalized confusion matrix of the original YOLOv11s model for classifying different vehicle types. Figure 20 To improve the YOLOv11s model's classification of different vehicle models, a normalized confusion matrix diagram is shown. Figure 21 This is a schematic diagram of the visualization detection results from Faster R-CNN. Figure 22 This is a schematic diagram of the original YOLOv11s detection results. Figure 23 This diagram illustrates the improved visualization of YOLOv11s detection results. The diagram shows the identification results and a comparison with other existing technologies. The results demonstrate that the technical solution of this application is superior to existing technologies.

[0034] Specifically, this application has the following advantages over the prior art: Enhanced small target detection capability. This application combines the parameterless attention module SimAM with the C3K2 unit of the backbone network to construct the C3K2_SimAM feature enhancement module. This module requires no additional parameters or computational overhead, and can adaptively highlight the feature regions of small vehicles from the UAV's perspective, suppressing background noise interference. It effectively solves the problem of small targets being easily missed due to weak features, and significantly reduces the false negative rate of small target vehicles.

[0035] Improved Fine-Grained Classification Accuracy. This application addresses the issue of confusion between similar-looking vehicles such as sedans and SUVs by designing a dual-branch fine-grained head. The conventional branch focuses on bounding box regression and basic feature extraction, while the fine-grained branch utilizes Local Spatial Alignment (LSA) and Dynamic Adaptive Fusion (DAF) modules to uncover subtle appearance differences between vehicle models, effectively improving the accuracy of fine-grained vehicle classification and mitigating category confusion.

[0036] Improved recognition performance for engineering vehicles. This application addresses the issue of complex structures and the easy loss of key component features in engineering vehicles by adding a Hollow Multi-Scale Enhancement Module (DREM) during the neck feature fusion stage. This approach performs layered enhancement on the P3, P4, and P5 features output from the backbone, thereby specifically enhancing the large-scale structural and detailed features of engineering vehicles. This solves the problems of difficult feature extraction for engineering vehicles and easy confusion with conventional vehicles, significantly improving the recognition accuracy of engineering vehicles.

[0037] Speed ​​and accuracy are balanced and optimized. The overall improvement plan is based on the principle of lightweight design, introducing lightweight enhancement modules only in the backbone, neck and detection head. The inference speed is basically maintained at the original real-time level of YOLOv11s, thus improving detection performance while taking into account the edge deployment and real-time detection requirements of UAV airborne platforms.

[0038] In one embodiment of this application, the SimAM parameterless attention module is embedded into the C3K2 module to obtain the C3K2_SimAM improved module. The importance weight of each neuron in the C3K2_SimAM improved module is calculated using an energy function, which is:

[0039] in, This represents the average number of neurons within the current channel. The variance of neurons within the current channel. For learnable bias terms These are neurons within the current channel. The attention weights, obtained based on the energy function, are: .

[0040] The final enhanced features are: ,in, X The image is used as the input backbone network.

[0041] Specifically, refer to Figure 2To address the issue of weak feature response for small targets in the original YOLOv11s backbone network, the parameterless SimAM attention module is embedded into the C3K2 module of the backbone network to construct the C3K2_SimAM improved module. Based on neuroscience theory, the C3K2_SimAM improved module automatically infers the importance weight of each neuron in the feature map through an energy function. Without adding additional training parameters, it achieves coordinated attention enhancement across spatial and channel dimensions, effectively improving the feature response intensity of small target regions without increasing the model's parameter count or computational overhead.

[0042] The core energy function and weight calculation process of this module are shown above. This module can effectively improve the feature response intensity of small target regions without increasing the number of model parameters and computational overhead.

[0043] In an optional embodiment of this application, reference is made to Figure 3 The FineGrainedHead dual-branch fine-grained detection head uses a CBS convolutional structure for its regular branches, which are used for bounding box regression and target confidence prediction tasks. The fine-grained branches of the FineGrainedHead dual-branch fine-grained detection head include a Local Spatial Alignment (LSA) module and a Dynamic Adaptive Fusion (DAF) module. The LSA module is used to focus on local regions to mine fine-grained local features between different vehicle models. The DAF module is used to concatenate global features and fine-grained local features, predict dynamic fusion weights, and perform weighted fusion.

[0044] Specifically, addressing the issue of classification confusion among similar-looking vehicles such as sedans and SUVs, this application modifies the native detection head of YOLOv11s, proposing a dual-branch fine-grained head. This detection head employs a parallel, decoupled dual-branch architecture: the first is a regular branch, using the native YOLOv11 (Conv-Batch Norm-SiLU, CBS) convolutional structure, focusing on bounding box regression and target confidence prediction tasks to ensure accurate localization; the second is a fine-grained branch, which first uses a Local Spatial Alignment (LSA) module to focus on key local areas such as headlights and grilles, uncovering subtle appearance differences between different vehicle models, and then uses a Dynamic Adaptive Fusion (DAF) module to adaptively weight and fuse fine-grained local features with global features, ultimately outputting a fine-grained classification result. This design decouples the localization and classification tasks, effectively improving the ability to distinguish between similar vehicle models. The regular branch and the fine-grained branch are set in parallel. The outputs of the two branches are concatenated in the channel dimension and then weighted by the dynamic fusion weights predicted by the dynamic adaptive fusion module to output the fine-grained classification result.

[0045] Further, refer to Figure 4 In the Local Spatial Alignment (LSA) module, the scale parameter is predicted by the network. s Translation parameters t The affine transformation of the input features is expressed as follows:

[0046] in, These are the aligned features; Affine is the spatial alignment transformation operation. These are the transformation parameters, which include the scale parameter. s Translation parameters t; The Local Spatial Alignment (LSA) module includes an average pooling layer, a first fully connected layer, a SiLU activation layer, a second fully connected layer, and a Sigmoid activation layer connected in sequence. The input of the average pooling layer and the output of the Sigmoid activation layer are multiplied and then input into the first depthwise separable convolutional layer, which outputs the final affine transformation result.

[0047] Specifically, the core calculation process of the Local Spatial Alignment (LSA) module is as follows: First, the transformation parameters of key local regions are obtained through network prediction. ,in For scale parameters, The translation parameters are then applied to the input features. The alignment transformation, as shown in the previous expression, aligns key local areas (such as headlights and grilles) of different car models to a unified spatial position, eliminating feature offset caused by posture differences, thereby uncovering subtle appearance differences between different car models.

[0048] For example, refer to Figure 5 In the Dynamic Adaptive Fusion Module (DAF), global features and fine-grained local features are concatenated after passing through average pooling layers. The concatenated result is then input into a convolutional layer and a sigmoid activation layer. The output of the sigmoid activation layer and the inputs of the two average pooling layers are multiplied and then input into a second depthwise separable convolutional layer, which outputs the weighted fusion weights.

[0049] Specifically, the core computation process of the dynamic adaptive fusion DAF module is as follows: global features Aligned local features If the data is concatenated, the predicted dynamic fusion weights are:

[0050] in, It is the Sigmoid activation function. These are the learnable weights and biases, respectively. This is a feature splicing operation.

[0051] The final characteristics after fusion are:

[0052] This module enables adaptive weighted fusion of local fine-grained features and global features, effectively improving the ability to classify and distinguish similar vehicle models.

[0053] Further, refer to Figure 6 In the dilated multi-scale augmentation module DREM, for the input P3, P4, and P5 layers of features, dilated convolutions with different dilation rates are used for parallel feature extraction. Specifically, the lower-level P3 feature uses a dilation rate of... r =1,2 dilated convolutions enhance texture details; the middle layer P4 feature uses a dilation rate of... r Multi-dilation rate dilated convolutions with values ​​of 2, 4, and 8 expand the receptive field to capture large-scale component features; high-level P5 features employ dilation rate... r Standard convolution with a value of 1 enhances the overall semantic contour; Features extracted with different porosity are concatenated and then adaptively fused through convolutional layers to obtain enhanced features, which are then incorporated into the neck of the improved YOLOv11s network structure.

[0054] Specifically, this application addresses the problem that features of key components such as buckets, booms, and tracks in engineering vehicles are easily lost during network fusion and are prone to category confusion with conventional vehicles. To address this, a three-scale parallel feature enhancement path for engineering vehicles is added between the YOLOv11s backbone network output and the neck feature fusion path. This enhancement path introduces a Dilated Multi-scale Enhancement Module (DREM) after the P3, P4, and P5 scale features output from the backbone. The low-level P3 features are enhanced with DREM to improve texture details; the mid-level P4 features expand the receptive field through multi-dilation convolution to specifically capture large-scale component features of engineering vehicles; and the high-level P5 features enhance the overall semantic contour. The enhanced features are directly fed into the subsequent fusion process of the neck without modifying the original neck structure. This achieves layered enhancement of engineering vehicle features without affecting the detection performance of conventional vehicles, effectively solving the category confusion problem between engineering vehicles and conventional vehicles.

[0055] In actual execution, the processor can handle the scale features of the input. (i=1, 2, 3), parallel feature extraction is performed using dilated convolutions with different dilation rates:

[0056] in, For the dilation rate of dilated convolution, different dilation rate combinations are used for features of different scales: the low-level P3 features use small dilation rates of r=1 and 2 to enhance texture details; the mid-level P4 features use multiple dilation rates of r=2, 4 and 8 to expand the receptive field to capture large-scale part features; and the high-level P5 features use standard convolution with r=1 to enhance the overall semantic contour.

[0057] The features extracted with different porosity rates are concatenated and fused to obtain the enhanced features:

[0058] in, These are learnable fusion weights.

[0059] This enables adaptive fusion of multi-scale void features, ultimately outputting enhanced features that are directly fed into the subsequent fusion process in the neck region.

[0060] To address the aforementioned technical problems, a second embodiment of this application provides an image domain vehicle target detection device based on a deep neural network to solve the same technical problems as the method embodiment. This device may include the following modules: The network building module is used to construct an improved YOLOv11s network structure. The improved YOLOv11s network structure includes: embedding a SimAM parameterless attention module into the C3K2 module of the backbone network to form the C3K2_SimAM improved module, used to enhance the response to small target features; and constructing the native YOLOv11s network's detection head as a dual-branch fine-grained detection head, FineGrainedHead. The dual-branch fine-grained detection head, FineGrainedHead, includes a fine-grained branch that mines subtle feature differences between similar vehicle models through a local spatial alignment module and a dynamic adaptive fusion module; and adds a dedicated feature extraction branch for engineering vehicles, introducing a dilated multi-scale enhancement module (DREM) between the backbone network output and the neck feature fusion path, thereby strengthening engineering vehicle features through the dedicated feature extraction branch.

[0061] The training module is used to preprocess the vehicle image dataset from the perspective of the UAV, and uses transfer learning and a multi-task joint loss function to train the improved YOLOv11s network structure end-to-end.

[0062] The inference module takes the optical image to be detected as input to the trained improved YOLOv11s network structure. After feature extraction, feature fusion and detection head processing, it outputs the bounding box coordinates, class confidence and class label of the vehicle target.

[0063] It is understood that the above-described device embodiments correspond to the method embodiments. Therefore, the device embodiments have all the beneficial effects of the method embodiments, and will not be repeated here.

[0064] The third embodiment of this application provides an electronic device, including a processor and a memory. The memory stores a computer program, and when the processor executes the computer program, it implements the image domain vehicle target detection method based on a deep neural network as provided in the previous embodiments. This electronic device can be a computing device in a deployment environment such as an unmanned aerial vehicle (UAV) airborne platform, a ground control station, or a server.

[0065] The fourth embodiment of this application provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the image domain vehicle target detection method based on a deep neural network as provided in the previous embodiments. The computer-readable storage medium may be ROM, RAM, disk, or optical disk, etc.

[0066] The fifth embodiment of this application provides a computer program product including computer-readable instructions that, when executed by a processor, implement the image domain vehicle target detection method based on a deep neural network as provided in the preceding embodiments. This computer program product can be provided as a software package, facilitating deployment and operation on different hardware platforms.

[0067] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are included within the patent protection scope of this application.

Claims

1. A deep neural network-based image domain vehicle target detection method, characterized in that, Applications of optical imaging systems mounted on drones for detecting ground vehicles include: An improved YOLOv11s network structure is constructed, comprising: embedding a SimAM parameterless attention module into the C3K2 module of the backbone network to form an improved C3K2_SimAM module, used to enhance the response to small target features; constructing the native YOLOv11s network's detector head as a dual-branch fine-grained head, the FineGrained Head including parallel regular branches and fine-grained branches, the fine-grained branches mining subtle feature differences between similar vehicle models through a local spatial alignment module and a dynamic adaptive fusion module; the dual-branch fine-grained head FineGrained Head The regular branch is responsible for target bounding box regression and class confidence prediction. The features output by the regular branch and the fine-grained branch are dynamically and adaptively fused to obtain the fine-grained classification result. A special feature extraction branch for engineering vehicles is added, and a dilated multi-scale enhancement module (DREM) is introduced between the backbone network output and the neck feature fusion path. In the dilated multi-scale enhancement module (DREM), for the input P3, P4, and P5 three-layer features, dilated convolutions with different dilation rates are used for parallel feature extraction. Among them, the low-level P3 feature uses a dilation rate of... r =1,2 dilated convolutions enhance texture details; the middle layer P4 feature uses a dilation rate of... r Multi-dilation rate dilated convolutions with values ​​of 2, 4, and 8 expand the receptive field to capture large-scale component features; high-level P5 features employ dilation rates of... r Standard convolution with a value of 1 enhances the overall semantic contour; engineering vehicle features are enhanced through a dedicated feature extraction branch. The vehicle image dataset from the perspective of the UAV is preprocessed, and the improved YOLOv11s network structure is trained end-to-end using a multi-task joint loss function; wherein, the multi-task joint loss function is composed of a weighted sum of bounding box regression loss, classification loss and fine-grained feature measurement loss; The optical image to be detected is input into the trained improved YOLOv11s network structure. After feature extraction, feature fusion and detection head processing, the bounding box coordinates, class confidence and class label of the vehicle target are output.

2. The image domain vehicle target detection method based on deep neural networks as described in claim 1, characterized in that, The SimAM parameterless attention module is embedded into the C3K2 module to obtain the improved C3K2_SimAM module. The importance weight of each neuron in the improved C3K2_SimAM module is calculated using an energy function, which is: in, This represents the average number of neurons within the current channel. The variance of neurons within the current channel. For learnable bias terms, The neurons within the current channel; The attention weights are obtained based on the energy function: ; The final enhanced features are: ,in, X The image is used as the input backbone network.

3. The image domain vehicle target detection method based on deep neural networks as described in claim 1, characterized in that, The regular branches of the dual-branch fine-grained head FineGrainedHead employ a CBS convolution structure for bounding box regression and target confidence prediction tasks. The fine-grained branches of the dual-branch fine-grained head include the Local Spatial Alignment (LSA) module and the Dynamic Adaptive Fusion (DAF) module. The Local Spatial Alignment (LSA) module is used to focus on local regions to uncover fine-grained local features between different vehicle models. The Dynamic Adaptive Fusion (DAF) module is used to concatenate global features with fine-grained local features, predict dynamic fusion weights, and perform weighted fusion.

4. The image domain vehicle target detection method based on deep neural networks as described in claim 3, characterized in that, In the Local Spatial Alignment (LSA) module, scale parameters are predicted via a network. s Translation parameters t The affine transformation of the input features is expressed as follows: in, These are the aligned features; Affine is the spatial alignment transformation operation. These are the transformation parameters, which include the scale parameter. s Translation parameters t; The Local Spatial Alignment (LSA) module includes an average pooling layer, a first fully connected layer, a SiLU activation layer, a second fully connected layer, and a Sigmoid activation layer connected in sequence. The input of the average pooling layer and the output of the Sigmoid activation layer are multiplied and then input into the first depthwise separable convolutional layer, which outputs the final affine transformation result.

5. The image domain vehicle target detection method based on deep neural networks as described in claim 3, characterized in that, In the Dynamic Adaptive Fusion Module (DAF), global features and fine-grained local features are respectively input into an average pooling layer and then concatenated. The concatenated result is then sequentially input into a convolutional layer and a sigmoid activation layer. The output of the sigmoid activation layer and the inputs of the two average pooling layers are multiplied and then input into a second depthwise separable convolutional layer, which outputs the weighted fusion weights.

6. A vehicle target detection device based on a deep neural network in the image domain, applied to the detection of ground vehicles by an optical imaging system mounted on an unmanned aerial vehicle, comprising: A network construction module is used to construct an improved YOLOv11s network structure. This improved YOLOv11s network structure includes: embedding a SimAM parameterless attention module into the C3K2 module of the backbone network to form an improved C3K2_SimAM module, used to enhance the response to small target features; constructing the native YOLOv11s network's detection head as a dual-branch fine-grained detection head, FineGrainedHead, which includes parallel regular branches and fine-grained branches. The fine-grained branches mine subtle feature differences between similar vehicle models through a local spatial alignment module and a dynamic adaptive fusion module. The regular branch is responsible for target bounding box regression and class confidence prediction. The features output by the regular branch and the fine-grained branch are dynamically and adaptively fused to obtain the fine-grained classification result. A special feature extraction branch for engineering vehicles is added, and a dilated multi-scale enhancement module (DREM) is introduced between the backbone network output and the neck feature fusion path. In the dilated multi-scale enhancement module (DREM), for the input P3, P4, and P5 three-layer features, dilated convolutions with different dilation rates are used for parallel feature extraction. Among them, the low-level P3 feature uses a dilation rate of... r =1,2 dilated convolutions enhance texture details; the middle layer P4 feature uses a dilation rate of... r Multi-dilation rate dilated convolutions with values ​​of 2, 4, and 8 expand the receptive field to capture large-scale component features; high-level P5 features employ dilation rates of... r Standard convolution with a value of 1 enhances the overall semantic contour; engineering vehicle features are enhanced through a dedicated feature extraction branch. The training module is used to preprocess the vehicle image dataset from the perspective of the UAV and to train the improved YOLOv11s network structure end-to-end using a multi-task joint loss function; wherein, the multi-task joint loss function is composed of a weighted sum of bounding box regression loss, classification loss and fine-grained feature measurement loss. The inference module is used to input the optical image to be detected into the trained improved YOLOv11s network structure. After feature extraction, feature fusion and detection head processing, it outputs the bounding box coordinates, class confidence and class label of the vehicle target.

7. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the image domain vehicle target detection method based on a deep neural network as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the image domain vehicle target detection method based on a deep neural network as described in any one of claims 1 to 6.

9. A computer program product comprising computer-readable instructions that, when executed by a processor, implement the image domain vehicle target detection method based on a deep neural network as described in any one of claims 1 to 5.

Citation Information

Patent Citations

  • Street scene infrared target detection method

    CN117274794A

  • Unmanned aerial vehicle aerial image target detection method based on spatial detail perception enhancement and cross-layer semantic fusion

    CN121170524A