Aerial photography target detection method based on enhanced YOLOv5

Abstract

The invention discloses a target detection method based on an enhanced YOLOv5 network, and the method comprises the steps: extracting a picture containing a vehicle and a ship from an aerial image, drawing a minimum enclosing rectangular frame for each target in the picture, marking the corresponding target type and position information, and constructing a training data set; using a K-means clustering algorithm to carry out clustering analysis on all labeled target frame sizes; an EMA module is integrated into a C3 layer of a backbone to form a C3-EMA structure, a BiFPN structure is used for replacing an FPN + PAN structure in YOLO v5, an original detection head is replaced by a DAFNE structure in a head layer, and an EMA-BIFPN-DAFNE fusion model based on an improved YOLOv5 network is constructed for predicting vehicle and ship information in an aerial image; carrying out feature extraction and detection model training, and storing an optimal EMA-BIFPN-DAFNE fusion model for reasoning deployment; and inputting an aerial image collected in an actual engineering scene into the trained optimal EMA-BIFPN-DAFNE fusion model to carry out target detection. According to the method, the detection precision is remarkably improved while the reasoning speed is kept.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a high-precision target detection method based on an enhanced YOLOv5 network. BACKGROUND

[0002] In the UAV aerial scene, small targets account for more than 60% of the total target quantity, but the small target miss detection rate of existing detection algorithms is as high as 40%-50%, which directly affects the application effect of UAV in key fields such as border patrol, disaster relief, traffic monitoring, etc. Especially in the emergency rescue scene, missing one trapped person may cause irreparable loss. Therefore, it is of great practical significance and urgent need to develop an algorithm that can not only maintain real-time performance but also significantly improve the detection accuracy of small targets.

[0003] The existing technical solutions mainly include three directions. In the aspect of feature pyramid improvement, PANet of CVPR 2018 shortens the information propagation path through bottom-up path enhancement, and improves the small target AP on the COCO dataset by 3.9%. BiFPN proposed by EfficientDet of CVPR 2020 improves the detection accuracy while reducing the parameter amount through bidirectional feature fusion and weighted feature fusion. Patent CN110826478A proposes a UAV target detection method based on improved FPN, which retains more small target features by increasing the number of horizontal connection layers. In the aspect of attention mechanism, CBAM of ECCV 2018 achieves significant improvement by concatenating channel attention and spatial attention modules, and EPSANet of 2021 proposes a pyramid segmentation attention module to improve small target detection through multi-scale feature extraction. Patent US20210256311A1 describes a multi-scale attention-based aerial target detection system. In the aspect of data augmentation, "Augmentation for small object detection" of 2019 proposes an oversampling strategy for small targets, which improves the AP on the VisDrone dataset by 7.1%, and Slicing Aided Hyper Inference of ICIP 2020 processes high-resolution aerial images through image slicing strategy.

[0004] However, these existing solutions have obvious technical defects. PANet enhances information flow, but its fixed fusion weight cannot adapt to the dynamic needs of targets of different scales. In aerial scenes, when the scale difference of targets is large, the fixed weight leads to feature suppression of small targets. The detection recall rate of 10-30 pixel targets on the VisDrone dataset is only increased by 5%. The CBAM module is designed for image classification and has a receptive field mismatch problem when directly applied to the detection task. Its 7x7 fixed convolution kernel cannot effectively model extremely small targets in aerial images, resulting in an overly smooth attention map that loses the location information of small targets.

[0005] Although BiFPN reduces the computational load compared to complete dense connection, multiple up-sampling operations still bring significant memory overhead when processing 4K aerial images. Experimental results show that the inference speed drops to 12FPS on a 2080Ti GPU when processing high-resolution images, which cannot meet the real-time requirement. The slicing strategy can improve small target detection accuracy, but it destroys the global context information of the target. When the target is located at the boundary of the slice, the target will be segmented, leading to missed detection or repeated detection. At the same time, multiple inferences increase the inference time by 3-4 times. Data augmentation methods such as oversampling and copy-paste strategy can easily lead to model overfitting to enhanced small target patterns. Experiments show that such methods have a performance drop of 15%-20% in real scene testing, indicating that the model has learned enhanced artifacts rather than real target features.

[0006] These technical defects show that existing solutions either sacrifice real-time performance for accuracy or introduce other problems when improving small target detection capability. Therefore, there is an urgent need for a solution that can maintain efficient inference while optimizing for aerial small target characteristics. SUMMARY

[0007] The purpose of the present application is to provide an aerial target detection method based on enhanced YOLOv5.

[0008] The technical solution for achieving the purpose of the present application is: a target detection method based on an enhanced YOLOv5 network, comprising the following steps:

[0009] Step 1: Extract pictures containing vehicles and ships from aerial images, draw the minimum bounding rectangle for each target in the picture, and mark the corresponding target class and location information to construct a training dataset; use the K-means clustering algorithm to analyze the clustering of all labeled target box sizes;

[0010] Step 2: Form a C3-EMA structure by integrating the EMA module into the C3 layer of the backbone, replace the FPN+PAN structure in YOLOv5 with the BiFPN structure, and replace the original detection head in the head layer with the DAFNE detection head to construct an EMA-BIFPN-DAFNE fusion model based on the improved YOLOv5 network for predicting vehicle and ship information in aerial images, wherein:

[0011] The C3-EMA module serves as a feature extraction backbone network that combines a cross-stage partial network structure with an efficient multi-scale attention mechanism to extract and attention-weight features from input images, generating enhanced feature representations. The BiFPN module receives the enhanced features and uses a weighted aggregation method to realize multi-scale information interaction through a bidirectional feature fusion strategy, generating fused multi-scale feature representations. The DAFNE detection head receives the multi-scale features and is divided into two branches: a classification branch that uses focal loss to predict target classes, and a regression branch that uses a center-to-corner strategy to predict four corner coordinates and introduces a direction perception centrality function to improve prediction quality.

[0012] Step 3: Construct a loss function based on the classification focal loss, regression smooth L1 loss, and binary cross-entropy loss of centrality, perform forward reasoning based on the training data set, and use a stochastic gradient descent optimizer to train and save the EMA-BIFPN-DAFNE fusion model for target detection deployment.

[0013] Step 4: Input the aerial images collected in actual engineering scenarios into the trained optimal EMA-BIFPN-DAFNE fusion model for target detection.

[0014] Further, extract pictures containing vehicles and ships from aerial images, draw the minimum bounding rectangle for each target in the picture, and mark the corresponding target class and location information to construct a training data set, wherein:

[0015] The annotation results are saved as txt format files, recording the class information and location information of each target, where the location information includes the normalized coordinates (x, y) of the target center point, the normalized width w, and the height h of the bounding box.

[0016] Further, use multiple data augmentation strategies to expand the training samples, including:

[0017] Use Mosaic data augmentation technology to randomly select four different training pictures for splicing;

[0018] Use the MixUp mixing enhancement method to pixel-level fuse two pictures at a certain ratio with a set mixing probability;

[0019] The random cropping technique is applied, and the cropping ratio is set between 0.5 and 1.0 to ensure that the cropped picture retains enough target information;

[0020] Random horizontal flipping is used, and the flipping probability is set to 0.5 to increase the diversity of the data.

[0021] Further, step 2: integrate the EMA module into the C3 layer of the backbone to form a C3-EMA structure, replace the FPN+PAN structure in YOLO v5 with a BiFPN structure, and replace the original detection head in the head layer with a DAFNE detection head to construct an EMA-BIFPN-DAFNE fusion model based on the improved YOLOv5 network for predicting vehicle and ship information in aerial images. The specific method is:

[0022] (1) C3_EMA module

[0023] The preprocessed input image is first subjected to feature extraction by the backbone network to generate multi-level feature representations, among which the key multi-scale feature maps (P3, P4, P5) are output to the EMA module for enhancement processing.

[0024] The EMA module comprises a feature reshaping unit, a directional pooling unit, a feature fusion unit, an attention generating unit, a feature enhancement unit, a soft attention unit and an output reshaping unit. The feature reshaping unit first receives multi-scale feature maps output from the backbone network, including P3 feature maps (high resolution low semantic level), P4 feature maps (medium resolution medium semantic level) and P5 feature maps (low resolution high semantic level), which have the same channel dimension [B, C, H, W] but different spatial sizes. The feature reshaping unit reshapes the input features into group features group x according to the groups parameter, with a dimension of [B x groups, C / groups, H, W]. The group features group x are then input in parallel to two processing branches: the directional pooling unit and the feature enhancement unit. The directional pooling unit performs spatial dimension reduction operations to generate horizontal features x h by horizontal pooling pool h (group x) with a dimension of [B x groups, C / groups, H, 1], and vertical features x w by vertical pooling pool w (group x) with a dimension of [B x groups, C / groups, 1, W]. The feature fusion unit concatenates the two directional features in the spatial dimension to form combined features with a dimension of [B x groups, C / groups, H+W, 1], and generates fusion features hw through 1x1 convolution processing. The attention generating unit constructs a spatial attention mechanism. First, group normalization is applied to the group features group x to obtain standardized features. Then, the standardized features are adaptively fused with the fusion features hw. The fusion strategy includes element-wise addition after spatial dimension alignment or channel dimension concatenation. The fused features are activated by a sigmoid function to generate spatial attention weights. Meanwhile, the feature enhancement unit generates enhanced features by 3x3 convolution on the group features group x. The enhanced features are multiplied by the spatial attention weights to obtain spatial weighted features. The soft attention unit calculates the final attention weights for the spatial weighted features and applies them to the original group features group x to achieve adaptive enhancement of the features. The output reshaping unit reshapes the attention-enhanced features back to the original dimension [B, C, H, W] to generate enhanced P3, P4 and P5 feature maps.

[0025] (2) BiFPN module

[0026] The BiFPN module comprises a feature receiving unit, a channel unification unit, a top-down path unit, a bottom-up path unit, a deep separable convolution processing unit and an output unit. The feature receiving unit receives features from three different levels of the backbone, including P3 features from C3_EMA output dimension [B, 128, 80, 80], P4 features from C3_EMA output dimension [B, 256, 40, 40], and P5 features from SPPF output dimension [B, 512, 20, 20]. The channel unification unit unifies the P3 features, the P4 features and the P5 features to 256 channels through lateral connection lateral_convs to generate initial pyramid features. The top-down path unit up-samples the P5 features to the P4 size, and the P4 features are weightedly fused through a learnable weight w1 to generate enhanced P4_td features. Then, the P4_td features are up-sampled to the P3 size, and the P3 features are weightedly fused through a learnable weight w2 to generate enhanced P3_out features. The bottom-up path unit down-samples the P3_out features to the P4 size, and the P4_td features and the P4 features are weightedly fused through a learnable weight w3 to generate final P4_out features. Then, the P4_out features are down-sampled to the P5 size, and the original P5 features are weightedly fused through a learnable weight w4 to generate final P5_out features. The deep separable convolution processing unit processes the P3_out features, the P4_out features and the P5_out features to generate three enhanced feature maps P3_final feature map, P4_final feature map and P5_final feature map, which are outputted by the output unit and transmitted to the DAFNE detection head.

[0027] (3) DAFNE detection head

[0028] The DAFNE detection head comprises a feature receiving unit, a classification prediction tower unit, a regression prediction tower unit, a multi-branch prediction head unit, and a post-processing unit. The feature receiving unit receives three scale features from the BiFPN output: a P3_final feature map with a dimension of [B, C_in=256, H3, W3], a P4_final feature map with a dimension of [B, C_in=256, H4, W4], and a P5_final feature map with a dimension of [B, C_in=256, H5, W5]. The classification prediction tower unit constructs a deep feature extraction network for the input feature xi in each detection level Pi through 4 layers of consecutive 3×3 convolution, each layer of convolution followed by BatchNorm2d normalization and ReLU activation function, keeping the channel number 256 unchanged, to generate classification features cls_feat. The regression prediction tower unit uses the same network structure parameters as the classification prediction tower but with independent weights, to generate regression features reg_feat for feature representation of position and shape prediction. The multi-branch prediction head unit comprises four parallel prediction branches: a classification prediction branch that converts the classification features to class prediction logits with a dimension of [B, num_classes, Hi, Wi] through a 3×3 convolution layer, a center point prediction branch that converts the regression features to target center point coordinate prediction with a dimension of [B, center_dims, Hi, Wi] through a 3×3 convolution layer, a corner point prediction branch that converts the regression features to four corner point coordinate offset prediction with a dimension of [B, corner_dims, Hi, Wi] through a 3×3 convolution layer, and a directed centrality prediction branch that converts the regression features to prediction box quality value prediction with a dimension of [B, centerness_dims, Hi, Wi] through a 3×3 convolution layer and a sigmoid activation function. The post-processing unit first adds the center point coordinates and the corner point offsets to obtain the final quadrilateral corner point coordinates through a center point to corner point conversion strategy, then calculates a special quality evaluation value using the DAFNE formula, and decodes the coordinates of the detection results of the three scales, filters the confidence threshold, processes the NMS algorithm for directed bounding boxes using non-maximum suppression and limits the maximum number of detections to generate the final detection results, including bounding box regression, confidence prediction, and class classification.

[0029] Further, step 3: a loss function is constructed based on the classification-based focal loss, the regression-based smooth L1 loss, and the centrality-based binary cross-entropy loss. Forward reasoning is performed based on the training data set, and a stochastic gradient descent optimizer is used to train the saved EMA-BIFPN-DAFNE fusion model for target detection deployment, wherein:

[0030] The single GPU batch size is set as bs (batch size), the equivalent batch size bs_eff (effective batch size) is realized through the gradient accumulation technology, that is, after accumulating the gradient of n_acc batches, the parameter is updated once (n_acc = bs_eff / bs); the initial learning rate is set as lr_init, a large initial value is used to accelerate the early convergence; the learning rate scheduling adopts a combination strategy, the warmup strategy is used for the first n_warmup rounds, the learning rate is linearly increased from 0 to lr_init, then the cosine annealing strategy is used, the learning rate is smoothly decayed from lr_init to the final value lr_final;

[0031] The optimizer selects SGD, the momentum parameter is set as beta to provide training stability, and the weight decay coefficient is set as lambda to prevent model overfitting.

[0032] A target detection system based on an enhanced YOLOv5 network, implementing the target detection method based on the enhanced YOLOv5 network, realizing target detection based on the enhanced YOLOv5 network, and four modules are used to execute steps 1-4.

[0033] A computer device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, when the processor executes the computer program, the target detection method based on the enhanced YOLOv5 network is implemented, and target detection based on the enhanced YOLOv5 network is realized.

[0034] A computer readable storage medium, having a computer program stored thereon, when the computer program is executed by a processor, the target detection method based on the enhanced YOLOv5 network is implemented, and target detection based on the enhanced YOLOv5 network is realized.

[0035] Compared with the prior art, the present application has the following advantages: 1) the high-efficiency multi-scale attention mechanism is introduced through the C3_EMA module, which significantly improves the expression ability of the features, especially the adaptability of different scale targets; 2) the BiFPN bidirectional feature pyramid network is used to realize the bidirectional flow and adaptive fusion of the features, and the feature information of different levels is fully utilized; 3) the DAFNE is used to adopt the direction-oriented centrality function and the center-to-corner bounding box prediction strategy, the normalized vertical distance from the feature map position to the edge of the directional bounding box is measured to reduce the weight of low-quality prediction, and the positioning performance of the directional target detection is improved; 4) the optimization of the overall architecture enables the model to maintain the inference speed while significantly improving the detection accuracy, especially the detection performance of small targets and complex scenes; the modular design enables the components to be flexibly combined and replaced, facilitating optimization according to specific application scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 The left side is the backbone network, the middle part is the neck network, and the right side is the detection head.

[0037] Figure 2 The C3-EMA module structure diagram of the application.

[0038] Figure 3 The BiFPN structure diagram of the application.

[0039] Figure 4 The DAFNE detection head structure diagram of the application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0041] A target detection method based on an enhanced YOLOv5 network, comprising the following steps:

[0042] Step 1: Extract pictures containing vehicles and ships from aerial images, draw the minimum bounding rectangle frame for each target in the picture, and mark the corresponding target class and position information, construct a training data set, and use K-means clustering algorithm to analyze the clustering of all labeled target frame sizes;

[0043] First, extract pictures containing targets such as vehicles and ships from aerial images as raw data. For each picture, use LabelImg labeling software for fine labeling, draw the minimum bounding rectangle frame for each target in the picture, and mark the corresponding target class.

[0044] The labeling result is saved as a txt format file, which records the class information and position information of each target, including the normalized coordinates (x, y) of the target center point, the normalized width w and height h of the bounding box. These values are normalized with respect to the image size. The specific method is: divide the target center point coordinates by the width and height of the image respectively, and divide the width and height of the bounding box by the width and height of the image respectively, so that all coordinate values are normalized to the [0, 1] interval, thereby ensuring that the labeling data of different size images have a unified numerical range.

[0045] In the data preprocessing stage, first, the original image of the input training data set is adjusted to a uniform size and data enhancement operations are performed, and through various data enhancement strategies, the training samples are expanded and the generalization ability of the model is improved. First, the Mosaic data enhancement technology is used, and four different training pictures are randomly selected for splicing to form a new training sample, effectively increasing the number of small targets in a single picture and enriching the complexity of the background. Second, the MixUp mixed enhancement method is used, and two pictures are pixel-level fused according to a certain proportion with a set mixing probability. In addition, the random cropping technique is also applied, and the cropping ratio is set between 0.5 and 1.0 to ensure that the cropped picture retains enough target information; at the same time, random horizontal flipping is used with a flipping probability of 0.5 to increase the diversity of the data.

[0046] The K-means clustering algorithm is used to cluster all the labeled target box sizes in the data set, and according to the scale distribution characteristics of the targets in the aerial image, 9 optimized anchor sizes are obtained. These 9 anchors are arranged in order from small to large, and every 3 are assigned to the network's P3(8 times down-sampling), P4(16 times down-sampling), and P5(32 times down-sampling) three different scale detection layers, so that each detection layer is responsible for detecting targets within a corresponding scale range.

[0047] The preprocessed complete data set is randomly divided according to an 8:2 ratio, of which 80% is used as the training set (about 2500 pictures) for model parameter learning and optimization; the remaining 20% is used as the test set (about 600 pictures) for evaluating the generalization performance of the model.

[0048] Step 2: Integrate the EMA module into the C3 layer of the backbone to form a C3-EMA structure, replace the FPN+PAN structure in YOLOv5 with a BiFPN structure, and replace the original detection head in the head layer with a DAFNE detection head. Construct an EMA-BIFPN-DAFNE fusion model based on the improved YOLOv5 network to predict vehicle and ship information in aerial images;

[0049] (1) Design and implement the C3_EMA module

[0050] The preprocessed input image is first passed through the backbone network for feature extraction to generate multi-level feature representations, among which the key multi-scale feature maps (P3, P4, P5) are output to the EMA module for enhancement processing.

[0051] The EMA module comprises a feature reshaping unit, a directional pooling unit, a feature fusion unit, an attention generating unit, a feature enhancement unit, a soft attention unit and an output reshaping unit. The feature reshaping unit first receives multi-scale feature maps output from the backbone network, including a P3 feature map (high resolution low semantic level), a P4 feature map (medium resolution medium semantic level) and a P5 feature map (low resolution high semantic level), which have the same channel dimension [B, C, H, W] but different spatial sizes. The feature reshaping unit reshapes the input features into group features group x according to the groups parameter, with a dimension of [B x groups, C / groups, H, W]. The group features group x are then input in parallel to two processing branches: the directional pooling unit and the feature enhancement unit. The directional pooling unit performs spatial dimension reduction operations to generate horizontal features x h by horizontal pooling pool h (group x) with a dimension of [B x groups, C / groups, H, 1], and vertical features x w by vertical pooling pool w (group x) with a dimension of [B x groups, C / groups, 1, W]. The feature fusion unit concatenates the two directional features in the spatial dimension to form combined features with a dimension of [B x groups, C / groups, H+W, 1], and generates fusion features hw through 1x1 convolution processing. The attention generating unit constructs a spatial attention mechanism. First, group normalization is applied to the group features group x to obtain standardized features. Then, the standardized features are adaptively fused with the fusion features hw. The fusion strategy includes element-wise addition after spatial dimension alignment or channel dimension concatenation. The fused features are activated by a sigmoid function to generate spatial attention weights. Meanwhile, the feature enhancement unit generates enhanced features by 3x3 convolution on the group features group x. The enhanced features are multiplied by the spatial attention weights to obtain spatial weighted features. The soft attention unit calculates the final attention weights for the spatial weighted features and applies them to the original group features group x to achieve adaptive enhancement of the features. The output reshaping unit reshapes the attention-enhanced features back to the original dimension [B, C, H, W] to generate enhanced P3, P4 and P5 feature maps.

[0052] (2) Design and implement the BiFPN structure

[0053] The BiFPN module comprises a feature receiving unit, a channel unification unit, a top-down path unit, a bottom-up path unit, a deep separable convolution processing unit and an output unit. The feature receiving unit receives features from three different levels of the backbone, including P3 features from C3_EMA output dimension [B, 128, 80, 80], P4 features from C3_EMA output dimension [B, 256, 40, 40], and P5 features from SPPF output dimension [B, 512, 20, 20]. The channel unification unit unifies the P3 features, the P4 features and the P5 features to 256 channels through lateral connection lateral_convs to generate initial pyramid features. The top-down path unit up-samples the P5 features to the P4 size, and the P4 features are weightedly fused through a learnable weight w1 to generate enhanced P4_td features. Then, the P4_td features are up-sampled to the P3 size, and the P3 features are weightedly fused through a learnable weight w2 to generate enhanced P3_out features. The bottom-up path unit down-samples the P3_out features to the P4 size, and the P4_td features and the P4 features are weightedly fused through a learnable weight w3 to generate final P4_out features. Then, the P4_out features are down-sampled to the P5 size, and the original P5 features are weightedly fused through a learnable weight w4 to generate final P5_out features. The deep separable convolution processing unit processes the P3_out features, the P4_out features and the P5_out features to generate three enhanced feature maps P3_final feature map, P4_final feature map and P5_final feature map, which are outputted by the output unit and transmitted to the DAFNE detection head.

[0054] (3) Design and implement the DAFNE detection head

[0055] The DAFNE detection head comprises a feature receiving unit, a classification prediction tower unit, a regression prediction tower unit, a multi-branch prediction head unit and a post-processing unit, wherein the feature receiving unit receives three scale features from the BiFPN output: a P3_final feature map with a dimension of [B, C_in=256, H3, W3], a P4_final feature map with a dimension of [B, C_in=256, H4, W4] and a P5_final feature map with a dimension of [B, C_in=256, H5, W5]; the classification prediction tower unit constructs a deep feature extraction network through 4 layers of consecutive 3x3 convolution for the input feature xi in each detection level Pi, each layer of convolution is followed by BatchNorm2d normalization and ReLU activation function, the channel number is kept unchanged at 256, and a classification feature cls_feat is generated; the regression prediction tower unit adopts the same network structure parameters as the classification prediction tower but has independent weights, and generates a regression feature reg_feat for feature representation of position and shape prediction; the multi-branch prediction head unit comprises four parallel prediction branches: a classification prediction branch converts the classification feature into a class prediction logits with a dimension of [B, num_classes, Hi, Wi] through a 3x3 convolution layer, a center point prediction branch converts the regression feature into a target center point coordinate prediction with a dimension of [B, center_dims, Hi, Wi] through a 3x3 convolution layer, a corner point prediction branch converts the regression feature into a four corner point coordinate offset prediction with a dimension of [B, corner_dims, Hi, Wi] through a 3x3 convolution layer, and a directed centrality prediction branch converts the regression feature into a prediction box quality value prediction with a dimension of [B, centerness_dims, Hi, Wi] through a 3x3 convolution layer and a sigmoid activation function; the post-processing unit first adds the center point coordinates and the corner point offset to obtain the final quadrilateral corner point coordinates through a center point to corner point conversion strategy, then calculates a special quality evaluation value using the DAFNE formula, decodes the coordinates of the detection results of the three scales, filters the confidence threshold, adopts a non-maximum suppression and limits the maximum number of detections for the NMS algorithm processing of the directed boundary box, and generates the final detection results.

[0056] Step 3: Feature extraction and detection model training is performed, first forward inference: the C3-EMA module as the feature extraction backbone network combines high-efficiency multi-scale attention mechanism through cross-stage partial network structure to generate enhanced feature representation by feature extraction and attention weighting processing on the input image, the BiFPN module receives the EMA enhanced feature to generate the fused multi-scale feature representation by using the weighted aggregation method to realize multi-scale information interaction through the bidirectional feature fusion strategy, the DAFNE detection head receives the multi-scale feature, which is divided into classification and regression two branches: the classification branch uses focal loss to predict the target category, the regression branch uses the center-to-corner strategy to predict the four corner coordinates, and introduces the direction perception center degree function to improve the prediction quality; the loss function includes the focal loss of classification, the smooth L1 loss of regression and the binary cross entropy loss of center degree; the random gradient descent optimizer is used for training, and the final model is saved for target detection deployment.

[0057] The DAFNE total loss function adopts a weighted fusion strategy, specifically:

[0058] L_total = λ_cls × L_cls + λ_reg × L_reg + λ_centerness × L_centerness × balance_i

[0059] Where: L_cls = -α(1-p_t)^γ×log(p_t), p_t = {p_i if y_i = 1, 1-p_i if y_i = 0} L_reg = SmoothL1(pred_coords-target_coords) L_centerness = -[t_centerness×log(p_centerness)+(1-t_centerness)×log(1-p_centerness)]

[0060] The loss function is composed of three main components: the classification loss L cls = -a(1-p t )^g x log(p t ), where p t is the class prediction probability, a is the balance factor, and g is the focus parameter, which solves the class imbalance problem through adaptive weight adjustment; the bounding box regression loss L reg is calculated using the smooth L1 loss, where b is the smoothing parameter, which regresses the difference between the predicted four corner coordinates and the true corner coordinates, and uses the N-point loss modulation strategy to improve training stability; the direction-aware centerness loss L centerness uses the binary cross-entropy loss, and the a parameter controls the decay rate, which is used to predict the normalized distance of the feature map position relative to the center of the direction bounding box, and the center degree target value is constructed by calculating the perpendicular distance to the four sides, and the multi-scale balance weight balance i is used to ensure the balanced contribution of different detection layers. The total loss function L total realizes the joint optimization of classification accuracy, corner positioning accuracy and centerness prediction, which is especially suitable for handling target detection tasks with arbitrary directions. The model training uses the following parameter configurations: batch size, total iteration number, initial learning rate, learning rate decay at milestone = [m1, m2,...], warm-up iteration number, warm-up initial learning rate, weight decay, momentum, to ensure the best performance and full convergence of the model.

[0061] The optimizer selects SGD, the momentum parameter is set to b to provide training stability, and the weight decay coefficient is set to l to prevent model overfitting. The loss function includes three parts, the classification loss weight is set to w cls to balance the importance of the classification task, the regression loss weight is set to w reg, and the direction-aware centerness loss weight is set to w centerness as an important optimization target. A strict monitoring strategy is implemented during the training process. Record each loss value every n_log batch to monitor the training dynamics, and calculate the mAP index on the validation set after each epoch to evaluate the model performance. The model saving strategy automatically saves the model weight file with the highest validation set mAP, and also retains the latest n_ckpt checkpoints for easy backtracking and analysis.

[0062] Step 4: The original data collected in the actual engineering scene is input into the trained optimal target detection model for detection. Due to the differences in image quality and other aspects between the actual collected data and the training data, it is necessary to perform preprocessing operations on the original data to ensure that the data format is consistent with the model input requirements. After the preprocessed data is input into the model, the accurate position positioning and class recognition results of the target are obtained, realizing the stable application of the model in the actual engineering environment.

[0063] The trained model is comprehensively evaluated on the DOTA test set, and a multi-dimensional evaluation index system is adopted. mAP@0.5 represents the average precision calculated when the IoU threshold is set to 0.5, reflecting the basic detection ability of the model. mAP@0.5:0.95 represents the average precision of the IoU threshold from 0.5 to 0.95 with a step of 0.05, reflecting the positioning accuracy of the model. For different scale targets, the average precision of AP_small (small target with area less than 32 2 pixels), AP_medium (medium target with area between 32 2 and 96 2 pixels), and AP_large (large target with area greater than 96 2 pixels) is calculated respectively. At the same time, the inference speed is tested, measured by FPS (frames per second), and all speed tests are performed on an RTX 3090 graphics card.

[0064] In summary, the present application integrates EMA efficient multi-scale attention, BiFPN bidirectional feature pyramid and DAFNE double attention fusion detection head, and achieves significant performance improvement on multiple data sets. This method not only improves the detection accuracy, especially the detection ability of small targets, but also maintains a high inference speed, and has good practical value. Specifically as follows:

[0065] The EMA module solves the problem of fixed convolution receptive field in traditional methods, which cannot adapt to extreme scale changes. Through multi-scale attention mechanism, the network can adaptively focus on targets of different sizes;

[0066] BiFPN replaces the single-direction FPN structure of YOLOv5, realizes bidirectional flow and dynamic weighted fusion of features, and avoids the attenuation of small target information in hierarchical transmission;

[0067] The DAFNE detection head overcomes the defect of the original detection head being sensitive to background noise, and accurately locates the target area through double attention mechanism.

[0068] Embodiment

[0069] In this embodiment, the performance of the enhanced YOLOv5 is verified on the remote sensing target detection data set DOTA.

[0070] Create a Python 3.8 virtual environment and install PyTorch 1.12.0 with CUDA 11.6 hardware configuration: NVIDIA RTX3090 GPU with 24GB of video memory dataset: DOTAv1.0 contains 2500 remote sensing images, a total of 188,282 labeled instances target categories: plane, ship, storage tank, baseball diamond, tennis court, basketball court, ground track field, harbor, bridge, large vehicle, small vehicle, helicopter, roundabout, soccerball field, swimming pool, etc. 15 categories

[0071] Training configuration adjustments:

[0072] Input resolution: 1024x1024 (adapted to the large size characteristics of remote sensing images)

[0073] Image cutting: cut the original large image into 1024x1024 sub-images with 200 pixel overlap

[0074] Anchor recalculation: optimized according to the target size distribution of DOTA, especially for small targets

[0075] Class weight: adjust the loss weight according to the class imbalance, and enhance the weight of small target classes

[0076] Rotation data augmentation: 0°, 90°, 180°, 270° rotation, suitable for the arbitrary direction characteristics of remote sensing images

[0077] Experimental results:

[0078] Plane class AP: 95.2% (original YOLOv5: 89.8%)

[0079] Ship class AP: 88.7% (original YOLOv5: 82.3%)

[0080] Small vehicle class AP: 79.5% (original YOLOv5: 71.2%)

[0081] Storage tank class AP: 86.3% (original YOLOv5: 80.6%)

[0082] Overall mAP improved by 6.8%, especially the small target detection performance improved significantly

[0083] Ablation experiment analysis:

[0084] Table 1 ablation experiment results

[0085]

[0086] As can be seen from Table 1, compared with the initial model, the final model after optimization has an accuracy P increased by 4.9%, a recall R increased by 0.8%, an mAP@0.5 increased by 2.9%, and an mAP@0.5:0.95 increased by 4.5%. This shows that the method of the application can effectively improve the detection performance of the model.

[0087] The experimental results show that:

[0088] 1. The EMA module significantly improves the accuracy, from 88.9% to 92.5%

[0089] 2. The BiFPN further improves the recall rate and mAP index

[0090] 3. The DAFNE optimizes the overall detection performance

[0091] 4. The modules work together to comprehensively improve the detection accuracy.

[0092] In summary, compared with the original YOLOv5s benchmark model, the enhanced model of the application has achieved significant performance improvement. The overall detection accuracy mAP@0.5 is improved from 88.9% to 92.1%, an increase of 3.2 percentage points. The more stringent evaluation index mAP@0.5:0.95 is improved from 65.1% to 69.6%, an increase of 4.5 percentage points, proving a significant improvement in model positioning accuracy. It is particularly noteworthy that the average precision AP small for small target detection is increased by 5.8 percentage points, fully verifying the significant improvement of the EMA attention mechanism, BiFPN feature fusion and DAFNE double attention detection head on small target detection capability. At the same time, the inference speed of the model remains at a real-time level of 85FPS, fully meeting the needs of actual application scenarios.

[0093] The technical features of the above embodiments can be combined in any way. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not contradict, they should be considered within the scope of the present application.

[0094] The above-described embodiments only express several implementation manners of the application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the application. It should be noted that for those skilled in the art, without departing from the concept of the application, a number of modifications and improvements can be made, which are within the scope of the application. Therefore, the protection scope of the application should be subject to the appended claims.

Claims

1. A target detection method based on an enhanced YOLOv5 network, characterized in that, The method comprises the following steps: Step 1: Extract pictures containing vehicles and ships from aerial images, draw the minimum bounding rectangle frame for each target in the picture, and mark the corresponding target class and position information to construct a training data set; use K-means clustering algorithm to cluster all the labeled target frame sizes; Step 2: Integrate the EMA module into the C3 layer of the backbone to form a C3-EMA structure, replace the FPN+PAN structure in YOLO v5 with a BiFPN structure, and replace the original detection head in the head layer with a DAFNE detection head to construct an EMA-BIFPN-DAFNE fusion model based on the improved YOLOv5 network for predicting vehicle and ship information in aerial images, wherein: The C3-EMA module serves as a feature extraction backbone network that combines a cross-stage partial network structure with an efficient multi-scale attention mechanism to perform feature extraction and attention weighting processing on the input image to generate enhanced feature representation. The BiFPN module receives the enhanced features and uses a weighted aggregation method to realize multi-scale information interaction to generate a fused multi-scale feature representation through a bidirectional feature fusion strategy. The DAFNE detection head receives the multi-scale features and is divided into a classification branch and a regression branch: the classification branch uses focal loss to predict target classes, and the regression branch uses a center-to-corner strategy to predict four corner coordinates and introduces a direction perception centrality function to improve prediction quality; Step 3: Construct a loss function based on the focal loss of classification, the smooth L1 loss of regression, and the binary cross-entropy loss of centrality, perform forward reasoning based on the training data set, and use a stochastic gradient descent optimizer to train and save the EMA-BIFPN-DAFNE fusion model for target detection deployment; Step 4: Input the aerial images collected in the actual engineering scene into the trained optimal EMA-BIFPN-DAFNE fusion model for target detection.

2. The target detection method based on the enhanced YOLOv5 network according to claim 1, wherein, Extract pictures containing vehicles and ships from aerial images, draw the minimum bounding rectangle frame for each target in the picture, and mark the corresponding target class and position information to construct a training data set, wherein: The annotation results are saved as txt format files, recording the class information and position information of each target, wherein the position information includes the normalized coordinates (x, y) of the target center point, the normalized width w and height h of the bounding box.

3. The target detection method based on the enhanced YOLOv5 network according to claim 1, wherein, Various data augmentation strategies are used to expand the training samples, including: Use Mosaic data augmentation technology to randomly select 4 different training pictures for splicing; Use the MixUp mixing enhancement method to pixel-level fuse two pictures at a certain ratio with a set mixing probability; Apply random cropping technology with a cropping ratio set between 0.5 and 1.0 to ensure that the cropped picture retains enough target information; Use random horizontal flipping with a flipping probability set to 0.5 to increase the diversity of the data.

4. The target detection method based on the enhanced YOLOv5 network according to claim 1, wherein, Step 2: Form a C3-EMA structure by integrating the EMA module into the C3 layer of the backbone, replace the FPN+PAN structure in YOLO v5 with the BiFPN structure, and replace the original detection head in the head layer with the DAFNE structure to construct an EMA-BIFPN-DAFNE fusion model based on the improved YOLOv5 network for predicting vehicle and ship information in aerial images. The specific method is as follows: (1) C3_EMA module The preprocessed input image is first passed through the backbone network for feature extraction, generating multi-level feature representations. The key multi-scale feature maps are output to the EMA module for enhancement processing. The EMA module includes a feature reshaping unit, a directional pooling unit, a feature fusion unit, an attention generation unit, a feature enhancement unit, a soft attention unit, and an output reshaping unit. The feature reshaping unit first receives multi-scale feature maps output from the backbone network, including P3 feature maps, P4 feature maps, and P5 feature maps. These feature maps have the same channel dimension [B, C, H, W] but different spatial sizes. The feature reshaping unit reshapes the input features into group features group_x according to the groups parameter, with a dimension of [B×groups, C / groups, H, W]. Group features group_x are then input in parallel to two processing branches: the directional pooling unit and the feature enhancement unit. The directional pooling unit performs spatial dimension reduction operations. It generates horizontal features x_h through horizontal pooling pool_h(group_x), with a dimension of [B×groups, C / groups, H, 1], and vertical features x_w through vertical pooling pool_w(group_x), with a dimension of [B×groups, C / groups, 1, W]. The feature fusion unit concatenates these two directional features in the spatial dimension to form combined features with a dimension of [B×groups, C / groups, H+W, 1], and generates fusion features hw through 1×1 convolution processing. The attention generation unit constructs a spatial attention mechanism. It first applies group normalization to the group features group_x to obtain standardized features, and then performs adaptive feature fusion between the standardized features and the fusion features hw. The fusion strategy includes element-wise addition after spatial dimension alignment or channel dimension concatenation. The fused features are activated by a sigmoid function to generate spatial attention weights. Meanwhile, the feature enhancement unit generates enhanced features through 3×3 convolution on the group features group_x. The enhanced features are multiplied by the spatial attention weights to obtain spatial weighted features. The soft attention unit calculates the final attention weights for the spatial weighted features and applies them to the original group features group_x to achieve adaptive enhancement of the features. The output reshaping unit reshapes the attention-enhanced features back to the original dimension [B, C, H, W] to generate enhanced P3, P4, and P5 feature maps. (2) BiFPN module The BiFPN module comprises a feature receiving unit, a channel unification unit, a top-down path unit, a bottom-up path unit, a deep separable convolution processing unit and an output unit. The feature receiving unit receives features from three different levels of the backbone, including P3 features from C3_EMA output dimension [B, 128, 80, 80], P4 features from C3_EMA output dimension [B, 256, 40, 40] and P5 features from SPPF output dimension [B, 512, 20, 20]. The channel unification unit unifies the P3 features, the P4 features and the P5 features to 256 channels through lateral connection lateral_convs to generate initial pyramid features. The top-down path unit up-samples the P5 features to the P4 size, and the P4 features are weightedly fused through a learnable weight w1 to generate enhanced P4_td features. Then, the P4_td features are up-sampled to the P3 size, and the P3 features are weightedly fused through a learnable weight w2 to generate enhanced P3_out features. The bottom-up path unit down-samples the P3_out features to the P4 size, and the P4_td features and the P4 features are weightedly fused through a learnable weight w3 to generate final P4_out features. Then, the P4_out features are down-sampled to the P5 size, and the original P5 features are weightedly fused through a learnable weight w4 to generate final P5_out features. The deep separable convolution processing unit processes the P3_out features, the P4_out features and the P5_out features to generate three enhanced feature maps P3_final feature map, P4_final feature map and P5_final feature map, which are transmitted to the DAFNE detection head by the output unit. (3) DAFNE detection head The DAFNE detection head comprises a feature receiving unit, a classification prediction tower unit, a regression prediction tower unit, a multi-branch prediction head unit and a post-processing unit, wherein the feature receiving unit receives three scale features from the BiFPN output: a P3_final feature map with a dimension of [B, C_in=256, H3, W3], a P4_final feature map with a dimension of [B, C_in=256, H4, W4] and a P5_final feature map with a dimension of [B, C_in=256, H5, W5]; the classification prediction tower unit constructs a deep feature extraction network through 4 layers of consecutive 3×3 convolution for the input feature xi in each detection level Pi, each layer of convolution is followed by BatchNorm2d normalization and ReLU activation function, the channel number 256 is kept unchanged, and the classification feature cls_feat is generated; the regression prediction tower unit adopts the same network structure parameters as the classification prediction tower but has independent weights, and generates the regression feature reg_feat for feature representation of position and shape prediction; the multi-branch prediction head unit comprises four parallel prediction branches: the classification prediction branch converts the classification feature into a class prediction logits with a dimension of [B, num_classes, Hi, Wi] through a 3×3 convolution layer, the center point prediction branch converts the regression feature into a target center point coordinate prediction with a dimension of [B, center_dims, Hi, Wi] through a 3×3 convolution layer, the corner point prediction branch converts the regression feature into a four corner point coordinate offset prediction with a dimension of [B, corner_dims, Hi, Wi] through a 3×3 convolution layer, and the directed center prediction branch converts the regression feature into a prediction box quality value prediction with a dimension of [B, centerness_dims, Hi, Wi] through a 3×3 convolution layer and a sigmoid activation function; the post-processing unit firstly adds the center point coordinates and the corner point offset to obtain the final quadrilateral corner point coordinates through a center point to corner point conversion strategy, then calculates the quality evaluation value by using the DAFNE formula, decodes the coordinates of the detection results of the three scales, filters the confidence threshold, adopts the non-maximum suppression and limits the maximum number of detections for the NMS algorithm processing of the directed bounding box, and generates the final directed target detection result, including the quadrilateral bounding box regression, the quality evaluation of each detection result, and the target class label and probability.

5. The target detection method based on the enhanced YOLOv5 network according to claim 1, wherein, Step 3: based on the classification-based focal loss, the regression-based smooth L1 loss and the binary cross-entropy loss of the center degree, a loss function is constructed, a forward inference is performed based on a training data set, a stochastic gradient descent optimizer is used to train the saved EMA-BIFPN-DAFNE fusion model, and the model is used for target detection deployment, wherein: The single GPU batch size is set to bs, and the equivalent batch size bs_eff is realized through the gradient accumulation technique, that is, the parameter is updated once after accumulating n_acc batches of gradients; the initial learning rate is set to lr_init, and a large initial value is used to accelerate the early convergence; the learning rate scheduling adopts a combination strategy, the first n_warmup rounds use the warmup strategy, the learning rate is linearly increased from 0 to lr_init, and then the cosine annealing strategy is used, the learning rate is smoothly decayed from lr_init to the final value lr_final; The optimizer selects SGD, the momentum parameter is set to β to provide training stability, and the weight decay coefficient is set to λ to prevent model overfitting.

6. A target detection system based on an enhanced YOLOv5 network, characterized in that, Implementing the target detection method based on the enhanced YOLOv5 network according to any one of claims 1-5, realizing the target detection based on the enhanced YOLOv5 network, and executing steps 1-4 in four modules. 7.A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, a target detection method based on the enhanced YOLOv5 network according to any one of claims 1-5 is implemented, and target detection based on the enhanced YOLOv5 network is realized. 8.A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, a target detection method based on the enhanced YOLOv5 network according to any one of claims 1-5 is implemented, and target detection based on the enhanced YOLOv5 network is realized.

Images