Unmanned aerial vehicle visual angle target detection method based on improved YOLO11n

Abstract

The invention discloses an unmanned aerial vehicle visual angle target detection method based on improved YOLO11n, and the method comprises the steps: 1, carrying out the preprocessing of an unmanned aerial vehicle aerial image data set, and dividing the preprocessed unmanned aerial vehicle aerial image data set into a training set, a verification set, and a test set; 2, constructing an improved YOLO11n model, replacing a C3k2 module in a backbone network of the original YOLO11n model with a C3PA2 module, replacing a neck network in the original YOLO11n model with an FIFFN frame, performing feature conversion on a feature map by the FIFFN through 1 * 1 convolution and the C3PA2 module, splicing deep feature maps in an up-sampling stage and a down-sampling stage with a shallow feature map from the backbone network, and constructing an improved YOLO11n model; increasing the resolution of the prediction feature map to 160 * 160; 3, firstly setting parameters, then training the improved YOLO11n model by using the training set, and adjusting and optimizing hyper-parameters of the improved YOLO11n model by using the verification set to obtain an optimal improved YOLO11n model; and 4, inputting a to-be-detected unmanned aerial vehicle aerial image into the optimal improved YOLO11n model, and outputting a detection result, thereby improving the precision of unmanned aerial vehicle visual angle target detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and in particular is a method for detecting targets from the perspective of an unmanned aerial vehicle based on improved YOLO11n. Background Art

[0002] As drone technology becomes increasingly mature, target detection from the perspective of drones can significantly improve the application efficiency of drones. Target detection from the perspective of drones combines intelligent target detection technology, enabling drones to autonomously locate targets of interest and give full play to their unique aerial perspective and high maneuverability. However, the existing target detection network model faces the following challenges when it is carried on drones for target detection: 1) Due to the flexibility of drone flight altitude and shooting angle, there is an extreme scale imbalance problem in the targets in drone images. This drastic change in scale puts higher requirements on the generalization ability of the target detection network model; 2) The coverage area of ​​foreground targets in drone images is often very small, making the learning process of the detection model more easily disturbed by background noise; 3) Targets from the perspective of drones are often concentrated in a small area of ​​the image, causing the problem of dense occlusion between targets. Therefore, the detection model needs to pay more attention to effectively distinguish different targets.

[0003] At present, the target detection methods from the perspective of drones are divided into machine learning methods and deep learning methods. Among them: the machine learning-based methods usually require manually designed feature extractors. However, manually designed features are often difficult to fully cover the various changes and complexities of the target. Moreover, due to the diversity of shooting perspectives, lighting conditions, target types and forms, it is difficult for drones to obtain comprehensive and high-quality training data, resulting in the generalization ability of machine learning-based target detection models being limited; the target detection method based on deep learning has become the optimal method for target detection from the perspective of drones. The target detection method based on deep learning can be roughly divided into two categories: one is a two-stage detector represented by Faster R-CNN, and the other is a single-stage detector represented by the YOLO series. The two-stage algorithm is generally not as accurate and fast as the one-stage algorithm. In contrast, the single-stage detector has the advantages of rapidity and high accuracy, which is more in line with the requirements of target detection from the perspective of drones.

[0004] However, existing single-stage detectors based on deep learning still find it difficult to cope with the problems of foreground targets in drone aerial images, such as small coverage area, strong background noise interference, little visual information, multi-scale changes, mutual occlusion and blurred boundaries, resulting in low accuracy of detection results. Summary of the invention

[0005] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a drone perspective target detection method based on improved YOLO11n, which improves the accuracy of drone perspective target detection.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for detecting a target from a drone's perspective based on improved YOLO11n includes the following steps:

[0008] Step 1: preprocess the drone aerial image dataset and divide the preprocessed drone aerial image dataset into a training set, a validation set and a test set;

[0009] Step 2: Build a drone view target detection model based on improved YOLO11n. The specific process is as follows:

[0010] Step 2.1, replace the Bottleneck module in the original YOLO11n model C2f module with C3PA or LK-MBAM to obtain the C3PA2 module. When the parameter c3PA of the C3PA2 module is True, the component of the C3PA2 module is C3PA. When the parameter c3PA of the C3PA2 module is False, the component of the C3PA2 module is LK-MBAM. Then use the C3PA2 module to replace the C3k2 module in the original YOLO11n model backbone network.

[0011] The C3PA module divides the input feature map equally in the channel dimension and inputs it into the left branch and the right branch respectively. The left branch contains a 1×1 convolution layer, and the right branch contains a 1×1 convolution layer and the LK_PAB module. The feature maps output by the left branch and the right branch are concatenated by the Concat module, and feature fusion is performed using 1×1 convolution to obtain the output feature map of the C3PA module.

[0012] The LK_PAB module divides the input feature map equally in the channel dimension, and then inputs the feature maps into the left branch of the LK_AB module and the right branch of the LK_MBAM module respectively. The feature maps output by the LK_AB module and the LK_MBAM module are concatenated by the Concat module, and 1×1 convolution is used for feature fusion to obtain the output feature map of the LK_PAB module.

[0013] The LK_AB module passes through the LayerNorm2d layer, the LKASA module and the DropPath layer in sequence to obtain an intermediate feature map. After the intermediate feature map is added to the feature map of the input LK_AB module, a feature map F1 is obtained. The feature map F1 passes through the LayerNorm2d layer, the CGLU layer and the DropPath layer in sequence to the feature map F2. The feature map F2 is added to the intermediate feature map to obtain the output feature map of the LK_AB module;

[0014] The LKASA module uses a 1×1 convolution module to triple the number of channels of the input feature map to obtain a feature map G1, and divides the feature map G1 into three parts in the channel dimension and assigns them to Q, K and V respectively. The feature maps corresponding to Q and K are respectively passed through the SLK-Atten module and then added to obtain the added feature map. The added feature map is multiplied with the feature map corresponding to V to obtain the output feature map of the LKASA module;

[0015] The SLK-Atten module performs feature sparseness on the input feature map through the AvgPool2d layer to obtain a feature map I′, which is sequentially subjected to a cascade of 1×3 depth convolution, 3×1 depth convolution, 1×5 expansion depth convolution, 5×1 expansion depth convolution and 1×1 depth convolution to obtain a feature map I″, which is subjected to a Hardswish activation function to obtain a weighted feature map with an attention mechanism, which is multiplied by the input feature map I of the SLK-Atten module to obtain an output feature map, which is residually connected to the feature map I′ to obtain an output feature map I of the SLK-Atten module out ;

[0016] After the LK_MBAM module performs feature conversion on the input feature map through a 3×3 convolution module, the feature map after feature conversion is divided into two parts in the channel dimension, one part is input into the identity mapping module, and the other part passes through the cascaded 5×5 depth convolution module and the 7×7 depth convolution module in sequence, and then the feature map output by the identity mapping module and the feature map output by the 7×7 depth convolution module are spliced ​​through the concat module, and then 1×1 convolution is used for feature conversion, and then the key features are focused through the SLK-Atten module to obtain the output feature map of the LK_MBAM module;

[0017] Step 2.2, use the FIFFN framework to replace the neck network in the original YOLO11n model. The FIFFN framework first performs feature conversion on each feature map output from the backbone network through a 1×1 convolution module and a C3PA2 module, and then concatenates the deep feature maps of the upsampling and downsampling stages of the FIFFN framework with the shallow feature maps of the same size from the backbone network through a jump connection, thereby replacing the 20×20 prediction feature map used in the neck network of the original YOLO11n model with a 160×160 prediction feature map;

[0018] Step 3: first set the parameters, then use the training set to train the drone view target detection model based on the improved YOLO11n, and use the validation set to tune the hyperparameters of the drone view target detection model based on the improved YOLO11n during the training process to obtain the optimal drone view target detection model based on the improved YOLO11n;

[0019] Step 4: Input the drone aerial image to be detected into the optimal drone perspective target detection model based on improved YOLO11n, and output the detection result.

[0020] Furthermore, the drone aerial image dataset of step 1 is a public VisDrone dataset, which includes 10 data categories, namely: pedestrians, people, bicycles, cars, vans, trucks, tricycles, sunshade tricycles, buses and motorcycles.

[0021] Furthermore, the processing of the input feature map by the C3PA2 module in step 2.1 includes the following steps:

[0022] Step 2.1.1: When the parameter c3PA of the C3PA2 module is True, the component of C3PA2 is C3PA, and the processing process of C3PA2 on the input feature map is:

[0023] Step 2.1.1.1, first transform the input feature map through the first 1×1 convolutional layer to generate an intermediate feature map, and then split the intermediate feature map into two parts, one part is directly passed to the Concat module, and the other part is passed to the C3PA module;

[0024] Step 2.1.1.2, concatenate the intermediate feature map directly passed to the Concat module and the feature map output by the C3PA module through the Concat module, and use 1×1 convolution for feature fusion to obtain the output feature map of the C3PA2 module;

[0025] Step 2.1.2: When the parameter c3PA of the C3PA2 module is False, the component of C3PA2 is LK-MBAM, and the processing process of C3PA2 on the input feature map is:

[0026] Step 2.1.2.1, first transform the input feature map through the first 1×1 convolution layer to generate an intermediate feature map, and then split the intermediate feature map into two parts, one part is directly passed to the Concat module, and the other part is passed to the LK-MBAM module;

[0027] Step 2.1.2.2: After the intermediate feature map directly passed to the Concat module is concatenated with the feature map output by the LK-MBAM module through the Concat module, 1×1 convolution is used for feature fusion to obtain the output feature map of the C3PA2 module.

[0028] Furthermore, the process of processing the input feature map by the LKASA module in step 2.1 is expressed by the formula:

[0029] LKASA(Q,K,V)=γ(ω(K)+ω(Q))·V

[0030] Where Q, K and V are obtained through independent linear transformations, γ(·) represents the deep convolution of fused features, and ω(·) represents the processing of the SLK-Atten module.

[0031] Furthermore, the process of the SLK-Atten module processing the input features in step 2.1 is expressed by the formula:

[0032] I′=Avg(I)

[0033] I″=DConv 5×1 (DConv 1×5 (DWConv 3×1 (DWConv 1×3 (I′)))

[0034] I out =Hardswish(Conv 1×1 (I″))·I+I′

[0035]

[0036] Where: I represents the input feature map of the SLK-Atten module; I′ represents the feature map after AvgPool2d feature sparseness; I″ represents the feature map after cascading 1×3 depth convolution, 3×1 depth convolution, 1×5 expansion depth convolution, 5×1 expansion depth convolution and 1×1 depth convolution; Avg represents 3×3 average pooling; DWConv represents depth convolution; DConv represents depth expansion convolution; Hardswish(·) represents the activation feature map; t represents the pixel value; I outRepresents the output feature map of the SLK-Atten module.

[0037] Furthermore, the processing process of the input feature map by the FIFFN framework in step 2.2 is as follows:

[0038] Step 2.2.1, first use 1×1 convolution to perform feature conversion on the feature map K1 output by the backbone network, then use the C3PA2 module to perform feature conversion on the 1×1 convolution output feature map, and then use a 1×1 convolution to perform information fusion to obtain the intermediate feature map;

[0039] Step 2.2.2, first use the nearest neighbor upsampling to decode the intermediate feature map to obtain a feature map with a resolution of 40×40, then use 1×1 convolution to perform feature conversion on the feature map K2 output by the backbone network, then concatenate the feature map output by the 1×1 convolution with the feature map with a resolution of 40×40 obtained by the decoding operation to obtain a fused feature map, then use the C3PA2 module and the 1×1 convolution module in turn to perform feature extraction and feature conversion operations on the fused feature map to obtain feature map K5, finally use the nearest neighbor upsampling to decode the feature map K5 to obtain a feature map with a resolution of 80×80;

[0040] Step 2.2.3, first use 1×1 convolution to perform feature conversion on the feature map K3 output by the backbone network, then concatenate the feature map output by 1×1 convolution with the feature map with a resolution of 80×80 to obtain a fused feature map, then use the C3PA2 module to extract features from the fused feature map to obtain feature map K6, and then use the nearest neighbor upsampling to decode the feature map K6 to obtain a feature map with a resolution of 160×160;

[0041] Step 2.2.4, first use 1×1 convolution to perform feature conversion on the feature map K4 output by the backbone network, then concatenate the feature map output by 1×1 convolution with the feature map with a resolution of 160×160 to obtain a fused feature map, and then use the C3PA2 module to extract features from the fused feature map to obtain a decoded output feature map;

[0042] Step 2.2.5, pass the decoded output feature map through the convolution layer with a convolution kernel size of 3×3 and a stride of 2, the BN layer and the SiLU layer in turn, and perform downsampling operations to obtain feature map K7, then concatenate feature maps K3, K6 and K7 to obtain a fused feature map, then use the C3PA2 module to extract features from the fused feature map, and then pass through the convolution layer with a convolution kernel size of 3×3 and a stride of 2, the BN layer and the SiLU layer in turn for downsampling operations to obtain feature map K8;

[0043] Step 2.2.6: Concatenate the feature maps K2, K5, and K8 to obtain a fused feature map, and then use the C3PA2 module to extract features from the fused feature map to obtain a predicted feature map with the smallest resolution.

[0044] Furthermore, the process of setting parameters in step 3 is: setting the batch-size to 16, the initial learning rate to 0.001, the weight decay parameter to 0.0005, and the momentum factor to 0.937; using the Adam optimizer, the number of training rounds is set to 200 rounds, and the learning rate decay strategy uses the cosine annealing algorithm.

[0045] Compared with the prior art, the present invention has the following technical effects:

[0046] The target detection method from the perspective of unmanned aerial vehicles based on improved YOLO11n proposed in the present invention can cope with the common problems of small foreground target coverage area, strong background noise interference and rich target scales in target detection tasks from the perspective of unmanned aerial vehicles, and improve the accuracy of target detection from the perspective of unmanned aerial vehicles, which is specifically embodied in: 1) by introducing the C3PA2 module, the components C3PA and LK-MBAM in the C3PA2 module can be flexibly replaced according to the parameter c3PA of the C3PA2 module, which solves the problem of less visual information carried by small targets in unmanned aerial images, and improves the ability of the network model to learn global features and local features; 2) by using the FIFFN framework to replace the neck network in the original YOLO11n model, it can accept information from different levels and promote the interaction between information at different levels, reduce the difficulty of fusing features at different levels, and solve the problem of insufficient information of small targets in the decoding process. Finally, by introducing a 160×160 high-resolution prediction feature map, the network model's perception ability of the edge contour information of the foreground target is enhanced, and the detector is focused on the small target.

[0047] The experimental results of the present invention on the public dataset VisDrone show that it performs well in detecting drone aerial images, surpassing the original YOLO11n model in terms of precision, recall and mAP50, and can meet the needs of drone perspective target detection tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is the distribution diagram of each category of targets in the VisDrone dataset of the present invention;

[0049] Figure 2 It is a structural schematic diagram of the drone view target detection network model based on improved YOLO11n of the present invention;

[0050] Figure 3It is a schematic diagram of the structure of the C3PA2 module of the present invention;

[0051] Figure 4 It is a structural schematic diagram of the LK_AB module of the present invention;

[0052] Figure 5 It is a schematic diagram of the structure of the SLK_Atten module of the present invention;

[0053] Figure 6 A flowchart of the FIFFN framework of the present invention processing an input feature map;

[0054] Figure 7(a)-Figure 7(d) All of them are partial detection result diagrams of the present invention on the test set. DETAILED DESCRIPTION

[0055] The specific contents of the present invention are further explained in detail below in conjunction with embodiments.

[0056] A method for detecting a target from a drone's perspective based on improved YOLO11n includes the following steps:

[0057] Step 1: preprocess the drone aerial image dataset and divide the preprocessed drone aerial image dataset into a training set, a validation set and a test set;

[0058] The drone aerial image dataset is the public VisDrone dataset, such as Figure 1 As shown in the figure, there are 10 data categories, namely: pedestrian, people, bicycle, car, van, truck, tricycle, awning-tricycle, bus and motorcycle. First, convert the format of the label file in the dataset from .xml to .txt, and then divide it into training set, validation set and test set;

[0059] Step 2: Build Figure 2 The drone view target detection model based on improved YOLO11n is shown in the figure. The specific process is as follows:

[0060] Step 2.1, replace the Bottleneck module in the C2f module of the original YOLO11n model with C3PA or LK-MBAM, and get Figure 3 The C3PA2 module shown in the figure is then used to replace the C3k2 module in the original YOLO11n model;

[0061] Step 2.1.1, when the parameter c3PA of the C3PA2 module is True, the component of C3PA2 is C3PA, such as Figure 3 As shown in Figure 2, the processing process of the C3PA2 module on the input feature map is:

[0062] Step 2.1.1.1, first transform the input feature map through the first 1×1 convolutional layer to generate an intermediate feature map to enhance the feature expression ability of the model, and then split the intermediate feature map into two parts, one part is directly passed to the Concat module to retain the original feature information; the other part is passed to the C3PA module to extract more global features;

[0063] like Figure 3 As shown in the figure, the process of C3PA module processing the input feature map is as follows: after the input feature map is evenly divided in the channel dimension, it is input into the left branch and the right branch respectively. The left branch contains a 1×1 convolution layer, and the right branch contains a 1×1 convolution layer and LK_PAB module. After the feature maps output by the left branch and the right branch are spliced ​​through the Concat module, 1×1 convolution is used for feature fusion to obtain the output feature map of the C3PA module;

[0064] like Figure 3 As shown, the process of the LK_PAB module processing the input feature map is as follows: the feature map input to the LK_PAB module is evenly divided in the channel dimension, and then respectively input into the left branch containing the LK_AB module and the right branch containing the LK_MBAM module, and the feature maps output by the left branch containing the LK_AB module and the right branch containing the LK_MBAM module are spliced ​​through the Concat module, and 1×1 convolution is used for feature fusion to obtain the output feature map of the LK_PAB module;

[0065] like Figure 4 As shown, the process of the LK_AB module processing the input feature map is as follows: the feature map of the input LK_AB module passes through the LayerNorm2d layer, the LKASA module and the DropPath layer in sequence to obtain an intermediate feature map, the intermediate feature map is added to the feature map of the input LK_AB module to obtain a feature map F1, the feature map F1 passes through the LayerNorm2d layer, the CGLU layer and the DropPath layer in sequence to the feature map F2, the feature map F2 is added to the intermediate feature map to obtain the output feature map of the LK_AB module;

[0066] like Figure 4As shown, the process of the LKASA module processing the input feature map is as follows: using a 1×1 convolution module to triple the number of channels of the feature map input to the LKASA module to obtain a feature map G1, dividing the feature map G1 into three parts in the channel dimension and assigning them to Q, K and V respectively, and adding the feature maps represented by Q and K after passing through the SLK-Atten module to obtain the added feature map, and multiplying the added feature map with the feature map represented by V to obtain a feature map processed by the additive self-attention mechanism, that is, the output feature map of the LKASA module;

[0067] The process of the LKASA module processing the input feature map is expressed by the formula:

[0068] LKASA(Q,K,V)=γ(ω(K)+ω(Q))·V

[0069] Where Q, K and V are obtained through independent linear transformations, γ(·) represents the deep convolution of fused features, and ω(·) represents the processing of the sparse large kernel attention mechanism SLK-Atten module;

[0070] like Figure 5 As shown in the figure, the process of the SLK-Atten module processing the input feature map is as follows: the feature map of the input SLK-Atten module is subjected to feature sparseness through the AvgPool2d layer to obtain the feature map I′, the feature map I′ is sequentially subjected to cascaded 1×3 depth convolution, 3×1 depth convolution, 1×5 expansion depth convolution, 5×1 expansion depth convolution and 1×1 depth convolution to obtain the feature map I″, the feature map I″ is subjected to the Hardswish activation function to obtain the weight feature map with the attention mechanism, the weight feature map with the attention mechanism is multiplied with the input feature map I of the SLK-Atten module to obtain the output feature map, the output feature map is residually connected with the feature map I′ to prevent the gradient from disappearing, and the output feature map I of the SLK-Atten module is obtained. out ;

[0071] The process of the SLK-Atten module processing the input feature map is expressed as follows:

[0072] I′=Avg(I)

[0073] I″=DConv 5×1 (DConv 1×5 (DWConv 3×1 (DWConv 1×3 (I′)))

[0074] I out =Hardswish(Conv 1×1 (I″))·I+I′

[0075]

[0076] Where: I represents the input feature map of the SLK-Atten module; I′ represents the feature map after AvgPool2d feature sparseness; I″ represents the feature map after cascading 1×3 depth convolution, 3×1 depth convolution, 1×5 expansion depth convolution, 5×1 expansion depth convolution and 1×1 depth convolution; Avg represents 3×3 average pooling; DWConv represents depthwise separable convolution; DConv represents depthwise separable dilated convolution; Hardswish(·) represents the activated feature map; t represents the pixel value; I out Represents the output feature map of the SLK-Atten module;

[0077] The cascaded 1×3 depth convolution, 3×1 depth convolution, 1×5 dilated depth convolution, 5×1 dilated depth convolution, and 1×1 depth convolution in the SLK-Atten module of this embodiment are equivalent to the Large-Separable-Kernel-Attention module in the prior art;

[0078] like Figure 3 As shown in the figure, the process of the LK_MBAM module processing the input feature map is as follows: after the feature map of the input LK_MBAM module is converted by a 3×3 convolution module, the feature map after feature conversion is divided into two parts in the channel dimension, one part is input into the identity mapping module of the left branch, and the other part is sequentially passed through the cascaded 5×5 depth convolution module and the 7×7 depth convolution module in the right branch, and then the feature map output by the left branch and the feature map output by the right branch are spliced ​​by the concat module, and then 1×1 convolution is used for feature conversion, and then the key features are focused on by the SLK-Atten module to suppress irrelevant noise, and the output feature map of the LK_MBAM module is obtained;

[0079] In step 2.1.1.2 and step 2.1.1.1, the intermediate feature map directly transferred to the Concat module and the feature map output by the C3PA module are concatenated through the Concat module, and 1×1 convolution is used for feature fusion to enhance the feature expression capability of the C3PA2 module, thereby obtaining the output feature map of the C3PA2 module.

[0080] Step 2.1.2: When the parameter c3PA of the C3PA2 module is False, the component of C3PA2 is LK-MBAM, such as Figure 3 As shown in Figure 2, the processing process of the C3PA2 module on the input feature map is:

[0081] Step 2.1.2.1, first transform the input feature map through the first 1×1 convolution layer to generate an intermediate feature map to enhance the feature expression ability of the model, and then split the intermediate feature map into two parts, one part is directly passed to the Concat module to retain the original feature information; the other part is passed to the LK-MBAM module to extract more detailed features;

[0082] Step 2.1.2.2, after the intermediate feature map directly passed to the Concat module is concatenated with the feature map output by the LK-MBAM module through the Concat module, 1×1 convolution is used for feature fusion to enhance the feature expression capability of the C3PA2 module, and the output feature map of the C3PA2 module is obtained;

[0083] Step 2.2: Figure 2 As shown, the FIFFN framework is used to replace the neck network in the original YOLO11n model. Figure 2 The inner part of the red box is the structure of the FIFFN framework, such as Figure 2 and Figure 6 As shown in the figure, the processing process of the FIFFN framework on the input feature map is:

[0084] Step 2.2.1, first use 1×1 convolution to perform feature conversion on the feature map K1 output by the backbone network, and then use the C3PA2 module to perform feature conversion on the 1×1 convolution output feature map, so that the subsequent upsampled feature map can better match the semantic information of the shallow feature map, and then use a 1×1 convolution to fuse the information to obtain the intermediate feature map;

[0085] Step 2.2.2, first use the nearest neighbor upsampling to decode the intermediate feature map to obtain a feature map with a resolution of 40×40, then use 1×1 convolution to perform feature conversion on the feature map K2 output by the backbone network, and then concatenate the feature map after 1×1 convolution feature conversion and the feature map with a resolution of 40×40 obtained by the decoding operation to obtain a fused feature map, and then use the C3PA2 module and the 1×1 convolution module in turn to perform feature extraction and feature conversion operations on the fused feature map to obtain feature map K5, and finally use the nearest neighbor upsampling to decode the feature map K5 to obtain a feature map with a resolution of 80×80;

[0086] Step 2.2.3, first use 1×1 convolution to perform feature conversion on the feature map K3 output by the backbone network, then concatenate the feature map after feature conversion with the feature map with a resolution of 80×80 to obtain a fused feature map, then use the feature map fused by the C3PA2 module to extract features to obtain feature map K6, and then use the nearest neighbor upsampling to decode the feature map K6 to obtain a feature map with a resolution of 160×160;

[0087] Step 2.2.4, first use 1×1 convolution to perform feature conversion on the feature map K4 output by the backbone network, then concatenate the feature map after feature conversion with the feature map with a resolution of 160×160 to obtain a fused feature map, and then use the C3PA2 module to extract features from the fused feature map to obtain a decoded output feature map;

[0088] Step 2.2.5, pass the decoded output feature map through the convolution layer with a convolution kernel size of 3×3 and a stride of 2, the BN layer and the SiLU layer in turn, and perform downsampling operations to obtain feature map K7, then concatenate feature maps K3, K6 and K7 to obtain a fused feature map, then use the C3PA2 module to extract features from the fused feature map, and then pass through the convolution layer with a convolution kernel size of 3×3 and a stride of 2, the BN layer and the SiLU layer in turn for downsampling operations to obtain feature map K8;

[0089] Step 2.2.6, concatenate the feature maps K2, K5, and K8 to obtain a fused feature map, and then use the C3PA2 module to extract features from the fused feature map to obtain an output feature map with the minimum resolution;

[0090] In order to further narrow the semantic gap between shallow knowledge and deep knowledge, the frequent interaction feature fusion network FIFFN of this embodiment does not directly splice the feature map of the shallow output with the feature map of the corresponding size in the fusion network, but first uses the 1×1 convolution module and the C3PA2 module to perform feature conversion on each output feature map of the backbone network to enhance the expression ability of the feature map, and then uses the feature map converted by the C3PA2 module to perform feature conversion, so that the subsequent upsampled feature map can better match the semantic information of the shallow feature map. Finally, in order to enhance the robustness of the fusion network, the shallow feature map from the backbone network and the deep feature map of the same size in the upsampling stage and the downsampling stage of the fusion network are spliced ​​together by introducing jump connections, thereby greatly enriching the information of each level of the fusion network. Finally, the prediction feature map with a resolution of 20×20 used in the neck network of the original YOLO11n model is replaced with a prediction feature map with a resolution of 160×160 to solve the problem of insufficient detail information in the decoding process;

[0091] Step 3: first set the parameters, then use the training set to train the drone view target detection model based on the improved YOLO11n, and use the validation set to tune the hyperparameters of the drone view target detection model based on the improved YOLO11n during the training process to obtain the optimal drone view target detection model based on the improved YOLO11n;

[0092] The process of setting parameters is as follows: batch-size is set to 16, initial learning rate is set to 0.001, weight decay parameter is set to 0.0005, momentum factor is set to 0.937; Adam optimizer is used, the number of training rounds is set to 200 rounds, and the learning rate decay strategy uses cosine annealing algorithm;

[0093] During the training process, the best performing model weights are saved as the file best.pt, which is loaded into the improved YOLO11n drone view target detection model to obtain the optimal drone view target detection model based on improved YOLO11n for the drone view target detection task;

[0094] Step 4: Input the drone aerial image to be detected into the optimal drone view target detection model based on improved YOLO11n, and output the detection result. The specific process is as follows:

[0095] Input the drone aerial image to be detected into the drone perspective target detection model based on the improved YOLO11n, load the best model weights saved in step 3 and save it as the file best.pt, then use the optimal drone perspective target detection model based on the improved YOLO11n to detect the drone aerial image to be detected, and output the following: Figure 7(a) to Figure 7(d) The image shown in the figure is annotated with the detected object category and its corresponding confidence, which is the final detection result. Figure 7(a) to Figure 7(d) It can be seen that the drone perspective target detection model based on the improved YOLO11n can accurately detect target images of various categories.

[0096] In order to verify the detection accuracy of the drone view target detection model based on the improved YOLO11n proposed in this embodiment, the test set is input into the original YOLO11n model and the drone view target detection model based on the improved YOLO11n, and Precision, Recall and mAP are used as key indicators to evaluate the detection results. The results are shown in Table 2. The calculation formulas of Precision, Recall and mAP are expressed as follows:

[0097]

[0098] Where TP means that the network model correctly detects the annotated object, FP means that the network model incorrectly predicts the background area as the annotated object, FN means that the network model incorrectly predicts the annotated object as the background area, N means the number of categories, and mAP means the average precision of N categories.

[0099] The experimental environment parameters of this embodiment are shown in Table 1;

[0100] Table 1 Experimental environment parameters

[0101]

[0102] Table 2 Detection results of different algorithms on the test set

[0103]

[0104] It can be seen from Table 2 that compared with the original YOLO11n model, the mAP of the detection results of the drone view target detection model based on the improved YOLO11n is 50 The value increased by 9%, the recall rate of the model increased by 7.3%, and the precision rate increased by 8.3%, which effectively prevented missed detections and false detections, and significantly improved the detection accuracy.

Claims

1. A drone view target detection method based on improved YOLO11n, characterized in that: The steps include: Step 1: preprocess the drone aerial image dataset and divide the preprocessed drone aerial image dataset into a training set, a validation set and a test set; Step 2: Build a drone view target detection model based on improved YOLO11n. The specific process is as follows: Step 2.1, replace the Bottleneck module in the original YOLO11n model C2f module with C3PA or LK-MBAM to obtain the C3PA2 module. When the parameter c3PA of the C3PA2 module is True, the component of the C3PA2 module is C3PA. When the parameter c3PA of the C3PA2 module is False, the component of the C3PA2 module is LK-MBAM. Then use the C3PA2 module to replace the C3k2 module in the original YOLO11n model backbone network. The C3PA module divides the input feature map equally in the channel dimension and inputs it into the left branch and the right branch respectively. The left branch contains a 1×1 convolution layer, and the right branch contains a 1×1 convolution layer and the LK_PAB module. The feature maps output by the left branch and the right branch are concatenated by the Concat module, and feature fusion is performed using 1×1 convolution to obtain the output feature map of the C3PA module. The LK_PAB module divides the input feature map equally in the channel dimension, and then inputs the feature maps into the left branch of the LK_AB module and the right branch of the LK_MBAM module respectively. The feature maps output by the LK_AB module and the LK_MBAM module are concatenated by the Concat module, and 1×1 convolution is used for feature fusion to obtain the output feature map of the LK_PAB module. The LK_AB module passes through the LayerNorm2d layer, the LKASA module and the DropPath layer in sequence to obtain an intermediate feature map. After the intermediate feature map is added to the feature map of the input LK_AB module, a feature map F1 is obtained. The feature map F1 passes through the LayerNorm2d layer, the CGLU layer and the DropPath layer in sequence to the feature map F2. The feature map F2 is added to the intermediate feature map to obtain the output feature map of the LK_AB module; The LKASA module uses a 1×1 convolution module to triple the number of channels of the input feature map to obtain a feature map G1, and divides the feature map G1 into three parts in the channel dimension and assigns them to Q, K and V respectively. The feature maps corresponding to Q and K are respectively passed through the SLK-Atten module and then added to obtain the added feature map. The added feature map is multiplied with the feature map corresponding to V to obtain the output feature map of the LKASA module; The SLK-Atten module performs feature sparseness on the input feature map through the AvgPool2d layer to obtain the feature map I ′ , feature map I ′ After cascading 1×3 depth convolution, 3×1 depth convolution, 1×5 expansion depth convolution, 5×1 expansion depth convolution and 1×1 depth convolution, the feature map I is obtained. ″ , feature map I ″ After the Hardswish activation function, we get the weighted feature map with the attention mechanism, which is multiplied by the input feature map I of the SLK-Atten module to get the output feature map. ′ Residual connection, get the output feature map I of SLK-Atten module out ; After the LK_MBAM module performs feature conversion on the input feature map through a 3×3 convolution module, the feature map after feature conversion is divided into two parts in the channel dimension, one part is input into the identity mapping module, and the other part passes through the cascaded 5×5 depth convolution module and the 7×7 depth convolution module in sequence, and then the feature map output by the identity mapping module and the feature map output by the 7×7 depth convolution module are spliced ​​through the concat module, and then 1×1 convolution is used for feature conversion, and then the key features are focused through the SLK-Atten module to obtain the output feature map of the LK_MBAM module; Step 2.2, use the FIFFN framework to replace the neck network in the original YOLO11n model. The FIFFN framework first performs feature conversion on each feature map output from the backbone network through a 1×1 convolution module and a C3PA2 module, and then concatenates the deep feature maps of the upsampling and downsampling stages of the FIFFN framework with the shallow feature maps of the same size from the backbone network through a jump connection, thereby replacing the 20×20 prediction feature map used in the neck network of the original YOLO11n model with a 160×160 prediction feature map; Step 3: first set the parameters, then use the training set to train the drone view target detection model based on the improved YOLO11n, and use the validation set to tune the hyperparameters of the drone view target detection model based on the improved YOLO11n during the training process to obtain the optimal drone view target detection model based on the improved YOLO11n; Step 4: Input the drone aerial image to be detected into the optimal drone perspective target detection model based on improved YOLO11n, and output the detection result.

2. The method for detecting target from unmanned aerial vehicle perspective based on improved YOLO11n according to claim 1, characterized in that: The drone aerial image dataset in step 1 is the public VisDrone dataset, which includes 10 data categories, namely: pedestrians, people, bicycles, cars, vans, trucks, tricycles, sunshade tricycles, buses and motorcycles.

3. The method for detecting target from unmanned aerial vehicle perspective based on improved YOLO11n according to claim 1, characterized in that: The processing process of the C3PA2 module on the input feature map in step 2.1 includes the following steps: Step 2.1.1: When the parameter c3PA of the C3PA2 module is True, the component of C3PA2 is C3PA, and the processing process of C3PA2 on the input feature map is: Step 2.1.1.1, first transform the input feature map through the first 1×1 convolutional layer to generate an intermediate feature map, and then split the intermediate feature map into two parts, one part is directly passed to the Concat module, and the other part is passed to the C3PA module; Step 2.1.1.2, concatenate the intermediate feature map directly passed to the Concat module and the feature map output by the C3PA module through the Concat module, and use 1×1 convolution for feature fusion to obtain the output feature map of the C3PA2 module; Step 2.1.2: When the parameter c3PA of the C3PA2 module is False, the component of C3PA2 is LK-MBAM, and the processing process of C3PA2 on the input feature map is: Step 2.1.2.1, first transform the input feature map through the first 1×1 convolution layer to generate an intermediate feature map, and then split the intermediate feature map into two parts, one part is directly passed to the Concat module, and the other part is passed to the LK-MBAM module; Step 2.1.2.2: After the intermediate feature map directly passed to the Concat module is concatenated with the feature map output by the LK-MBAM module through the Concat module, 1×1 convolution is used for feature fusion to obtain the output feature map of the C3PA2 module.

4. The method for detecting target from unmanned aerial vehicle perspective based on improved YOLO11n according to claim 1, characterized in that: The process of LKASA module processing the input feature map in step 2.1 is expressed by the formula: LKASA(Q,K,V)=γ(ω(K)+ω(Q))·V Where Q, K and V are obtained through independent linear transformations, γ(·) represents the deep convolution of fused features, and ω(·) represents the processing of the SLK-Atten module.

5. The method for detecting target from unmanned aerial vehicle perspective based on improved YOLO11n according to claim 1, characterized in that: The process of the SLK-Atten module processing the input features in step 2.1 is expressed by the formula: IN ′ =Avg(I) I ″ =DConv 5×1 (DConv 1×5 (DWConv 3×1 (DWConv 1×3 (I ′ ))) I out =Hardswish(Conv 1×1 (I ″ ))·I+I ′ Where: I represents the input feature map of the SLK-Atten module; I′ represents the feature map after AvgPool2d feature sparseness; I″ represents the feature map after cascading 1×3 depth convolution, 3×1 depth convolution, 1×5 dilated depth convolution, 5×1 dilated depth convolution and 1×1 depth convolution; Avg represents 3×3 average pooling; DWconv represents depth convolution; DConv represents depth dilated convolution; Hardswish(·) represents the activated feature map; t represents the pixel value; I out Represents the output feature map of the SLK-Atten module.

6. The method for detecting target from unmanned aerial vehicle perspective based on improved YOLO11n according to claim 1, characterized in that: The processing process of the input feature map by the FIFFN framework in step 2.2 is: Step 2.2.1, first use 1×1 convolution to perform feature conversion on the feature map K1 output by the backbone network, then use the C3PA2 module to perform feature conversion on the 1×1 convolution output feature map, and then use a 1×1 convolution to perform information fusion to obtain the intermediate feature map; Step 2.2.2, first use the nearest neighbor upsampling to decode the intermediate feature map to obtain a feature map with a resolution of 40×40, then use 1×1 convolution to perform feature conversion on the feature map K2 output by the backbone network, then concatenate the feature map output by the 1×1 convolution with the feature map with a resolution of 40×40 obtained by the decoding operation to obtain a fused feature map, then use the C3PA2 module and the 1×1 convolution module in turn to perform feature extraction and feature conversion operations on the fused feature map to obtain feature map K5, finally use the nearest neighbor upsampling to decode the feature map K5 to obtain a feature map with a resolution of 80×80; Step 2.2.3, first use 1×1 convolution to perform feature conversion on the feature map K3 output by the backbone network, then concatenate the feature map output by 1×1 convolution with the feature map with a resolution of 80×80 to obtain a fused feature map, then use the C3PA2 module to extract features from the fused feature map to obtain feature map K6, and then use the nearest neighbor upsampling to decode the feature map K6 to obtain a feature map with a resolution of 160×160; Step 2.2.4, first use 1×1 convolution to perform feature conversion on the feature map K4 output by the backbone network, then concatenate the feature map output by 1×1 convolution with the feature map with a resolution of 160×160 to obtain a fused feature map, and then use the C3PA2 module to extract features from the fused feature map to obtain a decoded output feature map; Step 2.2.5, pass the decoded output feature map through the convolution layer with a convolution kernel size of 3×3 and a stride of 2, the BN layer and the SiLU layer in turn, and perform downsampling operations to obtain feature map K7, then concatenate feature maps K3, K6 and K7 to obtain a fused feature map, then use the C3PA2 module to extract features from the fused feature map, and then pass through the convolution layer with a convolution kernel size of 3×3 and a stride of 2, the BN layer and the SiLU layer in turn for downsampling operations to obtain feature map K8; Step 2.2.6: Concatenate the feature maps K2, K5, and K8 to obtain a fused feature map, and then use the C3PA2 module to extract features from the fused feature map to obtain a predicted feature map with the smallest resolution.

7. The method for detecting target from unmanned aerial vehicle perspective based on improved YOLO11n according to claim 1, characterized in that: The process of setting parameters in step 3 is as follows: batch-size is set to 16, initial learning rate is set to 0.001, weight decay parameter is set to 0.0005, momentum factor is set to 0.937; Adam optimizer is used, the number of training rounds is set to 200 rounds, and the learning rate decay strategy uses cosine annealing algorithm.

Images