Unmanned aerial vehicle aerial photography small target detection method based on dynamic multi-scale optimization

Abstract

The invention discloses an unmanned aerial vehicle aerial photography small target detection method based on dynamic multi-scale optimization, and the method comprises the steps: introducing an anchor frame adaptive to the size of a small target, a space-depth convolution layer in an SPD-Conv module, a tiny target detection layer, a C3TR module, a CARAFE module, a GSConv module and a RepC3 module into a YOLOv5s model, and improving the model through adding cross-level connection to obtain an improved YOLOv5s model; according to the method, the improved YOLOv5s model has the characteristics of stronger small target feature capture capability, more efficient feature fusion mechanism and less calculation cost, then the improved YOLOv5s model is trained to obtain the trained improved YOLOv5s model, and when the unmanned aerial vehicle aerial small target is detected, the unmanned aerial vehicle aerial image is input into the trained improved YOLOv5s model, so that the detection accuracy of the unmanned aerial vehicle aerial small target is improved. The trained improved YOLOv5s model outputs a detection result; the method has the advantages that lost high-frequency feature information can be fully mined and reconstructed, feature information of different sizes can be fully and efficiently fused, the method adapts to the situation that the size of a small target changes greatly in the aerial photography scene of the unmanned aerial vehicle, and the detection precision is high.

Description

Technical Field

[0001] The present invention relates to a method for detecting small targets in drone aerial photography, and in particular to a method for detecting small targets in drone aerial photography based on dynamic multi-scale optimization. Background Art

[0002] As a fundamental task in the field of computer vision, target detection technology continues to drive innovation and development in areas such as intelligent video surveillance and autonomous driving. Drones, with their advantages of high maneuverability and low deployment costs, have demonstrated unique application value in precision agriculture, urban monitoring, and emergency rescue missions. However, due to limitations in the physical properties of sensors and the altitude of aerial photography, targets in drone images often exhibit sub-pixel microscopic morphology, and face problems such as large scale differences among similar targets and complex background occlusion. Although existing target detection methods can perform well in medium and large target detection tasks, the detection of small targets (where the ratio of the width and height of the target bounding box to the width and height of the original image is less than 0.1) in complex environments still suffers from deficiencies such as missed detections, false detections, and low detection rates.

[0003] In the task of detecting small targets in drone aerial photography, due to the small size and dense distribution of targets, problems such as false detection and missed detection are prone to occur, resulting in low detection accuracy.

[0004] The existing method for detecting small targets in drone aerial photography is based on an improved YOLOv5s model, which is then used to implement detection. The improved YOLOv5s model is derived from the following improvements to the YOLOv5s model: 1. All standard convolutions in the C3 modules in the original YOLOv5s model's backbone network are replaced with deformable convolutions, and a fused CBAM attention mechanism is added after the backbone network to achieve adaptive feature enhancement. 2. A small target detection layer is added between layers 17 and 18 of the original YOLOv5s model, consisting of a CBS module, an UpSample module, a Concat module, a C3 module, a CBS module, a Concat module, and a C3 module. This layer increases target detection for feature maps with a resolution of 1 / 4 the input drone aerial image, capturing more detailed information about small targets.

[0005] During detection, the original image is input into the improved YOLOv5s model. The backbone network of the improved YOLOv5s model extracts the multi-scale features of the image and then enhances the features by integrating the CBAM attention mechanism. The feature map after the enhanced features is input into the Neck network. The four detection layers of the Neck network respectively extract features from the feature map after the enhanced features, and the corresponding feature maps are output to the detection head. The detection head detects and classifies the target by combining the adaptive anchor frame group of the YOLOv5s model and the tiny anchor frame added based on the K-means algorithm. At the same time, the Focal-EIoU loss function is applied to accelerate convergence and improve detection accuracy, thereby realizing the detection of small targets in UAV aerial photography.

[0006] However, since small targets occupy fewer pixels, the existing UAV aerial photography small target detection method is prone to degradation or loss of edge information after continuous downsampling, and the improved C3 module can only enhance compensation information and restore some detail information. It is difficult to fully mine and reconstruct the lost high-frequency feature information. In scenes with dense small targets, false detection is prone to occur, resulting in low detection accuracy. Moreover, the feature fusion uses fixed and shared weights, which leads to semantic conflict problems and makes it difficult to fully and efficiently fuse feature information of different sizes. In addition, some parameters of the Focal-EIoU loss function are fixed, which makes it difficult to adapt to the large size changes of small targets in UAV aerial photography scenes. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide a small target detection method for drone aerial photography based on dynamic multi-scale optimization, which can fully mine and reconstruct lost high-frequency feature information, fully and efficiently integrate feature information of different sizes, and adapt to the situation where the size of small targets in drone aerial photography scenarios varies greatly, with high detection accuracy.

[0008] The technical solution adopted by the present invention to solve the above technical problems is: a method for detecting small targets in drone aerial photography based on dynamic multi-scale optimization, first improving the YOLOv5s model to obtain an improved YOLOv5s model, then training the improved YOLOv5s model to obtain a trained improved YOLOv5s model, when detecting small targets in drone aerial photography, inputting the drone aerial image into the trained improved YOLOv5s model, and the trained improved YOLOv5s model outputs the detection result, by introducing an anchor frame adapted to the small target size, a spatial-depth convolution layer in the SPD-Conv module, a small target detection layer, a C3TR module, a CARAFE module, a GSConv module and a RepC3 module in the YOLOv5s model, and adding cross-layer connections to improve it to obtain the improved YOLOv5s model, so that the improved YOLOv5s model has stronger small target feature capture capability, more efficient feature fusion mechanism and lower computational cost, thereby comprehensively improving the detection accuracy of small targets in drone aerial photography.

[0009] Compared with the prior art, the advantages of the present invention are:

[0010] 1. By introducing the spatial-depth convolution layer (SPD layer) in the SPD-Conv module into the original YOLOv5s model, the improved YOLOv5s model can convert the spatial dimension information of the feature map into the depth dimension during the downsampling process, retaining the detailed texture of the feature map and avoiding the loss of high-frequency information of small targets. The introduced C3TR module captures the long-range dependencies of the image through global attention, realizes the comprehensive coordination of contextual information, improves the improved YOLOv5s model's ability to perceive small targets, and enables the improved YOLOv5s model to extract richer fine-grained information of small target features.

[0011] 2. By introducing the CARAFE module, GSConv module, and RepC3 module, the upsampling process can retain richer edge feature information, reduce the number of model parameters and computational cost, and optimize the feature map transmission path, thereby enhancing the feature expression capability of the improved YOLOv5s model. 3. By adding a small target detection layer, the feature map with a resolution of 1 / 8 of the original UAV aerial image can be increased to 1 / 4 through upsampling, outputting a higher-resolution feature map. A new set of anchor frames adapted to small target sizes can more accurately cover the size distribution of small targets, achieve effective coverage of multi-scale targets, and enhance the improved YOLOv5s model's ability to capture the features of small targets.

[0012] 4. By adding cross-layer connections, cross-layer fusion of feature maps is achieved, which enhances the feature fusion capability of the improved YOLOv5s model. While ensuring the feature expression capability, it reduces the computational cost and ensures that the improved YOLOv5s model is suitable for UAV aerial photography scenarios, achieving higher detection accuracy and improving the detection accuracy of the improved YOLOv5s model.

[0013] Therefore, the present invention improves the YOLOv5s model to obtain an improved YOLOv5s model for small target detection in drone aerial photography. The improved YOLOv5s model can fully mine and reconstruct the lost high-frequency feature information, fully and efficiently integrate feature information of different sizes (i.e., multi-scale features), and adapt to the situation where the size of small targets in drone aerial photography scenes varies greatly, thereby significantly improving the detection accuracy of small targets in drone aerial photography and having high detection accuracy.

[0014] Furthermore, the YOLOv5s model includes a total of 25 layers of network structure, and the 25 layers of network structure are connected in sequence from front to back, and the 6th layer is also connected to the 12th layer, the 4th layer is also connected to the 16th layer, the 14th layer is also connected to the 19th layer, and the 10th layer is also connected to the 22nd layer; the 17th layer and the 20th layer are connected to the 24th layer respectively. The 25-layer network structure of the YOLOv5s model is called its 0th to 24th layers in the order of forward propagation from front to back, and the 0th layer is the CBS module, the 1st layer is the CBS module, the 2nd layer is the C3 module, the 3rd layer is the CBS module, the 4th layer is the C3 module, the 5th layer is the CBS module, the 6th layer is the C3 module, the 7th layer is the CBS module, the 8th layer is the C3 module, and the 9th layer is the CBS module. The 10th layer is the SPPF module, the 10th layer is the CBS module, the 11th layer is the UpSample module, the 12th layer is the Concat module, the 13th layer is the C3 module, the 14th layer is the CBS module, the 15th layer is the UpSample module, the 16th layer is the Concat module, the 17th layer is the C3 module, the 18th layer is the CBS module, the 19th layer is the Concat module, the 20th layer is the C3 module, the 21st layer is the CBS module, the 22nd layer is the Concat module, the 23rd layer is the C3 module, and the 24th layer is the Detect module; layers 0 to 9 constitute the Backbone network, layers 10 to 23 constitute the Neck network, and the 24th layer is the Head network.

[0015] Furthermore, the YOLOv5s model is improved in the following four aspects to obtain the improved YOLOv5s model:

[0016] First, add the spatial-depth convolution layer (SPD layer) in the SPD-Conv module after the 1st, 3rd, 5th, and 7th layers of the original YOLOv5s model to retain feature information, and use the C3TR module to replace the 8th layer of the original YOLOv5s model. At this time, the 1st and 2nd layers, 3rd and 4th layers, 5th and 6th layers, and 7th and 8th layers of the original YOLOv5s model are no longer directly connected, but are connected through an SPD layer in the SPD-Conv module respectively;

[0017] Second, the CARAFE module is used to replace the UpSample modules of the 11th and 15th layers of the original YOLOv5s model, the GSConv module is used to replace the CBS modules of the 10th, 14th, 18th and 21st layers of the original YOLOv5s model, and the RepC3 module is used to replace the C3 modules of the 20th and 23rd layers of the original YOLOv5s model;

[0018] 3. Add a small target detection layer between the 17th and 18th layers of the original YOLOv5s model. The small target detection layer is formed by connecting a GSConv module, a CARAFE module, a Concat module, a RepC3 module, a GSConv module, a Concat module and a RepC3 module in the forward propagation order. The front GSConv module in the small target detection layer is called its first GSConv module, the back GSConv module is called its second GSConv module, the front RepC3 module is called its first RepC3 module, the back RepC3 module is called its second RepC3 module, the front Concat module is called its first Concat module, and the back Concat module is called its In the second Concat module, the first GSConv module of the small target detection layer is also connected to its second Concat module. At this time, the 17th and 18th layers of the original YOLOv5s model are no longer directly connected. The 17th layer of the original YOLOv5s model is connected to the first GSConv module of the small target detection layer and is no longer connected to the 24th layer of the original YOLOv5s model. The 18th layer of the original YOLOv5s model is connected to the second RepC3 module of the small target detection layer. At the same time, the first Concat module of the small target detection layer is also connected to the second layer of the original YOLOv5s model; the first RepC3 module and the second RepC3 module of the small target detection layer are both connected to the 24th layer of the original YOLOv5s. At the same time, while retaining the three groups of adaptive anchor frames of the original YOLOv5s model, a group of anchor frames adapted to the small target size is added;

[0019] Fourth, add a connection between the 6th layer of the original YOLOv5s model and the 19th layer of the original YOLOv5s model, and add a connection between the 4th layer of the original YOLOv5s model and the second Concat module of the small object detection layer. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is the overall flow chart of the UAV aerial photography small target detection method based on dynamic multi-scale optimization of the present invention;

[0021] Figure 2 This is the structural diagram of the original YOLOv5s model;

[0022] Figure 3 This is a structural diagram of the improved YOLOv5s model in the UAV aerial photography small target detection method based on dynamic multi-scale optimization of the present invention. DETAILED DESCRIPTION

[0023] The present invention will be described in further detail below with reference to the accompanying drawings and embodiments.

[0024] Example 1: Figure 1 As shown in the figure, a small target detection method for UAV aerial photography based on dynamic multi-scale optimization is proposed. The YOLOv5s model is first improved to obtain an improved YOLOv5s model, and then the improved YOLOv5s model is trained to obtain a trained improved YOLOv5s model. When detecting small targets in UAV aerial photography, the UAV aerial image is input into the trained improved YOLOv5s model, and the trained improved YOLOv5s model outputs the detection result. The improved YOLOv5s model is obtained by introducing an anchor frame adapted to the small target size, a spatial-depth convolution layer in the SPD-Conv module, a small target detection layer, a C3TR module, a CARAFE module, a GSConv module and a RepC3 module in the YOLOv5s model, and adding cross-layer connections to improve it. The improved YOLOv5s model has stronger small target feature capture capability, more efficient feature fusion mechanism and lower computational cost, thereby comprehensively improving the detection accuracy of small targets in UAV aerial photography.

[0025] In this embodiment, the spatial-depth convolution layer (SPD layer) introduced in the SPD-Conv module enables the improved YOLOv5s model to convert the spatial dimension information of the feature map into the depth dimension during the downsampling process, retain the detailed texture of the feature map, and avoid the loss or loss of high-frequency information of small targets; the introduced C3TR module captures the long-distance dependency of the image through global attention, realizes the comprehensive coordination of contextual information, enhances the perception ability of the improved YOLOv5s model for small targets, and enables the improved YOLOv5s model to extract richer fine-grained information of small target features; the introduced small target detection layer can increase the resolution by upsampling operation. The feature map of the original UAV aerial photography image is increased from 1 / 8 to 1 / 4, and a higher resolution feature map is output; a new set of anchor frames adapted to the size of small targets can more accurately cover the size distribution of small targets, achieve effective coverage of multi-scale targets, and enhance the improved YOLOv5s model's ability to capture the features of small targets; adding cross-level connections realizes the cross-level fusion of feature maps, enhances the feature fusion capability of the improved YOLOv5s model, and while ensuring the feature expression capability, reduces the computational cost overhead, ensures that the improved YOLOv5s model is suitable for UAV aerial photography scenarios, achieves higher detection accuracy, and improves the detection accuracy of the improved YOLOv5s model.

[0026] Example 2: A method for detecting small targets in drone aerial photography based on dynamic multi-scale optimization, first improving the YOLOv5s model to obtain an improved YOLOv5s model, then training the improved YOLOv5s model to obtain a trained improved YOLOv5s model, when detecting small targets in drone aerial photography, inputting the drone aerial image into the trained improved YOLOv5s model, and the trained improved YOLOv5s model outputs the detection result; wherein, Figure 2As shown in the figure, the YOLOv5s model contains a total of 25 layers of network structure, and the 25 layers of network structure are connected from front to back, and the 6th layer is also connected to the 12th layer, the 4th layer is also connected to the 16th layer, the 14th layer is also connected to the 19th layer, and the 10th layer is also connected to the 22nd layer; the 17th layer and the 20th layer are connected to the 24th layer respectively. The 25-layer network structure of the YOLOv5s model is called its 0th to 24th layer in order from front to back according to data transmission. The 0th layer is the CBS module, the 1st layer is the CBS module, the 2nd layer is the C3 module, the 3rd layer is the CBS module, the 4th layer is the C3 module, the 5th layer is the CBS module, the 6th layer is the C3 module, the 7th layer is the CBS module, the 8th layer is the C3 module, the 9th layer is the SPPF module, and the 10th layer is the C BS module, the 11th layer is the UpSample module, the 12th layer is the Concat module, the 13th layer is the C3 module, the 14th layer is the CBS module, the 15th layer is the UpSample module, the 16th layer is the Concat module, the 17th layer is the C3 module, the 18th layer is the CBS module, the 19th layer is the Concat module, the 20th layer is the C3 module, the 21st layer is the CBS module, the 22nd layer is the Concat module, the 23rd layer is the C3 module, and the 24th layer is the Detect module; layers 0 to 9 constitute the Backbone network, layers 10 to 23 constitute the Neck network, and the 24th layer is the Head network; the YOLOv5s model is improved in the following four aspects Figure 3 The improved YOLOv5s model shown:

[0027] First, add the spatial-depth convolution layer (SPD layer) in the SPD-Conv module after the 1st, 3rd, 5th, and 7th layers of the original YOLOv5s model to retain feature information, and use the C3TR module to replace the 8th layer of the original YOLOv5s model. At this time, the 1st and 2nd layers, 3rd and 4th layers, 5th and 6th layers, and 7th and 8th layers of the original YOLOv5s model are no longer directly connected, but are connected through an SPD layer in the SPD-Conv module respectively;

[0028] Second, the CARAFE module is used to replace the UpSample modules of the 11th and 15th layers of the original YOLOv5s model, the GSConv module is used to replace the CBS modules of the 10th, 14th, 18th and 21st layers of the original YOLOv5s model, and the RepC3 module is used to replace the C3 modules of the 20th and 23rd layers of the original YOLOv5s model;

[0029] 3. Add a small target detection layer between the 17th and 18th layers of the original YOLOv5s model. The small target detection layer is formed by a GSConv module, a CARAFE module, a Concat module, a RepC3 module, a GSConv module, a Concat module and a RepC3 module in the forward propagation order. The front GSConv module in the small target detection layer is called its first GSConv module, the back GSConv module is called its second GSConv module, the front RepC3 module is called its first RepC3 module, the back RepC3 module is called its second RepC3 module, the front Concat module is called its first Concat module, and the back Concat module is called its second Concat module. The first GSConv module of the small target detection layer is also connected to its second Concat module. At this time, the 17th and 18th layers of the original YOLOv5s model are no longer directly connected. The 17th layer of the original YOLOv5s model and the first GSConv module of the small target detection layer are connected. The module is connected and is no longer connected to the 24th layer of the original YOLOv5s model. The 18th layer of the original YOLOv5s model is connected to the second RepC3 module of the small target detection layer. At the same time, the first Concat module of the small target detection layer is also connected to the 2nd layer of the original YOLOv5s model; the first RepC3 module and the second RepC3 module of the small target detection layer are both connected to the 24th layer of the original YOLOv5s. At the same time, while retaining the three groups of adaptive anchor frames of the original YOLOv5s model, a group of anchor frames adapted to the small target size is added; at this time, on the basis of improving the YOLOv5s In the model, the first and second RepC3 modules of the small target detection layer and the 20th and 23rd layers of the original YOLOv5s model are connected to the 24th layer of the original YOLOv5s respectively. The first and second RepC3 modules of the small target detection layer and the 20th and 23rd layers of the original YOLOv5s model use their respective output feature maps as the four input paths of the Detect module of the 24th layer of the original YOLOv5s model, corresponding to feature maps with input resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the UAV aerial image resolution, respectively.

[0030] Fourth, add a connection between the 6th layer of the original YOLOv5s model and the 19th layer of the original YOLOv5s model, and add a connection between the 4th layer of the original YOLOv5s model and the second Concat module of the small object detection layer to achieve cross-layer fusion of feature maps.

[0031] In this embodiment, the drone aerial image is input into the trained improved YOLOv5s model, and the specific process of the trained improved YOLOv5s model outputting the detection result is as follows:

[0032] A drone aerial image with a size of 640×640×3 (width×height×number of channels) is used as the input image and fed into the trained improved YOLOv5s model. The input image first enters the CBS module in layer 0. The CBS module in layer 0 uses a standard 6×6 convolution with a stride of 2 to downsample the input image in half and expand the channels. The resulting feature map is 320×320×64 and is output to the CBS module in layer 1.

[0033] The CBS module of the first layer uses a standard 3×3 convolution with a step size of 1 to perform channel expansion on the feature map of size 320×320×64 to obtain a feature map of size 320×320×128, which is output to the newly added SPD layer between the first and second layers. The SPD layer first extracts the feature information of all even rows and even columns of the feature map of size 320×320×128 to form a sub-feature map of size 160×160×128, the feature information of all odd rows and even columns to form a sub-feature map of size 160×160×128, the feature information of all even rows and odd columns to form a sub-feature map of size 160×160×128, and the feature information of all odd rows and odd columns to form a sub-feature map of size 160×160×128. Then, the four sub-feature maps are spliced ​​along the channel to form a feature map of size 160×160×512 and output to the C3 module of the second layer.

[0034] The C3 module of the second layer performs channel compression and extracts detail features on the feature map of size 160×160×512 to obtain a feature map of size 160×160×128, which is output to the CBS module of the third layer and the first Concat module in the newly added small target detection layer respectively; the CBS module of the third layer uses a standard 3×3 convolution with a step size of 1 to perform channel expansion on the feature map of size 160×160×128 to obtain a feature map of size 160×160×256, which is output to the newly added SPD layer between the third and fourth layers. Extract the feature information of all even rows and even columns of the feature map of size 160×160×256 to form a sub-feature map of size 80×80×256, the feature information of all odd rows and even columns to form a sub-feature map of size 80×80×256, the feature information of all even rows and odd columns to form a sub-feature map of size 80×80×256, and the feature information of all odd rows and odd columns to form a sub-feature map of size 80×80×256. Then, the four sub-feature maps are spliced ​​along the channel to form a feature map of size 80×80×1024 and output to the C3 module of the 4th layer;

[0035] The C3 module in the 4th layer performs channel compression and extracts detail features on the feature map of size 80×80×1024, obtaining a feature map of size 80×80×256, which is output to the CBS module in the 5th layer, the 16th layer, and the second Concat module in the newly added small object detection layer.

[0036] The CBS module of the 5th layer uses a standard 3×3 convolution with a step size of 1 to perform channel expansion on the feature map of size 80×80×256 to obtain a feature map of size 80×80×512, which is output to the newly added SPD layer between the 5th and 6th layers. The SPD layer first extracts the feature information of all even rows and even columns of the feature map of size 80×80×512 to form a sub-feature map of size 40×40×512, the feature information of all odd rows and even columns to form a sub-feature map of size 40×40×512, the feature information of all even rows and odd columns to form a sub-feature map of size 40×40×512, and the feature information of all odd rows and odd columns to form a sub-feature map of size 40×40×512. Then, the four sub-feature maps are spliced ​​along the channel to form a feature map of size 40×40×2048 and output to the C3 module of the 6th layer.

[0037] The C3 module in the 6th layer performs channel compression and extracts detail features on the feature map of size 40×40×2048, and obtains a feature map of size 40×40×512, which is output to the CBS module in the 7th layer, the Concat module in the 12th layer, and the Concat module in the 19th layer respectively;

[0038] The CBS module of the 7th layer uses a standard 3×3 convolution with a step size of 1 to perform channel expansion on the feature map of size 40×40×512 to obtain a feature map of size 40×40×1024, which is output to the newly added SPD layer between the 7th and 8th layers. The SPD layer first extracts the feature information of all even rows and even columns of the feature map of size 40×40×1024 to form a sub-feature map of size 20×20×1024, the feature information of all odd rows and even columns to form a sub-feature map of size 20×20×1024, the feature information of all even rows and odd columns to form a sub-feature map of size 20×20×1024, and the feature information of all odd rows and odd columns to form a sub-feature map of size 20×20×1024. Then, the four sub-feature maps are spliced ​​along the channel to form a feature map of size 20×20×4096 and output to the C3TR module of the 8th layer.

[0039] The C3TR module in the eighth layer performs channel compression and global self-attention on the feature map of size 20×20×4096 to capture the long-range dependencies in the image and refine the local feature information. The resulting feature map of size 20×20×1024 is output to the SPPF module in the ninth layer.

[0040] The SPPF module in the 9th layer performs a pooling operation on the feature map of size 20×20×1024 to capture the contextual information of different receptive fields, and obtains a feature map of size 20×20×1024, which is output to the GSConv module in the 10th layer;

[0041] The GSConv module of the 10th layer first uses a standard 1×1 convolution with a step size of 1 to compress the feature map of size 20×20×1024 to obtain the intermediate feature. Then, a 5×5 depth-wise convolution with a step size of 1 is used to independently extract spatial features in each channel of the intermediate feature to generate refined features. The intermediate features and refined features are then concatenated and rearranged in the channel dimension to obtain a feature map of size 20×20×512, which is output to the CARAFE module of the 11th layer and the Concat module of the 22nd layer respectively.

[0042] The CARAFE module in the 11th layer performs a content-aware dynamic upsampling strategy on the feature map of size 20×20×512, and outputs the feature map of size 40×40×512 to the Concat module in the 12th layer;

[0043] The Concat module of the 12th layer concatenates the feature map of size 40×40×512 output by the 11th layer and the feature map of size 40×40×512 output by the 6th layer in the channel dimension to obtain a feature map of size 40×40×1024, which is output to the C3 module of the 13th layer;

[0044] The C3 module in the 13th layer performs channel compression and extracts detail features on the feature map of size 40×40×1024, obtaining a feature map of size 40×40×512 and outputting it to the GSConv module in the 14th layer.

[0045] The GSConv module of the 14th layer first uses a standard 1×1 convolution with a step size of 1 to compress the feature map of size 40×40×512 to obtain the intermediate feature. Then, a 5×5 depth-wise convolution with a step size of 1 is used to extract spatial features independently in each channel of the intermediate feature to generate refined features. The intermediate features and refined features are then concatenated and rearranged in the channel dimension to obtain a feature map of size 40×40×256, which is output to the CARAFE module of the 15th layer and the Concat module of the 19th layer respectively.

[0046] The CARAFE module in the 15th layer performs a content-aware dynamic upsampling strategy on the feature map of size 40×40×256, and outputs the feature map of size 80×80×256 to the Concat module in the 16th layer;

[0047] The Concat module of the 16th layer concatenates the feature map of size 80×80×256 output by the 15th layer and the feature map of size 80×80×256 output by the 4th layer in the channel dimension, and obtains a feature map of size 80×80×512, which is output to the C3 module of the 17th layer.

[0048] The C3 module in the 17th layer performs channel compression and extracts detail features on the feature map of size 80×80×512, and obtains a feature map of size 80×80×256, which is output to the first GSConv module in the newly added small object detection layer.

[0049] The first GSConv module of the newly added small target detection layer first uses a standard 1×1 convolution with a step size of 1 to compress the feature map of size 80×80×256 to obtain the intermediate feature. Then, a 5×5 depth-wise convolution with a step size of 1 is used to independently extract spatial features in each channel of the intermediate feature to generate refined features. The intermediate features and refined features are then concatenated and rearranged in the channel dimension to obtain a feature map of size 80×80×128, which are output to the CERAFE module in the newly added small target detection layer and the second Concat module in the newly added small target detection layer respectively.

[0050] The CERAFE module in the newly added small object detection layer performs a content-aware dynamic upsampling strategy on the feature map of size 80×80×128, and outputs the feature map of size 160×160×128 to the first Concat module of the newly added small object detection layer;

[0051] The first Concat module of the newly added small target detection layer concatenates the 160×160×128 feature map output by the CERAFE module in the newly added small target detection layer with the 160×160×128 feature map output by the second layer in the channel dimension, and obtains a feature map of size 160×160×256, which is output to the first RepC3 module of the newly added small target detection layer.

[0052] The first RepC3 module of the newly added small target detection layer performs channel dimensionality reduction and optimizes the feature map transmission path on the feature map of size 160×160×256. The resulting feature map of size 160×160×128 is output to the second GSConv module of the newly added small target detection layer and the Detect module of the 24th layer respectively.

[0053] The second GSConv module of the newly added small target detection layer first uses a standard 3×3 convolution with a stride of 2 to compress the feature map of size 160×160×128 to obtain intermediate features. Then, a 5×5 depth-wise convolution with a stride of 1 is used to independently extract spatial features in each channel of the intermediate features to generate refined features. The intermediate features and refined features are then concatenated and rearranged in the channel dimension to obtain a feature map of size 80×80×128, which is output to the second Concat module of the newly added small target detection layer.

[0054] The second Concat module of the newly added small target detection layer performs channel-wise concatenation on the feature map of size 80×80×256 output by the 4th layer, the feature map of size 80×80×128 output by the first GSConv module of the newly added small target detection layer, and the feature map of size 80×80×128 output by the second GSConv module of the newly added small target detection layer. The obtained feature map of size 80×80×512 is output to the second RepC3 module of the newly added small target detection layer.

[0055] The second RepC3 module of the newly added small object detection layer performs channel dimensionality reduction and optimizes the feature map transmission path on the feature map of size 80×80×512, obtaining a feature map of size 80×80×256, which is output to the GSConv module of the 18th layer and the Detect module of the 24th layer respectively;

[0056] The GSConv module of the 18th layer first uses a standard 3×3 convolution with a stride of 2 to compress the feature map of size 80×80×256 to obtain intermediate features. It then uses a 5×5 depth-wise convolution with a stride of 1 to independently extract spatial features in each channel of the intermediate features to generate refined features. The intermediate features and refined features are then concatenated and rearranged in the channel dimension to obtain a feature map of size 40×40×256, which is output to the Concat module of the 19th layer.

[0057] The Concat module in the 19th layer concatenates the feature map of size 40×40×512 output by the 6th layer, the feature map of size 40×40×256 output by the 14th layer, and the feature map of size 40×40×256 output by the 18th layer in the channel dimension, and obtains a feature map of size 40×40×1024, which is output to the RepC3 module in the 20th layer.

[0058] The RepC3 module on the 20th layer performs channel dimensionality reduction and optimizes the feature map transmission path on the feature map of size 40×40×1024, obtaining a feature map of size 40×40×512, which is output to the GSConv module on the 21st layer and the Detect module on the 24th layer respectively;

[0059] The GSConv module of the 21st layer first uses a standard 3×3 convolution with a stride of 2 to compress the channel of the feature map of size 40×40×512 to obtain intermediate features. It then uses a 5×5 depth-wise convolution with a stride of 1 to independently extract spatial features in each channel of the intermediate features to generate refined features. The intermediate features and refined features are then concatenated and rearranged in the channel dimension to obtain a feature map of size 20×20×512, which is output to the Concat module of the 22nd layer.

[0060] The Concat module of the 22nd layer concatenates the features of the 10th layer (size 20×20×512) and the feature map of the 21st layer (size 20×20×512) in the channel dimension, and obtains a feature map of size 20×20×1024, which is output to the RepC3 module of the 23rd layer.

[0061] The RepC3 module in the 23rd layer optimizes the feature map transmission path for the feature map of size 20×20×1024, and outputs the feature map of size 20×20×1024 to the Detect module in the 24th layer;

[0062] The Detect module of the 24th layer receives the feature map of size 160×160×128 output by the first RepC3 module of the newly added small target detection layer, the feature map of size 80×80×256 output by the second RepC3 module of the newly added small target detection layer, the feature map of size 40×40×512 output by the 20th layer RepC3 module, and the feature map of size 20×20×1024 output by the 23rd layer RepC3 module. These feature maps are firstly used to perform anchor box matching, bounding box regression and category prediction using the preset four sets of anchor boxes to obtain feature map detection results with resolutions of 160×160, 80×80, 40×40 and 20×20. Subsequently, non-maximum suppression (NMS) is performed on the candidate boxes in the feature map detection results to obtain the final detection box and category.

[0063] In this embodiment, through the above-mentioned first improvement, the improved YOLOv5s model can convert the spatial dimension information of the feature map into the depth dimension during the downsampling process, retain the detailed texture of the feature map, and avoid the loss or loss of high-frequency information of small targets. The C3TR module that replaces the 8th layer of the original YOLOv5s model can capture the long-distance dependencies of the image through global attention, and realize the comprehensive coordination of contextual information, thereby improving the perception ability of the improved YOLOv5s model for small targets. The improved YOLOv5s model can extract richer fine-grained information of small target features; through the above-mentioned second improvement, the CARAFE module adaptively generates upsampling kernels through content perception, finely reconstructs the edges and textures of high-resolution features, and retains more feature edge information. The lightweight GSConv module fuses grouped convolution with depth-separable convolution to reduce computational cost while ensuring feature expression capability, thereby ensuring that the improved YOLOv5s model is suitable for UAV aviation. In the shooting scene, the RepC3 module combines multi-branch technology with residual structure to optimize the forward transmission path of the feature map and the inference speed of the improved YOLOv5s model, which can introduce lower computational cost to achieve higher detection accuracy, thereby improving the detection accuracy of the improved YOLOv5s model. Through the third improvement mentioned above, a small target detection layer is added between the 17th and 18th layers of the original YOLOv5s model, which can increase the resolution of the feature map with a resolution of 1 / 8 of the original drone aerial image to 1 / 4 through upsampling operation, and output a higher resolution feature map. On the basis of retaining the three sets of adaptive anchor frames of the original YOLOv5s model, a new set of anchor frames adapted to the size of small targets is added, which can more accurately cover the size distribution of small targets, achieve effective coverage of multi-scale targets, and enhance the feature capture capability of the improved YOLOv5s model for small targets. Through the fourth improvement mentioned above, two new paths are added to realize cross-layer fusion of feature maps, thereby enhancing the feature fusion capability of the improved YOLOv5s model.

[0064] To comprehensively evaluate the performance of the dynamic multi-scale optimization-based small target detection method for drone aerial photography, we conducted a comparative experiment on the VisDrone2019 dataset. Specifically, the other small target detection methods included eight models: YOLOv3, YOLOX, YOLOv5s, TPH-YOLOv5, YOLOv7-tiny, YOLOv8s, SOD-YOLO replicated on YOLOv5s, and HIC-YOLOv5. The experimental results are shown in Table 1.

[0065] Table 1 Comparison of experimental results

[0066] Model P / ％ R / ％ mAP@0.5 / % <![CDATA[Param / 10 6 ]]> FPS / fps YOLOv3 38.9 28.4 26.8 61.5 94.3 YOLOX 42.7 32.2 31.4 6.4 80.6 YOLOv5s 43.7 34.4 33.2 7.0 108.7 TPH-YOLOv5 47.4 34.5 35.4 7.5 66.7 YOLOv7-tiny 48.2 35.8 34.4 6.0 166.7 YOLOv8s 51.7 39.1 40.8 11.1 57.2 SOD-YOLO 54.2 38.5 40.8 12.9 87.7 HIC-YOLOv5 50.6 40.8 41.1 9.3 73.5 The present invention 49.1 41.3 41.0 7.9 75.8

[0067] In Table 1, P is the precision, which is calculated by calculating the proportion of correct predictions in all positive samples; R is the recall, which is calculated by calculating the proportion of correct predictions in actual samples; mAP is the average precision, which indicates the overall detection accuracy performance; Param is the number of introduced parameters; the fewer the number of parameters, the lighter the detection performance; FPS is the number of frames transmitted per second, which is used to detect the real-time processing speed.

[0068] According to the results analysis in Table 1, compared with the method using the baseline model YOLOv5s (i.e., the original YOLOv5s model), the UAV aerial photography small target detection method based on dynamic multi-scale optimization of the present invention achieves a 5.4% improvement in precision P, a 6.9% improvement in recall R, and a 7.8% improvement in the average precision mAP index under the condition of introducing fewer parameters. Compared with other methods, although the UAV aerial photography small target detection method based on dynamic multi-scale optimization ranks fourth in the precision (P) indicator, this is partly because the high recall rate strategy tends to cover more potential targets, thereby expanding the positive sample prediction base; while the recall rate (R) index ranks first, which is 0.5% higher than the method using HIC-YOLOv5, indicating that the UAV aerial photography small target detection method based on dynamic multi-scale optimization can detect more positive samples and reduce the risk of missed detection; in terms of the overall average precision (mAP) indicator, the UAV aerial photography small target detection method based on dynamic multi-scale optimization reaches 41.0%, which is only 0.1% lower than the method using HIC-YOLOv5, indicating that in terms of comprehensive accuracy, the UAV aerial photography small target detection method based on dynamic multi-scale optimization is comparable to the HIC-YOLOv5 detection algorithm, while making up for the shortcomings in precision. In terms of parameter quantity indicators, the small target detection method for drone aerial photography based on dynamic multi-scale optimization performs better than the methods using YOLOv8s, SOD-YOLO and HIC-YOLOv5. It achieves a significant improvement in detection accuracy while introducing fewer parameters, and is more suitable for deployment on devices with higher requirements for lightweight. The frame rate indicator also shows that the small target detection method for drone aerial photography based on dynamic multi-scale optimization meets the requirements of real-time detection tasks.

Claims

1. A method for detecting small targets in drone aerial photography based on dynamic multi-scale optimization, wherein the YOLOv5s model is first improved to obtain an improved YOLOv5s model, and then the improved YOLOv5s model is trained to obtain a trained improved YOLOv5s model. When detecting small targets in drone aerial photography, the drone aerial image is input into the trained improved YOLOv5s model, and the trained improved YOLOv5s model outputs a detection result, which is characterized in that The improved YOLOv5s model is obtained by introducing anchor frames adapted to small target sizes, the spatial-depth convolution layer in the SPD-Conv module, the small target detection layer, the C3TR module, the CARAFE module, the GSConv module and the RepC3 module in the YOLOv5s model, and adding cross-layer connections to improve it. The improved YOLOv5s model has stronger small target feature capture capabilities, more efficient feature fusion mechanism and lower computational cost, thereby comprehensively improving the detection accuracy of small targets in drone aerial photography.

2. The method for detecting small targets in drone aerial photography based on dynamic multi-scale optimization according to claim 1 is characterized in that The YOLOv5s model includes a total of 25 layers of network structure, which are connected from front to back, and the 6th layer is also connected to the 12th layer, the 4th layer is also connected to the 16th layer, the 14th layer is also connected to the 19th layer, the 10th layer is also connected to the 22nd layer, and the 17th layer and the 20th layer are respectively connected to the 24th layer; the 25-layer network structure of the YOLOv5s model is referred to as its 0th to 24th layers in the forward propagation order, the 0th layer is the CBS module, the 1st layer is the CBS module, the 2nd layer is the C3 module, the 3rd layer is the CBS module, the 4th layer is the C3 module, the 5th layer is the CBS module, the 6th layer is the C3 module, the 7th layer is the CBS module, the 8th layer is the C3 module, and the 9th layer is the SP PF module, the 10th layer is the CBS module, the 11th layer is the UpSample module, the 12th layer is the Concat module, the 13th layer is the C3 module, the 14th layer is the CBS module, the 15th layer is the UpSample module, the 16th layer is the Concat module, the 17th layer is the C3 module, the 18th layer is the CBS module, the 19th layer is the Concat module, the 20th layer is the C3 module, the 21st layer is the CBS module, the 22nd layer is the Concat module, the 23rd layer is the C3 module, and the 24th layer is the Detect module; layers 0 to 9 constitute the Backbone network, layers 10 to 23 constitute the Neck network, and the 24th layer is the Head network.

3. The method for detecting small targets in drone aerial photography based on dynamic multi-scale optimization according to claim 2 is characterized in that The improved YOLOv5s model is obtained by making the following four improvements to the YOLOv5s model: First, add the spatial-depth convolutional layers in the SPD-Conv module after the 1st, 3rd, 5th, and 7th layers respectively, and replace the 8th layer with the C3TR module; Second, replace the UpSample modules of the 11th and 15th layers of the original YOLOv5s model with the CARAFE module, replace the CBS modules of the 10th, 14th, 18th, and 21st layers of the original YOLOv5s model with the GSConv module, and replace the C3 modules of the 20th and 23rd layers of the original YOLOv5s model with the RepC3 module; 3. Add a small target detection layer between the 17th and 18th layers of the original YOLOv5s model. The small target detection layer is formed by connecting a GSConv module, a CARAFE module, a Concat module, a RepC3 module, a GSConv module, a Concat module and a RepC3 module in the forward propagation order. The front GSConv module in the small target detection layer is called its first GSConv module, the back GSConv module is called its second GSConv module, the front RepC3 module is called its first RepC3 module, the back RepC3 module is called its second RepC3 module, the front Concat module is called its first Concat module, and the back Concat module is called its In the second Concat module, the first GSConv module of the small target detection layer is also connected to its second Concat module. At this time, the 17th and 18th layers of the original YOLOv5s model are no longer directly connected. The 17th layer of the original YOLOv5s model is connected to the first GSConv module of the small target detection layer and is no longer connected to the 24th layer of the original YOLOv5s model. The 18th layer of the original YOLOv5s model is connected to the second RepC3 module of the small target detection layer. At the same time, the first Concat module of the small target detection layer is also connected to the second layer of the original YOLOv5s model; the first RepC3 module and the second RepC3 module of the small target detection layer are both connected to the 24th layer of the original YOLOv5s. At the same time, while retaining the three groups of adaptive anchor frames of the original YOLOv5s model, a group of anchor frames adapted to the small target size is added; Fourth, add a connection between the 6th layer of the original YOLOv5s model and the 19th layer of the original YOLOv5s model, and add a connection between the 4th layer of the original YOLOv5s model and the second Concat module of the small object detection layer.

Images