Detection method, device and equipment suitable for aerial photography of small target

Abstract

The invention provides a detection method, device and equipment suitable for aerial photography of a small target, and relates to the technical field of image processing, and the method comprises the steps: obtaining a to-be-detected image; adding jump connection between input and output features of the same feature layer by using a bidirectional feature pyramid network DBiFPN, fusing input feature layers with different resolutions, and extracting initial feature information from the to-be-detected image; capturing and strengthening the initial feature information by using a multi-element branch module C2fDB to obtain intermediate feature information; and optimizing the intermediate feature information by using a bounding box regression loss function to obtain target feature information, and expressing the target feature information through a triple attention mechanism and a dynamic head framework. According to the method, feature maps of different scales are fused through bidirectional weighting by means of the improved DBiFPN, information exchange and transmission between targets of different scales are enhanced, the positioning performance and generalization ability of the model are enhanced, and the feature expression ability is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a detection method, device and equipment suitable for small targets in aerial photography. Background Art

[0002] With the acceleration of urbanization, tasks such as urban land surveys, building monitoring, and ecological environment monitoring are becoming increasingly important. Traditional methods based on manual field surveys are not only costly but also inefficient. In the field of target detection, with the development of deep learning technology, models such as Faster R-CNN, YOLO series, and SSD have been proposed. These models perform well in general target detection tasks, but still face significant challenges in drone aerial photography scenarios. Drone aerial images have the characteristics of low target resolution, small target proportion, and complex background. Traditional target detection algorithms have limitations in dealing with these problems. First, small targets only occupy very few pixels in the image, and existing multi-scale feature fusion methods are difficult to retain and extract key information of small targets. In addition, the complex background and occlusion problems in aerial photography scenes further increase the difficulty of detection. Especially in scenarios that require efficient processing such as real-time monitoring and disaster emergency response, the model must not only have high accuracy, but also maintain high computational efficiency. However, existing target detection algorithms are difficult to strike a balance between accuracy and speed when dealing with small target detection and complex backgrounds. Although existing mainstream target detection algorithms, such as the YOLO series, have advantages in speed, their ability to detect small targets still has a lot of room for improvement, and the target recognition accuracy and generalization ability in complex scenes also need to be optimized. Summary of the invention

[0003] The purpose of the present invention is to solve the problem that the current target detection algorithm is difficult to strike a balance between accuracy and speed when dealing with small target detection and complex background. The present invention provides a detection method, device and equipment suitable for aerial photography of small targets.

[0004] The technical solution of the embodiment of the present application is implemented as follows: A first aspect of an embodiment of the present application provides a detection method for small targets in aerial photography, comprising: Acquire the image to be detected; A bidirectional feature pyramid network DBiFPN is used to add a skip connection between the input and output features of the same feature layer, fuse the input feature layers of different resolutions, and extract the initial feature information from the image to be detected; the bidirectional feature pyramid network DBiFPN is formed by fusing the BiFPN network and the multi-branch module C2fDB; The initial feature information is captured and enhanced by using a multi-branch module C2fDB to obtain intermediate feature information; the multi-branch module C2fDB is formed by fusing a multi-branch structure DBB and a C2f module of YOLOv8; The intermediate feature information is optimized using a bounding box regression loss function to obtain target feature information, and the target feature information is expressed through a triple attention mechanism and a dynamic head framework.

[0005] Optionally, the method of adding a skip connection between input and output features of the same feature layer by using a bidirectional feature pyramid network DBiFPN, fusing input feature layers of different resolutions, and extracting initial feature information from the image to be detected includes: Use network learning to determine the weights corresponding to different input feature layers; Based on the weights corresponding to the input feature layers, input feature layers of different resolutions are fused to obtain an input fusion layer; The image to be detected is input into the input fusion layer to extract initial feature information.

[0006] Optionally, the use of the bidirectional feature pyramid network DBiFPN adds a jump connection between the input and output features of the same feature layer, fuses input feature layers of different resolutions, and extracts initial feature information from the image to be detected, including: upward fusion and downward fusion; the principle of the upward fusion is: ; is the fused feature map, It is a higher-level fusion feature map. is the original feature map of the current level, and is the learned weight coefficient, Upsample is the upsampling operation, and Conv is the convolution operation; The principle of downward fusion is: ; is the upward fusion output of the current layer, It is the original feature map of the previous lower layer, and Down is the downsampling operation.

[0007] Optionally, before the initial feature information is captured and enhanced by the multi-branch module C2fDB to obtain the intermediate feature information, the method further includes: The DBB module is introduced into the convolution kernel in the Bottleneck module of the C2f module to obtain the multi-branch module C2fDB.

[0008] Optionally, the optimizing the intermediate feature information by using a bounding box regression loss function to obtain target feature information, and expressing the target feature information by using a triple attention mechanism and a dynamic head framework, includes: A bounding box regression loss function is created using a dynamic non-monotonic focusing mechanism; the bounding box regression loss function includes a first loss function WIoU v1, a second loss function WIoU v2 and a third loss function WIoU v3; wherein, The expression of WIoU v1 is: ; ; The expression of WIoU v2 is: ; WIoU v3 defines outliers To measure the quality of the anchor box, ; The expression of WIoU v3 is: ; here, represents the weighted IoU loss, represents the distance penalty term in the IoU loss, b represents the predicted box, gt represents the real box, and the subscripts cx and cy represent the x and y coordinates of the center point respectively. Represents the x-coordinate of the center point of the real box, Represents the x-coordinate of the center point of the prediction box. Similarly, Represents the y coordinate of the center point of the real box, represents the y coordinate of the center point of the prediction box, cw and ch represent the width and height of the minimum closure area that can contain the prediction box and the real box, respectively. , .

[0009] Optionally, before optimizing the intermediate feature information by using a bounding box regression loss function to obtain target feature information, and expressing the target feature information by using a triple attention mechanism and a dynamic head frame, the method further includes: Add a triple attention mechanism to the last layer of the YOLOv8 backbone network.

[0010] Optionally, the optimizing the intermediate feature information by using a bounding box regression loss function to obtain target feature information, and expressing the target feature information by using a triple attention mechanism and a dynamic head framework, includes: The properties of the convolution kernel are dynamically adjusted through the dynamic head frame and the target feature information to determine the spatial position and shape of the target to be detected.

[0011] The second aspect of the embodiment of the present application provides a detection device suitable for aerial photography of small targets, including: a camera module, a bidirectional feature pyramid network DBiFPN, a multi-branch module C2fDB and an expression module; wherein, A camera module, configured to acquire an image to be detected; A bidirectional feature pyramid network DBiFPN is formed by fusing a BiFPN network and a multi-branch module C2fDB, and is configured to add a skip connection between input and output features of the same feature layer, fuse input feature layers of different resolutions, and extract initial feature information from the image to be detected; The multi-branch module C2fDB is formed by fusing the multi-branch structure DBB and the C2f module of YOLOv8, and is configured to capture and enhance the initial feature information to obtain intermediate feature information; The expression module is configured to optimize the intermediate feature information using a bounding box regression loss function to obtain target feature information, and express the target feature information through a triple attention mechanism and a dynamic head framework.

[0012] A third aspect of an embodiment of the present application provides an electronic device, including a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements the detection method for small targets in aerial photography described in the first aspect.

[0013] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0014] Compared with the prior art, the technical solution provided by this application has the following beneficial effects: The present invention provides a detection method, device and equipment suitable for small aerial photography targets. The method uses a bidirectional feature pyramid network DBiFPN to add jump connections between input and output features of the same feature layer, fuses input feature layers of different resolutions, extracts initial feature information from the image to be detected, and uses a multi-branch module C2fDB to capture and enhance the initial feature information to obtain intermediate feature information, thereby enhancing information exchange and transmission between targets of different scales. On this basis, the intermediate feature information is optimized using a bounding box regression loss function to obtain target feature information, thereby reducing the gradient gain between high-quality samples and low-quality samples, thereby enhancing the positioning performance and generalization ability of the model. Finally, the target feature information is expressed through a triple attention mechanism and a dynamic head framework, thereby significantly improving the feature expression capability while ensuring lightweight. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1A schematic diagram of a flow chart of a detection method for small targets in aerial photography provided in an embodiment of the present application; Figure 2 A schematic diagram of the structure of a detection device suitable for aerial photography of small targets provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0016] Below, embodiments of the present application will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present application. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present application. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present application. The terms used herein are only for describing specific embodiments and are not intended to limit the present application. The terms "include", "comprising", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components. All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0017] Some block diagrams and / or flow charts are shown in the accompanying drawings. It should be understood that some blocks or combinations thereof in the block diagrams and / or flow charts may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that these instructions, when executed by the processor, may create a device for implementing the functions / operations described in these block diagrams and / or flow charts.

[0018] In some embodiments, see Figure 1 , Figure 1 A flow chart of a detection method for a small target in aerial photography provided in an embodiment of the present application; a detection method for a small target in aerial photography provided in an embodiment of the present application includes: S110, obtaining an image to be detected.

[0019] Here, the image to be detected may be an image containing the target to be detected obtained by shooting with a camera mounted on a drone. After obtaining the image to be detected, a series of preprocessing, such as denoising, may be performed on the image to be detected.

[0020] S120, using the bidirectional feature pyramid network DBiFPN to add jump connections between the input and output features of the same feature layer, fuse the input feature layers of different resolutions, and extract initial feature information from the image to be detected; the bidirectional feature pyramid network DBiFPN is formed by fusing the BiFPN network and the multi-branch module C2fDB.

[0021] Bi-directional Feature Pyramid Network (BiFPN) is a weighted bidirectional feature pyramid network. By introducing a feature fusion mechanism of a bidirectional path, bidirectional information flow between high-resolution and low-resolution features is allowed, thereby enhancing the complementarity of features at different levels and improving the network's ability to recognize multi-scale objects. The C2f module is a key component in the YOLO algorithm for feature extraction and conversion. Here, a bidirectional feature pyramid network DBiFPN is used to add a jump connection between the input and output features of the same layer. Since the scale is the same, adding a jump connection can better extract and transmit feature information.

[0022] In some embodiments, S120, using a bidirectional feature pyramid network DBiFPN to add a skip connection between input and output features of the same feature layer, fusing input feature layers of different resolutions, and extracting initial feature information from the image to be detected, includes: Use network learning to determine the weights corresponding to different input feature layers; Based on the weights corresponding to each input feature layer, input feature layers of different resolutions are fused to obtain an input fusion layer; The image to be detected is input into the input fusion layer to extract the initial feature information.

[0023] In this embodiment, DBiFPN uses weighted feature fusion to fuse input feature layers of different resolutions. The weights corresponding to input layers of different resolutions are also different. The weight parameters of each input layer can be automatically learned by the network, which allows the network to learn the importance between different feature layers and better represent the overall feature information.

[0024] In some embodiments, S120, a bidirectional feature pyramid network DBiFPN is used to add a jump connection between the input and output features of the same feature layer, fuse the input feature layers of different resolutions, and extract the initial feature information from the image to be detected, including: upward fusion and downward fusion; the principle of upward fusion is: ; is the fused feature map, It is a higher-level fusion feature map. is the original feature map of the current level, and is the learned weight coefficient, Upsample is the upsampling operation, and Conv is the convolution operation; The principle of downward fusion is: ; is the upward fusion output of the current layer, It is the original feature map of the previous lower layer, and Down is the downsampling operation.

[0025] In this embodiment, upward fusion and downward fusion involve upsampling and downsampling, wherein upsampling is the process of converting a low-resolution image into a high-resolution image. In the upsampling process, image data is gradually increased, and new pixel values ​​are usually generated by an interpolation algorithm to generate a high-resolution image. Downsampling is to convert a high-resolution image into a low-resolution image, and the image is reduced layer by layer, and each layer is the downsampling result of the previous layer image.

[0026] S130, using the multi-branch module C2fDB to capture and enhance the initial feature information to obtain intermediate feature information; the multi-branch module C2fDB is formed by integrating the multi-branch structure DBB and the C2f module of YOLOv8.

[0027] After the initial feature information is input into C2fDB, C2fDB further processes and transforms the initial feature information through its diverse branch structures, which can capture different aspects and details in the initial feature information, thereby generating richer and more diverse intermediate feature information. The intermediate feature information may contain more advanced and abstract features, such as the shape, outline, color, etc. of an object.

[0028] In some embodiments, at S130, before the initial feature information is captured and enhanced by the multi-branch module C2fDB to obtain the intermediate feature information, the method further includes: The DBB module is introduced into the convolution kernel in the Bottleneck module of the C2f module to obtain the multi-branch module C2fDB.

[0029] The C2f module is a key component in the YOLO algorithm for feature extraction and transformation. The input data is feature transformed through two convolutional layers (cv1 and cv2). The input data is processed in two branches, one branch is passed directly to the output, and the other branch is processed by multiple Bottleneck modules. The Bottleneck module is a widely used structure in neural networks, especially in object detection, image classification, and segmentation tasks. The number of input channels is reduced through a 1x1 convolution layer, and then a 3x3 or larger convolution kernel is used for feature extraction, and finally the number of channels is restored through another 1x1 convolution layer. The multi-branch structure of the DBB module can capture more features of different scales, thereby enhancing the feature extraction capability. It should be noted that here, it is necessary to ensure that the number of input and output channels of the DBB module is consistent with that of the Bottleneck module to keep the overall structure and function of the C2f module unaffected.

[0030] S140, optimizes the intermediate feature information using the bounding box regression loss function to obtain the target feature information, and expresses the target feature information through the triple attention mechanism and the dynamic head frame.

[0031] As an important part of the target detection loss function, the bounding box loss function can bring significant performance improvements to the target detection model if it is well defined. Considering that the current bounding box loss function regression effect is not ideal, especially for objects with irregular shapes and large scale changes, a dynamic non-monotonic focusing mechanism is introduced here to improve the bounding box loss function.

[0032] In some embodiments, S140, the intermediate feature information is optimized using a bounding box regression loss function to obtain target feature information, and the target feature information is expressed through a triple attention mechanism and a dynamic head frame, including: A bounding box regression loss function is created using a dynamic non-monotonic focusing mechanism; the bounding box regression loss function includes a first loss function WIoU v1, a second loss function WIoU v2, and a third loss function WIoU v3; wherein, The expression of WIoU v1 is: ; ; The expression of WIoU v2 is: ; WIoU v3 defines outliers To measure the quality of the anchor box, ; The expression of WIoU v3 is: ; here, represents the weighted IoU loss, represents the distance penalty term in the IoU loss, b represents the predicted box, gt represents the real box, and the subscripts cx and cy represent the x and y coordinates of the center point respectively. Represents the x-coordinate of the center point of the real box, Represents the x-coordinate of the center point of the prediction box. Similarly, Represents the y coordinate of the center point of the real box, represents the y coordinate of the center point of the prediction box, cw and ch represent the width and height of the minimum closure area that can contain the prediction box and the real box, respectively. , .

[0033] A lower β value indicates a higher quality anchor box, and a smaller r value is assigned to it, thereby reducing the weight of these high-quality anchor boxes when calculating a larger loss function. Conversely, a higher β value indicates a lower quality anchor box, and such an anchor box will receive a smaller gradient gain, reducing the harmful gradients generated by low-quality anchor boxes. Through this dynamic allocation of gradient gains, WIoU v3 optimizes the weight balance of high-quality and low-quality anchor boxes in loss calculation, making the model pay more attention to samples of average quality, thereby improving overall performance.

[0034] In an example, three versions of WIoU were tested for performance, and the best result was achieved when WIoU v3 was used. The results of each version of WIoU are shown in Table 1.

[0035] Table 1: Performance test of different versions of WIoU

[0036] For target detection tasks in drone aerial photography scenes, the high proportion of small targets increases the detection difficulty. WIoUv3 can dynamically optimize the loss weights of small targets and improve the detection performance of the model.

[0037] In some embodiments, at S140, before optimizing the intermediate feature information using the bounding box regression loss function to obtain the target feature information and expressing the target feature information using the triple attention mechanism and the dynamic head frame, it also includes: Add a triple attention mechanism to the last layer of the YOLOv8 backbone network.

[0038] Triplet attention is a method for computing attention weights by capturing cross-dimensional interactions using a three-branch structure. The three branches use attention on rows, columns, and channels respectively. For the input tensor, the triplet attention mechanism establishes dependencies between dimensions by rotation operations and then uses residual transformations to encode inter-channel and spatial information with negligible computational overhead. Triplet Attention can enhance the feature representation of small objects through the attention mechanism in three dimensions. In traditional convolutional networks, the features of small objects become insignificant due to their relatively small size. Triplet Attention helps the model better identify and locate small objects by enhancing the feature responses related to small objects. By applying attention on different dimensions (such as width, height, and channel), Triplet Attention can promote the fusion of features of different scales and levels. This cross-scale feature integration can effectively improve the detection performance of small objects that are sensitive to scale changes. By focusing on features in different dimensions, Triplet Attention can also help the model learn richer and more diverse feature expressions, which can enhance the model's generalization and recognition capabilities for unseen small objects of different sizes. In addition, the attention mechanism can guide the model to pay more attention to the area containing the target, reduce the consumption of computing resources in the background area, and is more conducive to lightweight.

[0039] In some embodiments, S140, the intermediate feature information is optimized using a bounding box regression loss function to obtain target feature information, and the target feature information is expressed through a triple attention mechanism and a dynamic head frame, including: The properties of the convolution kernel are dynamically adjusted through the dynamic head frame and target feature information to determine the spatial position and shape of the target to be detected.

[0040] The dynamic head framework DyHead implements an integrated detection head by effectively combining multiple self-attention mechanisms on the feature layer scale, spatial position, and task-aware output channels. The model is implemented through a combination of scale attention, spatial attention, and task attention. The core advantage of Dynamic Head is that it can dynamically adjust the properties of the convolution kernels (such as size, shape, and number) according to the input features, thereby providing flexible adaptability to objects of different sizes, especially small objects. This dynamicity enables the detection head to optimize the allocation of computing resources according to the specific characteristics of the target, thereby enhancing the ability to handle small targets. At the same time, it also helps to learn the complex patterns and features of small targets. In addition, by dynamically adjusting the internal structure and parameters, Dynamic Head is able to improve the accuracy and robustness of small target detection while maintaining an efficient computing speed, thereby effectively improving the overall detection efficiency without sacrificing performance. By integrating DyHead into the existing YOLOv8 detection head component, the performance is significantly improved compared to popular object detectors.

[0041] The embodiment of the present invention uses a bidirectional feature pyramid network DBiFPN to add jump connections between the input and output features of the same feature layer, fuses input feature layers of different resolutions, extracts initial feature information from the image to be detected, and uses a multi-branch module C2fDB to capture and enhance the initial feature information to obtain intermediate feature information, thereby enhancing the information exchange and transmission between targets of different scales. On this basis, the intermediate feature information is optimized using a bounding box regression loss function to obtain target feature information, thereby reducing the gradient gain between high-quality samples and low-quality samples, thereby enhancing the positioning performance and generalization ability of the model, and finally expressing the target feature information through a triple attention mechanism and a dynamic head framework, thereby significantly improving the ability of feature expression while ensuring lightweight.

[0042] In some embodiments, see Figure 2 , Figure 2 A structural diagram of a detection device for small aerial targets provided in an embodiment of the present application; an embodiment of the present application provides a detection device 200 for small aerial targets, including: a camera module 210, a bidirectional feature pyramid network DBiFPN220, a multi-branch module C2fDB230 and an expression module 240; wherein, The camera module 210 is configured to obtain an image to be detected; The bidirectional feature pyramid network DBiFPN220 is formed by fusing the BiFPN network and the multi-branch module C2fDB, and is configured to add skip connections between the input and output features of the same feature layer, fuse input feature layers of different resolutions, and extract initial feature information from the image to be detected; The multi-branch module C2fDB230 is formed by fusing the multi-branch structure DBB and the C2f module of YOLOv8, and is configured to capture and enhance the initial feature information to obtain the intermediate feature information; The expression module 240 is configured to optimize the intermediate feature information using the bounding box regression loss function to obtain the target feature information, and express the target feature information through the triple attention mechanism and the dynamic head frame.

[0043] In some embodiments, the bidirectional feature pyramid network DBiFPN220 is specifically configured as follows: Use network learning to determine the weights corresponding to different input feature layers; Based on the weights corresponding to each input feature layer, input feature layers of different resolutions are fused to obtain an input fusion layer; The image to be detected is input into the input fusion layer to extract initial feature information.

[0044] In some embodiments, the bidirectional feature pyramid network DBiFPN220 is specifically configured to be upward fusion and downward fusion; the principle of upward fusion is: ; is the fused feature map, It is a higher-level fusion feature map. is the original feature map of the current level, and is the learned weight coefficient, Upsample is the upsampling operation, and Conv is the convolution operation; The principle of downward fusion is: ; is the upward fusion output of the current layer, It is the original feature map of the previous lower layer, and Down is the downsampling operation.

[0045] In some embodiments, the detection device further includes a construction module, and the construction module is specifically configured as follows: The DBB module is introduced into the convolution kernel in the Bottleneck module of the C2f module to obtain the multi-branch module C2fDB.

[0046] In some embodiments, the expression module 240 is specifically configured as follows: A bounding box regression loss function is created using a dynamic non-monotonic focusing mechanism; the bounding box regression loss function includes a first loss function WIoU v1, a second loss function WIoU v2 and a third loss function WIoU v3; wherein, The expression of WIoU v1 is: ; ; The expression of WIoU v2 is: ; WIoU v3 defines outliers To measure the quality of the anchor box, ; The expression of WIoU v3 is: ; here, represents the weighted IoU loss, represents the distance penalty term in the IoU loss, b represents the predicted box, gt represents the real box, and the subscripts cx and cy represent the x and y coordinates of the center point respectively. Represents the x-coordinate of the center point of the real box, Represents the x-coordinate of the center point of the prediction box. Similarly, Represents the y coordinate of the center point of the real box, represents the y coordinate of the center point of the prediction box, cw and ch represent the width and height of the minimum closure area that can contain the prediction box and the real box, respectively. , .

[0047] In some embodiments, the building block is further configured to: Add a triple attention mechanism to the last layer of the YOLOv8 backbone network.

[0048] In some embodiments, the expression module 240 is specifically configured as follows: The properties of the convolution kernel are dynamically adjusted through the dynamic head frame and target feature information to determine the spatial position and shape of the target to be detected.

[0049] The detection device for small targets in aerial photography provided in the embodiment of the present application can implement each process in the embodiment corresponding to the above-mentioned detection method for small targets in aerial photography. To avoid repetition, they will not be described again here.

[0050] It should be noted that the detection device for small targets in aerial photography provided in the embodiment of the present application and the detection method for small targets in aerial photography provided in the embodiment of the present application are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned detection method for small targets in aerial photography, and the repeated parts will not be repeated.

[0051] In an optional embodiment, the detection method suitable for small aerial targets provided in the embodiment of the present application is used for verification. First, the image to be detected is obtained from the VisDrone2021 dataset. The VisDrone2021 dataset is an open dataset collected and developed by Tianjin University in China and the data mining team AISKYEYE, focusing on computer vision analysis of complex drone images and videos. It covers multiple tasks such as target detection, tracking, counting, and attribute analysis, including target categories such as pedestrians, vehicles, and bicycles. The target features in the images of this dataset match the characteristics of our research scene, so we use this dataset for research.

[0052] (1) Experimental environment and parameter configuration Table 2: Training environment and hardware platform parameters

[0053] This paper uses the improved YOLOv8s network structure for training, and the number of training epochs is set to 150, at which point convergence has been achieved. The initial learning rate is set to 0.001, and the minimum learning rate is 0.0001. As the training progresses, the learning rate is continuously reduced. The initial learning rate is set larger to save training time, and the minimum learning rate is set to prevent the network from being trapped in the local minimum. Through experiments, the hyperparameters of Wise-loU we use are WIoU v3, that is, α (WIoU) is 1.9, and δ (WIoU) is 3.

[0054] (2) Ablation experiment In the later stage of the experiment, in order to verify the influence of different improvement strategies on the original network, four groups of ablation experiments were conducted on the lightweight YOLOv8s, and the experimental results are shown in Table 2. Each group of experiments was completed on the visdrone2021 dataset and the same training parameters. It can be seen that when various improvements are added separately, each improvement has a certain improvement on the model performance. As can be seen from the table, after replacing the original backbone network with the network model invented by us, the performance of the original model has been significantly improved, mAP50 has increased by 1.3 percentage points, and the precision and recall rates have been significantly improved; on this basis, after the model was optimized using Wise-IoU, mAP50 exceeded 40%, reaching 40.2%; when the attention mechanism was added to the last layer of the backbone network of YOLOv8, mAP50 was further improved, especially the precision rate, which was improved by 2.3%; in order to further examine the network performance, all the above improvement strategies were added, and the final performance was higher than all the previous experiments, with mAP50 reaching 41.1%. In order to verify the effectiveness of various improvements, ablation experiments were used for testing.

[0055] Table 3: Ablation study of four improvements on the VisDrone2021 dataset

[0056] Note: “—” indicates that the method is not used in the model, and “√” indicates that the method is used in the model.

[0057] The VisDrone2021 dataset has a small target ratio of 87.77% and a large target ratio of only 0.26%. It is a very typical small target dataset in the field of aerial photography, which is enough to show that our method has more advantages than existing methods in small target detection. UAVDT is one of the typical datasets from the perspective of drone aerial photography, which contains a large number of small targets. The DOTA dataset is a dataset specifically used for target detection in aerial images. A large number of targets appear in small sizes in the images, such as airplanes, ships, vehicles, etc., and it is also an ideal choice for studying small target detection.

[0058] Different target detection models were compared on VisDrone2021 and verified on the public datasets UAVDT and DOTA. From the comparative experimental results on the VisDrone2021 dataset, it can be seen that compared with the more representative algorithms in the field of target detection, the improved model has the highest mAP50. Although YOLOv5, which is widely used in engineering practice, can also achieve 41% mAP50 when using the x model, its parameter volume is much higher than that of this model, and its detection speed is far lower than that of this model. On the UAVDT and DOTA datasets, this model performed the best, with mAP50 reaching 98.6% and 83.1% respectively.

[0059] Compared with yolov5s, this model has much higher accuracy with similar parameters. Compared with the lightweight algorithm YOLOv7-tiny, the improved algorithm has better performance with a small increase in parameters. The improved algorithm also has the fastest detection speed, and all indicators of the improved algorithm show great advantages compared with the baseline algorithm YOLOv8s. Combining various indicators, our improved model has a balanced performance on the three data sets, especially in processing speed and accuracy, with the best overall performance, and has good detection capabilities for small targets in drone aerial images.

[0060] Table 4: Performance comparison of different target detection algorithms on VisDrone2021, UAVDT and DOTA datasets

[0061] Table 5: Comparison results of VisDrone2021 dataset 2

[0062] As shown in the table, the improved model was evaluated in detail and compared with the current state-of-the-art models on the VisDrone2021 dataset. Although this model has a certain gap in accuracy compared to DPNet-ensemble (the best performer in terms of average precision), it shows higher performance than the latest YOLO algorithm-gold-yolo. In comparison with many leading SOTA algorithms, this model ranks among the top in terms of comprehensive performance. In addition, compared with several benchmark algorithms, this model shows obvious advantages, proving its high usability in practical applications.

[0063] In some embodiments, see Figure 3 , Figure 3 The electronic device 300 provided in the embodiment of the present application includes a processor 310 and a memory 320; the memory 320 stores a computer program, wherein the computer program implements the above-mentioned detection method for small aerial targets when executed by the processor.

[0064] Specifically, the processor 310 may include, for example, a general-purpose microprocessor, an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 310 may also include an onboard memory for cache purposes. The processor 310 may be a single processing unit or multiple processing units for executing different actions of the method flow according to the embodiment of the present application.

[0065] The memory 320 may be any medium capable of containing, storing, conveying, propagating or transmitting instructions. For example, the memory 320 may include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device, component or propagation medium. Specific examples of the memory 320 include: a magnetic storage device, such as a magnetic tape or a hard disk (HDD); an optical storage device, such as a compact disk (CD-ROM); a random access memory (RAM) or a flash memory; and / or a wired / wireless communication link. The present application also provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned detection method suitable for small aerial targets is implemented. The computer-readable medium may be included in the device / apparatus / system described in the above-mentioned embodiments; or it may exist independently without being assembled into the device / apparatus / system. The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed, the method according to the embodiment of the present application is implemented.

[0066] According to an embodiment of the present application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, optical cable, radio frequency signal, etc., or any suitable combination of the above.

[0067] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present application. In particular, without departing from the spirit and teachings of the present application, the features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present application. Therefore, the scope of the present application should not be limited to the above-described embodiments, but should be determined not only by the appended claims, but also by the equivalents of the appended claims.

Claims

1. A detection method for small aerial targets, applied to the YOLOv8 model, characterized in that: include: Acquire the image to be detected; A bidirectional feature pyramid network DBiFPN is used to add a skip connection between the input and output features of the same feature layer, fuse the input feature layers of different resolutions, and extract the initial feature information from the image to be detected; the bidirectional feature pyramid network DBiFPN is formed by fusing the BiFPN network and the multi-branch module C2fDB; The initial feature information is captured and enhanced by using a multi-branch module C2fDB to obtain intermediate feature information; the multi-branch module C2fDB is formed by fusing a multi-branch structure DBB and a C2f module of YOLOv8; The intermediate feature information is optimized using a bounding box regression loss function to obtain target feature information, and the target feature information is expressed through a triple attention mechanism and a dynamic head framework.

2. The detection method for small targets in aerial photography according to claim 1, characterized in that: The method of using a bidirectional feature pyramid network DBiFPN to add a skip connection between input and output features of the same feature layer, fuse input feature layers of different resolutions, and extract initial feature information from the image to be detected includes: Use network learning to determine the weights corresponding to different input feature layers; Based on the weights corresponding to the input feature layers, input feature layers of different resolutions are fused to obtain an input fusion layer; The image to be detected is input into the input fusion layer to extract initial feature information.

3. The detection method for small targets in aerial photography according to claim 1, characterized in that: The bidirectional feature pyramid network DBiFPN is used to add a jump connection between the input and output features of the same feature layer, fuse the input feature layers of different resolutions, and extract the initial feature information from the image to be detected, including: upward fusion and downward fusion; the principle of the upward fusion is: ； is the fused feature map, It is a higher-level fusion feature map. is the original feature map of the current level, and is the learned weight coefficient, Upsample is the upsampling operation, and Conv is the convolution operation; The principle of downward fusion is: ； is the upward fusion output of the current layer, It is the original feature map of the previous lower layer, and Down is the downsampling operation.

4. The detection method for small targets in aerial photography according to claim 1, characterized in that: Before the initial feature information is captured and enhanced by the multi-branch module C2fDB to obtain the intermediate feature information, the method further includes: The DBB module is introduced into the convolution kernel in the Bottleneck module of the C2f module to obtain the multi-branch module C2fDB.

5. The detection method for small targets in aerial photography according to claim 1, characterized in that: The method of optimizing the intermediate feature information by using the bounding box regression loss function to obtain target feature information, and expressing the target feature information by using a triple attention mechanism and a dynamic head frame, includes: A bounding box regression loss function is created using a dynamic non-monotonic focusing mechanism; the bounding box regression loss function includes a first loss function WIoU v1, a second loss function WIoU v2 and a third loss function WIoU v3; wherein, The expression of WIoU v1 is: ； ； The expression of WIoU v2 is: ； WIoU v3 defines outliers To measure the quality of the anchor box, ; The expression of WIoU v3 is: ； here, represents the weighted IoU loss, represents the distance penalty term in the IoU loss, b represents the predicted box, gt represents the real box, and the subscripts cx and cy represent the x and y coordinates of the center point respectively. Represents the x-coordinate of the center point of the real box, Represents the x-coordinate of the center point of the prediction box. Similarly, Represents the y coordinate of the center point of the real box, represents the y coordinate of the center point of the prediction box, cw and ch represent the width and height of the minimum closure area that can contain the prediction box and the real box, respectively. , .

6. The detection method for small targets in aerial photography according to claim 1, characterized in that: Before optimizing the intermediate feature information by using the bounding box regression loss function to obtain the target feature information and expressing the target feature information by using the triple attention mechanism and the dynamic head frame, the method further includes: Add a triple attention mechanism to the last layer of the YOLOv8 backbone network.

7. The detection method for small targets in aerial photography according to claim 1, characterized in that: The method of optimizing the intermediate feature information by using the bounding box regression loss function to obtain target feature information, and expressing the target feature information by using a triple attention mechanism and a dynamic head frame, includes: The properties of the convolution kernel are dynamically adjusted through the dynamic head frame and the target feature information to determine the spatial position and shape of the target to be detected.

8. A detection device suitable for aerial photography of small targets, characterized in that: include: Camera module, bidirectional feature pyramid network DBiFPN, multi-branch module C2fDB and expression module; among them, A camera module, configured to acquire an image to be detected; A bidirectional feature pyramid network DBiFPN is formed by fusing a BiFPN network and a multi-branch module C2fDB, and is configured to add a skip connection between input and output features of the same feature layer, fuse input feature layers of different resolutions, and extract initial feature information from the image to be detected; The multi-branch module C2fDB is formed by fusing the multi-branch structure DBB and the C2f module of YOLOv8, and is configured to capture and enhance the initial feature information to obtain intermediate feature information; The expression module is configured to optimize the intermediate feature information using a bounding box regression loss function to obtain target feature information, and express the target feature information through a triple attention mechanism and a dynamic head framework.

9. An electronic device comprising a processor and a memory; the memory stores a computer program, wherein: When the computer program is executed by the processor, the computer program implements the detection method for small targets in aerial photography according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images