Aerial target detection method based on attention mechanism and adaptive feature fusion

Abstract

The application discloses an aerial target detection method based on an attention mechanism and adaptive feature fusion, and comprises the following steps: acquiring aerial image data to be detected; inputting the aerial image data to be detected into a pre-trained aerial small target detection model to obtain a target detection result of the aerial image data to be detected, wherein the aerial small target detection model is obtained by setting a lightweight adaptive feature fusion module and a mixed coordinate attention layer in a YOLO algorithm framework and training aerial image data and corresponding target detection results. The application can achieve higher recognition accuracy in aerial small target detection. For aerial small target detection of a UAV platform, the network speed is fast and the accuracy is high, and better detection effect can be achieved with lower parameters and calculation complexity.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to an aerial target detection method based on attention mechanism and adaptive feature fusion. Background Technology

[0002] In recent years, with the rapid development of UAV technology, target detection in UAV aerial images has been widely applied in various fields, such as urban transportation, precision agriculture, and environmental surveying. Due to the rich application value of aerial image target detection, researchers have conducted extensive research on it. Early research mainly focused on sliding window search and manual feature extraction techniques, which typically require a large amount of prior knowledge and complex formula derivations. With the rapid development of computer vision technology, target detection based on deep learning algorithms has demonstrated superior performance and has gradually become the mainstream technology and is widely used.

[0003] Deep learning-based object detection algorithms are mainly divided into two categories: (1) two-stage object detection algorithms, represented by R-CNN and Faster R-CNN, which regress the target region based on the generation of candidate boxes, and have high detection accuracy; (2) one-stage object detection algorithms, represented by YOLO and SSD, which directly predict the position and category of the target, and have faster detection speed. Although deep learning-based object detection methods are widely used in various practical scenarios, since drones are usually deployed in large scenes, this means that there are many small and densely distributed objects in an image. At the same time, the complex and diverse background information can easily interfere with the target to be detected, making it difficult for general object detection algorithms to achieve ideal detection results.

[0004] Small targets in drone aerial images occupy only a small portion of the image, and the information directly obtained from fine-grained local regions is very limited. Utilizing contextual information within the image can establish connections between small targets and other targets or the background, thus aiding the network in detection and inference. Multi-scale features are an effective method to improve the detection rate of small targets. Although methods such as Faster R-CNN and YOLO employ single-scale feature maps, this limits the expression of feature information. To enhance the network's ability to express multi-scale information, researchers have proposed the classic feature pyramid structure, combining the characteristics of feature maps at different levels. This structure employs a top-down architecture and lateral connections to interact with feature information from different levels. Furthermore, path aggregation networks add additional bottom-up connections to FPN to more efficiently transmit shallow information to higher levels, thereby reducing information loss during information flow. However, while the above structures can improve the network's multi-scale representation ability, when fusing feature maps from different levels, they operate directly through element-wise addition, ignoring the contribution of different levels of feature maps to small targets. For small objects, fine-grained feature information from shallow features is more necessary for identification. Equal fusion of deep features may weaken the learning ability of shallow features for small targets. In addition, since different feature maps have different levels of detail, directly fusing them by addition or channel-dimensional concatenation can easily generate redundant and noisy information, affecting the acquisition of contextual information and interfering with the expression of relationships between objects. Summary of the Invention

[0005] This invention provides an aerial target detection method based on attention mechanism and adaptive feature fusion, which helps to achieve efficient and accurate detection of small targets in large scenes by drones, meeting the needs of practical applications in multiple fields.

[0006] The first aspect of this invention provides an aerial target detection method based on an attention mechanism and adaptive feature fusion, comprising the following steps: acquiring aerial image data to be detected; inputting the aerial image data to be detected into a pre-trained aerial small target detection model to obtain the target detection result of the aerial image data to be detected, wherein the aerial small target detection model is obtained by setting a lightweight adaptive feature fusion module and a shuffled coordinate attention layer in the YOLO algorithm framework, and training it using training aerial image data and its corresponding target detection results.

[0007] Optionally, in one embodiment of the present invention, the aerial small target detection model includes: a backbone network, a neck network, a lightweight adaptive feature fusion module, and a detection head layer;

[0008] The backbone network includes: a starting processing layer and four stage processing layers. The starting processing layer contains two convolutional modules, each consisting of three sub-modules: two-dimensional convolution, batch normalization, and sigmoid weighted linear unit activation function. The first stage processing layer consists of a lightweight and efficient layer aggregation network block and a cascaded shuffled coordinate attention layer. The second, third, and fourth stage processing layers each consist of a max pooling layer, a lightweight and efficient layer aggregation network block, and a cascaded shuffled coordinate attention layer. The four stage processing layers output a four-layer feature map of the aerial image data.

[0009] The neck network includes a path aggregation feature pyramid network, which is used to obtain the four-layer feature maps from the backbone network. The feature maps output by the first-stage processing layer, the second-stage processing layer, and the third-stage processing layer are input into the path aggregation feature pyramid network after the channels are reduced by the convolution module. The feature map output by the fourth-stage processing layer is input into the path aggregation feature pyramid network after the channels are reduced by the lightweight fast spatial pyramid pooling cross-stage local network module, so as to perform feature fusion using the path aggregation feature pyramid network.

[0010] The lightweight adaptive feature fusion module is located at the output end of the neck network. The lightweight adaptive feature fusion module consists of an expansion module that downsamples the feature map size through slicing and performs channel splicing to increase the number of channels, a fusion module that fuses channel information to reduce the number of channels, a content-aware feature reorganization upsampling module, and a soft pooling module. The lightweight adaptive feature fusion module is used to adaptively fuse the four-layer feature map output by the path aggregation feature pyramid network and send the feature map into the detection head layer.

[0011] The detection head layer includes multiple detection heads, each consisting of an additive implicit module, a convolutional module, and a multiplicative implicit module. The loss function of the detection head includes localization loss, classification loss, and confidence loss.

[0012] Optionally, in one embodiment of the present invention, the positioning loss adopts the SIOU loss function, and the confidence loss adopts the zoom loss function.

[0013] Optionally, in one embodiment of the present invention, the upsampling layer in the path aggregation network is a content-aware feature reorganization upsampling layer.

[0014] A second aspect of the present invention provides an aerial target detection device based on an attention mechanism and adaptive feature fusion, comprising: an acquisition module for acquiring aerial image data to be detected; and a detection module for inputting the aerial image data to be detected into a pre-trained aerial small target detection model to obtain a target detection result of the aerial image data to be detected, wherein the aerial small target detection model is obtained by setting a lightweight adaptive feature fusion module and a shuffled coordinate attention layer in the YOLO algorithm framework, and training it using training aerial image data and its corresponding target detection results.

[0015] Optionally, in one embodiment of the present invention, the aerial small target detection model includes: a backbone network, a neck network, a lightweight adaptive feature fusion module, and a detection head layer;

[0016] The backbone network includes: a starting processing layer and four stage processing layers. The starting processing layer contains two convolutional modules, each consisting of three sub-modules: two-dimensional convolution, batch normalization, and sigmoid weighted linear unit activation function. The first stage processing layer consists of a lightweight and efficient layer aggregation network block and a cascaded shuffled coordinate attention layer. The second, third, and fourth stage processing layers each consist of a max pooling layer, a lightweight and efficient layer aggregation network block, and a cascaded shuffled coordinate attention layer. The four stage processing layers output a four-layer feature map of the aerial image data.

[0017] The neck network includes a path aggregation feature pyramid network, which is used to obtain the four-layer feature maps from the backbone network. The feature maps output by the first-stage processing layer, the second-stage processing layer, and the third-stage processing layer are input into the path aggregation feature pyramid network after the channels are reduced by the convolution module. The feature map output by the fourth-stage processing layer is input into the path aggregation feature pyramid network after the channels are reduced by the lightweight fast spatial pyramid pooling cross-stage local network module, so as to perform feature fusion using the path aggregation feature pyramid network.

[0018] The lightweight adaptive feature fusion module is located at the output end of the neck network. The lightweight adaptive feature fusion module consists of an expansion module that downsamples the feature map size through slicing and performs channel splicing to increase the number of channels, a fusion module that fuses channel information to reduce the number of channels, a content-aware feature reorganization upsampling module, and a soft pooling module. The lightweight adaptive feature fusion module is used to adaptively fuse the four-layer feature map output by the path aggregation feature pyramid network and send the feature map into the detection head layer.

[0019] The detection head layer includes multiple detection heads, each consisting of an additive implicit module, a convolutional module, and a multiplicative implicit module. The loss function of the detection head includes localization loss, classification loss, and confidence loss.

[0020] Optionally, in one embodiment of the present invention, the positioning loss adopts the SIOU loss function, and the confidence loss adopts the zoom loss function.

[0021] Optionally, in one embodiment of the present invention, the upsampling layer in the path aggregation network is a content-aware feature reorganization upsampling layer.

[0022] A third aspect of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to perform the aerial target detection method based on attention mechanism and adaptive feature fusion as described in the above embodiments.

[0023] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to perform the aerial target detection method based on attention mechanism and adaptive feature fusion as described in the above embodiments.

[0024] This invention presents an aerial target detection method based on attention mechanisms and adaptive feature fusion. It utilizes a lightweight and efficient attention mechanism called "shuffling coordinate attention," which aims to achieve effective attention between features with low computational cost. By grouping feature map channels and applying coordinate attention—a combination of spatial and channel attention—to each group, a channel shuffling operation is used to facilitate information flow between different groups. This low computational cost makes it suitable for lightweight networks. Simultaneously, a new P2-level feature map detection layer is added, which can effectively detect smaller targets, improving detection performance. Furthermore, a lightweight adaptive feature fusion module is added after the path aggregation feature pyramid network in the neck network to achieve adaptive fusion of multi-scale features, improving detection accuracy. Finally, zoom loss and SIOU loss are used to further improve the accuracy of classification and localization.

[0025] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0026] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0027] Figure 1A flowchart illustrating an aerial target detection method based on attention mechanism and adaptive feature fusion according to an embodiment of the present invention;

[0028] Figure 2 This is a schematic diagram of the aerial small target detection model provided in an embodiment of the present invention;

[0029] Figure 3 This is a structural diagram of the shuffling coordinate attention module provided in an embodiment of the present invention;

[0030] Figure 4 A framework diagram of a content-aware feature reorganization upsampling operator provided according to an embodiment of the present invention;

[0031] Figure 5 A comparison chart of map metrics between the improved algorithm and the original algorithm provided according to an embodiment of the present invention;

[0032] Figure 6 This is a comparison chart of training process metrics between the improved algorithm and the original algorithm provided according to an embodiment of the present invention;

[0033] Figure 7 This is an example diagram of an aerial target detection device based on attention mechanism and adaptive feature fusion according to an embodiment of the present invention;

[0034] Figure 8 A schematic diagram of the structure of an electronic device provided for an embodiment of the invention. Detailed Implementation

[0035] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0036] Figure 1 This is a flowchart illustrating an aerial target detection method based on attention mechanism and adaptive feature fusion according to an embodiment of the present invention.

[0037] like Figure 1 As shown, this aerial target detection method based on attention mechanism and adaptive feature fusion includes the following steps:

[0038] In step S101, the aerial image data to be detected is acquired.

[0039] In embodiments of the present invention, aerial images include various categories such as pedestrians, cyclists, bicycles, electric vehicles, cars, trucks, and buses.

[0040] In step S102, the aerial image data to be detected is input into a pre-trained aerial small target detection model to obtain the target detection result of the aerial image data to be detected. The aerial small target detection model is obtained by setting a lightweight adaptive feature fusion module and a shuffled coordinate attention layer in the YOLO algorithm framework and training it using the training aerial image data and its corresponding target detection results.

[0041] In an embodiment of the present invention, the aerial image data to be detected is input into a pre-trained aerial small target detection model to obtain the target detection results of the aerial image data to be detected, such as multiple detection boxes and corresponding categories.

[0042] In embodiments of the present invention, aerial image data is collected by a drone and labeled to obtain a dataset. The dataset is divided into a training set, a validation set, and a test set. The dataset is then used to train, validate, and test the aerial small target detection model to obtain a trained aerial small target detection model.

[0043] This invention constructs an improved network model suitable for small target detection in aerial photography based on the YOLOv7 algorithm framework, resulting in the ASOD-YOLO algorithm. The ASOD-YOLO algorithm is trained on a training set, employing various data augmentation strategies during training, and parameters are fine-tuned using a validation set to obtain the optimal weight file. The trained optimal weight file is then loaded into the ASOD-YOLO algorithm to obtain the target detection network. The detection results are then tested on a test set, and the network is used to detect targets in subsequently acquired aerial images.

[0044] This invention selects the lightweight and fast YOLO series algorithms as the base network and improves upon them to propose an improved ASOD-YOLO algorithm, thereby enhancing the detection capability of small targets in aerial photography. Through this series of innovative improvements, it will help achieve efficient and accurate detection of small targets in large-scale scenes using drones, meeting the needs of practical applications in multiple fields.

[0045] The aerial small target detection model of this invention includes: a backbone network, a neck network, a lightweight adaptive feature fusion module, and a detection head layer.

[0046] The backbone network consists of an initial processing layer and four stage processing layers. The initial processing layer contains two convolutional modules, each consisting of three sub-modules: 2D convolution, batch normalization, and sigmoid weighted linear unit activation function. The first stage processing layer consists of a lightweight and efficient layer aggregation network block and a cascaded shuffled coordinate attention layer. The second, third, and fourth stage processing layers each consist of a max pooling layer, a lightweight and efficient layer aggregation network block, and a cascaded shuffled coordinate attention layer. The four stage processing layers output a four-layer feature map of the aerial image data.

[0047] The neck network includes a path aggregation feature pyramid network, which extracts four layers of feature maps from the backbone network. The feature maps output from the first, second, and third processing layers are reduced in number by convolutional modules before being input into the path aggregation feature pyramid network. The feature map output from the fourth processing layer is reduced in number by a lightweight, fast spatial pyramid pooling cross-stage local network module before being fed into the path aggregation feature pyramid network for feature fusion. The upsampling layer in the path aggregation network uses content-aware feature reassembly upsampling.

[0048] A lightweight adaptive feature fusion module is located at the output of the neck network. This module consists of an expansion module that downsamples the feature map size through slicing and performs channel concatenation to increase the number of channels, a fusion module that fuses channel information to reduce the number of channels, a content-aware feature reorganization upsampling module, and a soft pooling module. This module adaptively fuses the four-layer feature maps output from the path aggregation feature pyramid network, increasing feature extraction capabilities, and then feeds the feature maps into the detection head layer.

[0049] The detection head layer consists of multiple detection heads, each composed of additive implicit modules, convolutional modules, and multiplicative implicit modules. The loss function of the detection head includes localization loss, classification loss, and confidence loss.

[0050] During the training of the aerial small target detection model, an adaptive momentum estimation optimizer with weight decay was used as the optimizer. The initial learning rate was set to 1e-2, the minimum learning rate was 0.0004, the weight decay coefficient was 0.05, the network was trained in 300 batches, the number of samples in each training was set to 256, and the learning rate scheduling strategy was cosine annealing.

[0051] The aerial target detection method based on attention mechanism and adaptive feature fusion of the present invention will be described below through a specific embodiment.

[0052] First, the process of feature extraction from aerial images using the ASOD-YOLO network model is as follows: Figure 2 The backbone part of the network is shown in the diagram. The input image size is 640x640x3. The image first passes through the starter processing layer (Stem) module, which reduces the image size and increases the number of channels. The starter processing layer module consists of two convolutional modules, each composed of three sub-modules: 2D convolution, batch normalization, and sigmoid weighted linear unit activation function (Silu). After passing through the starter processing layer, the feature map size becomes 160x160x64.

[0053] The image then passes through four processing stages. The first stage consists of a cascaded lightweight, efficient layer aggregation network block and a shuffling coordinate attention module. The structure of the lightweight, efficient layer aggregation network block is as follows: Figure 2 As shown in the detailed section, the structure diagram of the shuffling coordinate attention module is as follows: Figure 3 As shown. The second, third, and fourth processing layers consist of a max pooling layer, a lightweight and efficient layer aggregation network block, and a cascaded shuffling coordinate attention module. The four processing layers output feature maps of different sizes, P2, P3, P4, and P5, respectively, with sizes of 160x160x64, 80x80x128, 40x40x256, and 20x20x512.

[0054] The P2, P3, and P4 level feature maps each pass through a convolutional module before entering the neck network. The P5 level feature map, however, is processed by the Lightweight Fast Spatial Pyramid Pooling Cross-Stage Local Network Module (TinySPPFCSPBlock) before entering the neck network. The detailed structure of the Lightweight Fast Spatial Pyramid Pooling Cross-Stage Local Network Module is as follows... Figure 2 As shown in the Detail section.

[0055] The feature maps from layers P2, P3, P4, and P5 of the neck network are fused through a path aggregation feature pyramid network. The upsampling layer of the path aggregation feature pyramid network is replaced with a content-aware feature reorganization upsampling operator, as shown below. Figure 4 As shown. The four-layer feature map output by the path aggregation feature pyramid network is processed by the lightweight adaptive feature fusion module (TinyASFF). Figure 2 The right half of the Neck (in the image) undergoes adaptive feature fusion to output a four-layer feature map, which is then fed into the head detection network for detection. The lightweight adaptive feature fusion module mainly consists of four modules: an expansion module that downsamples the feature map size through slicing and concatenates channels to increase the number of channels; a fusion module that reduces the number of channels by fusing channel information; a content-aware feature reorganization and upsampling module; and a soft pooling module.

[0056] The head detection network consists of four sub-modules: an additive implicit module (ImplicitA), a convolutional module, a multiplicative implicit module (ImplicitM), and a loss calculation module. Its detailed structure is as follows: Figure 2 The details section is shown below. The loss calculation module consists of three parts: the localization loss uses the SIOU loss function, and the classification loss and confidence loss use the Variational Loss function.

[0057] In convolutional networks that generate feature maps, neurons generating low-level feature maps undergo less pre-processing, resulting in a smaller receptive field on the original image. This approach prioritizes preserving details such as edges and textures. Conversely, neurons generating high-level feature maps undergo more pre-processing, resulting in a larger receptive field on the original image. This approach prioritizes preserving semantic information. High-level features undergo multiple downsampling steps, generally ignoring more detailed information. The original YOLOv7 utilizes feature maps output after 8x, 16x, and 32x downsampling for subsequent classification and regression tasks. It uses feature maps P3, P4, and P5 with receptive fields of 8x, 16x, and 32x, respectively, specifically 80x80x128, 40x40x256, and 20x20x512 pixels. However, small targets in aerial images typically have only a few pixels, and the semantic information that the network can extract from these limited pixels is very limited. In extreme cases, a small target in an image may correspond to only one point on a high-level feature map. Therefore, the detection of small targets requires more consideration of the feature maps extracted by neurons with smaller receptive fields. Thus, the ASOD-YOLO algorithm proposed in this invention adds a P2 layer feature map for detection, and correspondingly adds a P2 layer detection head.

[0058] The ASOD-YOLO algorithm proposed in this invention optimizes aspects such as the feature extraction backbone network output feature map, attention mechanism, feature fusion, detection layer, and loss calculation, effectively enhancing the network model's detection accuracy for small objects. Detection result metrics are as follows: Figure 5 As shown, the Map index has improved significantly. Model training is also more stable, with relevant metrics during the training process as follows: Figure 6 As shown.

[0059] The aerial target detection method based on attention mechanism and adaptive feature fusion proposed in this invention adds a lightweight and efficient attention mechanism, shuffling coordinate attention, which aims to achieve effective attention between features with low computational cost. By grouping feature map channels, applying coordinate attention combining spatial and channel attention to each group, and finally using a channel shuffling operation to facilitate information flow between different groups, the computational cost is low, making it suitable for lightweight networks. Simultaneously, a small target detection layer, a P2-level feature map detection layer, is added. The P2 detection layer can effectively detect smaller targets, improving detection performance. A lightweight adaptive feature fusion module is added after the path aggregation feature pyramid network in the neck network to achieve adaptive fusion of multi-scale features, improving detection accuracy. Finally, zoom loss and SIOU loss are used to further improve the accuracy of classification and localization.

[0060] Next, referring to the accompanying drawings, an aerial target detection device based on attention mechanism and adaptive feature fusion proposed according to an embodiment of the present invention is described.

[0061] Figure 7 This is an example diagram of an aerial target detection device based on attention mechanism and adaptive feature fusion according to an embodiment of the present invention.

[0062] like Figure 7 As shown, the aerial target detection device 10 based on attention mechanism and adaptive feature fusion includes: an acquisition module 100 and a detection module 200.

[0063] The acquisition module 100 is used to acquire the aerial image data to be detected. The detection module 200 is used to input the aerial image data to be detected into a pre-trained aerial small target detection model to obtain the target detection result of the aerial image data. The aerial small target detection model is obtained by setting a lightweight adaptive feature fusion module and a shuffled coordinate attention layer in the YOLO algorithm framework, and training it using training aerial image data and its corresponding target detection results.

[0064] In an embodiment of the present invention, the aerial small target detection model includes: a backbone network, a neck network, a lightweight adaptive feature fusion module, and a detection head layer;

[0065] The backbone network consists of a starting processing layer and four stage processing layers. The starting processing layer contains two convolutional modules, each consisting of three sub-modules: two-dimensional convolution, batch normalization, and sigmoid weighted linear unit activation function. The first stage processing layer consists of a lightweight and efficient layer aggregation network block and a shuffled coordinate attention layer cascaded together. The second, third, and fourth stage processing layers each consist of a max pooling layer, a lightweight and efficient layer aggregation network block, and a shuffled coordinate attention layer cascaded together. The four stage processing layers output four-layer feature maps of the aerial image data.

[0066] The neck network includes a path aggregation feature pyramid network, which is used to obtain four layers of feature maps from the backbone network. The feature maps output by the first, second and third processing layers are input into the path aggregation feature pyramid network after the channels are reduced by the convolution module. The feature map output by the fourth processing layer is input into the path aggregation feature pyramid network after the channels are reduced by the lightweight fast spatial pyramid pooling cross-stage local network module, so as to utilize the path aggregation feature pyramid network for feature fusion.

[0067] A lightweight adaptive feature fusion module is located at the output of the neck network. This module consists of an expansion module that downsamples the feature map size through slicing and performs channel concatenation to increase the number of channels, a fusion module that fuses channel information to reduce the number of channels, a content-aware feature reorganization upsampling module, and a soft pooling module. The lightweight adaptive feature fusion module performs adaptive feature fusion on the four-layer feature map output from the path aggregation feature pyramid network and feeds the feature map into the detection head layer.

[0068] The detection head layer consists of multiple detection heads, each composed of additive implicit modules, convolutional modules, and multiplicative implicit modules. The loss function of the detection head includes localization loss, classification loss, and confidence loss.

[0069] In embodiments of the present invention, the localization loss uses the SIOU loss function, and the confidence loss uses the zoom loss function. In embodiments of the present invention, the upsampling layer in the path aggregation network is a content-aware feature reorganization upsampling layer.

[0070] It should be noted that the foregoing explanation of the aerial target detection method based on attention mechanism and adaptive feature fusion also applies to the aerial target detection device based on attention mechanism and adaptive feature fusion in this embodiment, and will not be repeated here.

[0071] The aerial target detection device based on attention mechanism and adaptive feature fusion proposed in this invention adds a lightweight and efficient attention mechanism, shuffling coordinate attention, which aims to achieve effective attention between features with low computational cost. By grouping feature map channels, applying coordinate attention combining spatial and channel attention to each group, and finally using a channel shuffling operation to facilitate information flow between different groups, the computational cost is low, making it suitable for lightweight networks. Simultaneously, a small target detection layer, a P2-level feature map detection layer, is added. The P2 detection layer can effectively detect smaller targets, improving detection performance. A lightweight adaptive feature fusion module is added after the path aggregation feature pyramid network in the neck network to achieve adaptive fusion of multi-scale features, improving detection accuracy. Finally, zoom loss and SIOU loss are used to further improve the accuracy of classification and localization.

[0072] Figure 8 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. The electronic device may include:

[0073] The memory 801, the processor 802, and the computer program stored on the memory 801 and capable of running on the processor 802.

[0074] When the processor 802 executes the program, it implements the aerial target detection method based on attention mechanism and adaptive feature fusion provided in the above embodiments.

[0075] Furthermore, electronic devices also include:

[0076] Communication interface 803 is used for communication between memory 801 and processor 802.

[0077] The memory 801 is used to store computer programs that can run on the processor 802.

[0078] The memory 801 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0079] If the memory 801, processor 802, and communication interface 803 are implemented independently, then the communication interface 803, memory 801, and processor 802 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0080] Optionally, in a specific implementation, if the memory 801, processor 802, and communication interface 803 are integrated on a single chip, then the memory 801, processor 802, and communication interface 803 can communicate with each other through an internal interface.

[0081] The processor 802 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0082] This embodiment also provides a computer-readable storage medium storing a computer program, characterized in that the program, when executed by a processor, implements the above-described aerial target detection method based on attention mechanism and adaptive feature fusion.

[0083] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0084] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0085] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0086] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0087] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

Claims

1. An aerial target detection method based on attention mechanism and adaptive feature fusion, characterized in that, Includes the following steps: Acquire aerial image data to be detected; The aerial image data to be detected is input into a pre-trained aerial small target detection model to obtain the target detection result of the aerial image data to be detected. The aerial small target detection model is obtained by setting a lightweight adaptive feature fusion module and a shuffled coordinate attention layer in the YOLO algorithm framework and training it using training aerial image data and its corresponding target detection results. The aerial small target detection model includes: a backbone network, a neck network, a lightweight adaptive feature fusion module, and a detection head layer; The backbone network includes: a starting processing layer and four stage processing layers. The starting processing layer contains two convolutional modules, each consisting of three sub-modules: two-dimensional convolution, batch normalization, and sigmoid weighted linear unit activation function. The first stage processing layer consists of a lightweight and efficient layer aggregation network block and a shuffled coordinate attention layer cascaded together. The second, third, and fourth stage processing layers each consist of a max pooling layer, a lightweight and efficient layer aggregation network block, and a shuffled coordinate attention layer cascaded together. The four stage processing layers output a four-layer feature map of the aerial image data. The neck network includes a path aggregation feature pyramid network, which is used to obtain the four-layer feature maps from the backbone network. The feature maps output by the first-stage processing layer, the second-stage processing layer, and the third-stage processing layer are input into the path aggregation feature pyramid network after the channels are reduced by the convolution module. The feature map output by the fourth-stage processing layer is input into the path aggregation feature pyramid network after the channels are reduced by the lightweight fast spatial pyramid pooling cross-stage local network module, so as to perform feature fusion using the path aggregation feature pyramid network. The lightweight adaptive feature fusion module is located at the output end of the neck network. The lightweight adaptive feature fusion module consists of an expansion module that downsamples the feature map size through slicing and performs channel splicing to increase the number of channels, a fusion module that fuses channel information to reduce the number of channels, a content-aware feature reorganization upsampling module, and a soft pooling module. The lightweight adaptive feature fusion module is used to adaptively fuse the four-layer feature map output by the path aggregation feature pyramid network and send the feature map into the detection head layer. The detection head layer includes multiple detection heads, each consisting of an additive implicit module, a convolutional module, and a multiplicative implicit module. The loss function of the detection head includes localization loss, classification loss, and confidence loss.

2. The method according to claim 1, characterized in that, The positioning loss uses the SIOU loss function, and the confidence loss uses the zoom loss function.

3. The method according to claim 1, characterized in that, The upsampling layer in the path aggregation network is a content-aware feature reorganization upsampling layer.

4. An aerial target detection device based on attention mechanism and adaptive feature fusion, characterized in that, include: The acquisition module is used to acquire the aerial image data to be detected; The detection module is used to input the aerial image data to be detected into a pre-trained aerial small target detection model to obtain the target detection result of the aerial image data to be detected. The aerial small target detection model is obtained by setting a lightweight adaptive feature fusion module and a shuffled coordinate attention layer in the YOLO algorithm framework and training it using training aerial image data and its corresponding target detection results. The aerial small target detection model includes: a backbone network, a neck network, a lightweight adaptive feature fusion module, and a detection head layer; The backbone network includes: a starting processing layer and four stage processing layers. The starting processing layer contains two convolutional modules, each consisting of three sub-modules: two-dimensional convolution, batch normalization, and sigmoid weighted linear unit activation function. The first stage processing layer consists of a lightweight and efficient layer aggregation network block and a shuffled coordinate attention layer cascaded together. The second, third, and fourth stage processing layers each consist of a max pooling layer, a lightweight and efficient layer aggregation network block, and a shuffled coordinate attention layer cascaded together. The four stage processing layers output a four-layer feature map of the aerial image data. The neck network includes a path aggregation feature pyramid network, which is used to obtain the four-layer feature maps from the backbone network. The feature maps output by the first-stage processing layer, the second-stage processing layer, and the third-stage processing layer are input into the path aggregation feature pyramid network after the channels are reduced by the convolution module. The feature map output by the fourth-stage processing layer is input into the path aggregation feature pyramid network after the channels are reduced by the lightweight fast spatial pyramid pooling cross-stage local network module, so as to perform feature fusion using the path aggregation feature pyramid network. The lightweight adaptive feature fusion module is located at the output end of the neck network. The lightweight adaptive feature fusion module consists of an expansion module that downsamples the feature map size through slicing and performs channel splicing to increase the number of channels, a fusion module that fuses channel information to reduce the number of channels, a content-aware feature reorganization upsampling module, and a soft pooling module. The lightweight adaptive feature fusion module is used to adaptively fuse the four-layer feature map output by the path aggregation feature pyramid network and send the feature map into the detection head layer. The detection head layer includes multiple detection heads, each consisting of an additive implicit module, a convolutional module, and a multiplicative implicit module. The loss function of the detection head includes localization loss, classification loss, and confidence loss.

5. The apparatus according to claim 4, characterized in that, The positioning loss uses the SIOU loss function, and the confidence loss uses the zoom loss function.

6. The apparatus according to claim 4, characterized in that, The upsampling layer in the path aggregation network is a content-aware feature reorganization upsampling layer.

7. An electronic device, characterized in that, include: The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aerial target detection method based on attention mechanism and adaptive feature fusion as described in any one of claims 1-3.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the aerial target detection method based on attention mechanism and adaptive feature fusion as described in any one of claims 1-3.

Images