Target detection method based on attention mechanism and Gabor transform

Abstract

The invention discloses a target detection method based on an attention mechanism and Gabor transform, and designs an attention feature extraction module AFEM, and an attention feature extraction module based on an ELAN feature extraction network and a CBAM attention mechanism, so that the network can focus on an aircraft target while performing efficient feature extraction, and the target detection efficiency is improved. Interference caused by complex background information is reduced; a fractional order Gabor filter is adopted to transform a convolution FGTC module to strengthen texture, scale and direction features of an aircraft target, edge feature details of the aircraft target are strengthened, and richer feature information is provided for the network; an LDyHead dynamic detection head is introduced as a network head, and scale perception attention, space perception attention and task perception attention are combined, so that the network can more accurately predict the position and category of an aircraft target when facing a multi-scale aircraft target, and the convergence speed of the model is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a target detection method based on attention mechanism and Gabor transform. Background Art

[0002] In the field of image analysis, object detection is a challenging problem. For example, when analyzing images, objects such as airplanes and vehicles are usually small in size, diverse in shape, and mixed with complex background environments. These backgrounds may include airport runways, buildings, and natural landscapes, which makes it particularly difficult to accurately identify and locate the objects to be detected from the complex background. In addition, the diversity of the scale, orientation, and arrangement of the objects to be detected further increases the complexity of the detection task.

[0003] Specifically, when the background of the image is extremely complex and there are often many interference items in the surrounding environment of the target to be detected, for example, the environment where the aircraft is parked often includes a large number of buildings, woods, and vehicles that are similar in shape and size to the aircraft. Even the ground color and building color of some scenes are similar to the color of the aircraft. These factors make it difficult to accurately separate the aircraft target from the complex background, affecting the detection accuracy of the aircraft target.

[0004] Wu et al. proposed a large-scale, robust hyperspectral target detector TLH2TD based on transfer learning, which filtered out the complex background in the image; Shi et al. proposed a two-stage detection method based on deep neural network, which improved the ability to distinguish between background and target; Han et al. proposed a remote sensing image target detection method based on improved YOLOv5, which enhanced the model's location perception ability and suppressed the interference of redundant information. These methods have improved the detection performance to a certain extent, but there are still limitations in processing images with highly complex backgrounds and multi-scale targets.

[0005] In image target detection methods, faced with complex background conditions and small size of the target to be detected in the image, how to improve the model's ability to capture the texture and edge features of the target, and how to improve the model's ability to recognize the target, effectively suppress background noise and highlight target features, so as to more accurately locate and identify the target visually has become a key technology to improve the accuracy of image target detection.

[0006] Existing image target detection methods often have many limitations when facing special detection targets. For example, in the aircraft target detection method, due to the special shape of the aircraft and its similar aspect ratio, the ability to remove redundant background is limited in the target detection task. In addition, when processing targets, the general detection layer often has very limited perception of the scale and space of the target, resulting in inaccurate predictions in the classification and position regression stages, which in turn affects the overall detection performance of the model. Summary of the invention

[0007] Purpose of the invention: To solve the above problems, the present invention proposes a target detection method based on Attention Mechanisms and Garber Transformations Networks (AMGTNet), aiming to reduce the impact of complex background on the detection performance of the model and further improve the detection accuracy of small targets.

[0008] Technical solution: To achieve the purpose of the present invention, the technical solution adopted by the present invention is: a target detection method based on attention mechanism and Gabor transform, comprising the following steps:

[0009] Step 1: Obtain a remote sensing road image dataset, pre-process the dataset, and divide it into a training set and a test set according to a set ratio;

[0010] Step 2: Build an object detection network based on the improved attention mechanism and Gabor transform, including the backbone network, Neck network and LDyhead module;

[0011] The backbone network consists of 4 CBS modules, 4 AFEM modules and 3 transition modules; the Neck network consists of 1 global pyramid pooling layer, 2 transition modules, 4 upsampling, 4 Concat operations, 2 CBS modules, 2 FGTC modules and 4 CB-ELAN modules;

[0012] Step 3: Input the remote sensing images of the training set into the target detection network for training, calculate the loss function and perform back propagation, update the network parameters, and obtain the optimal parameter model;

[0013] Step 4: Input the test set into the optimal parameter model trained in step 3 and output the detection map of the remote sensing image.

[0014] Furthermore, in step 2, a target detection network is constructed, including:

[0015] Build a feature extraction network. After processing by four CBS modules, introduce the attention feature extraction module AFEM to construct a feature extraction network in the form of a feature pyramid to obtain feature maps at different levels. At the same time, the network also contains three transition modules for dimensionality reduction and feature compression.

[0016] Build a feature fusion network and perform downsampling. In the feature fusion part, use the FGTC module to enhance the feature information obtained in the feature extraction network, and use the FAUM module in the upsampling part.

[0017] After the entire feature fusion network passes through the global pyramid pooling layer, transition module, and CB-ELAN module, it continuously outputs three feature maps to the LDyhead module through three output terminals;

[0018] Build the LDyhead detection head. The output feature maps of the three output terminals correspond to the inputs of the three detection heads of three different sizes. The detection head includes a scale-aware attention module and a spatial-aware attention module. Through several convolution and maximum pooling layer operations, the features are extracted, adjusted and aggregated.

[0019] Connect the feature extraction network, feature fusion network and LDyhead detection head;

[0020] After the original image undergoes preliminary feature extraction through the CBS module of the feature extraction network, it is further enhanced through the AFEM module, pyramid pooling is performed through the global pyramid pooling layer module of the feature fusion network, and feature fusion and conversion are performed through the FGTC, FAUM, and transition modules. It then enters the LDyhead module for final prediction and outputs the location, category, and confidence of the detection box.

[0021] Furthermore, the feature extraction network first passes through 4 CBS modules, then passes through 4 attention feature extraction modules AFEM, and then passes through 3 transition modules for output, specifically:

[0022] The CBS module consists of a convolutional layer, normalization, and a silu activation function to perform preliminary extraction of image features and size transformation of the image;

[0023] The convolution kernel size of the first CBS module and the fourth CBS module is 3, and the step size is 1. The convolution kernel size of the second CBS module and the third CBS module is 3, and the step size is 2.

[0024] The attention feature extraction module consists of an ELAN module and a CBAM module;

[0025] The ELAN module is an improved efficient network structure. The input feature map is passed through two parallel convolution modules to obtain outputs X1 and X2. X2 is used as the input of two serial convolution modules to obtain output X3. X3 is used as the input of another two serial convolution modules to obtain output X4. Finally, the four outputs are merged to obtain the final output F0 of the ELAN module. The calculation formula is defined as follows:

[0026] X1=X2=Conv 3×3 (F1)

[0027] X3=Conv 3×3 (Conv 3×3 (X2))

[0028] X4=Conv 3×3 (Conv 3×3 (X3))

[0029] F0=Conv 1×1 (X1+X2+X3+X4)

[0030] The CBAM module includes two sub-modules, namely a channel attention module and a spatial attention module;

[0031] The channel attention module is composed of an adaptive maximum pooling layer, an adaptive average pooling layer and an MLP. The adaptive maximum pooling layer automatically processes the size of the input feature map and pools it into a fixed-size feature map;

[0032] The MLP is a multi-layer perceptron, implemented by stacking multiple fully connected layers, each of which uses a non-linear activation function;

[0033] The spatial attention module is composed of a maximum pooling layer with a convolution kernel size of 7 and an average pooling layer with a convolution kernel size of 7 connected in series, and the corresponding feature map is obtained by a convolution operation with a convolution kernel size of 7 and a step size of 3;

[0034] The transition module performs a merging operation by dividing the input feature map and selecting the maximum value of each divided area as the output. The transition module is composed of a maximum pooling layer, and a parallel structure consisting of two CBS modules with a convolution kernel size of 1 and a CBS module with a convolution kernel size of 3.

[0035] Furthermore, the channel attention mechanism module includes the following operations:

[0036] First, perform global average pooling and global maximum pooling operations on the collected feature maps to obtain two channel dimension feature vectors and obtain global feature information;

[0037] Then, the acquired feature information is passed through a fully connected network with shared weights, which contains two fully connected layers and ReLU activation function to generate two channel attention feature maps;

[0038] Finally, the two feature maps are element-wise added and passed through a Sigmoid activation function to obtain the final channel attention weight, which is used to adjust each channel of the original feature map, as shown in the following formula:

[0039] M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F)))

[0040] Where M c(F) represents the channel attention output feature map, F represents the input feature map, AvgPool() and MaxPool() represent average pooling and maximum pooling operations, MLP() represents the shared weight operation, and σ represents the Sigmoid activation function.

[0041] Furthermore, the spatial attention mechanism module includes the following operations:

[0042] First, use the global average pooling operation and the global maximum pooling operation to obtain two channel dimension feature vectors respectively to obtain global feature information;

[0043] Then, the acquired feature information is passed through a fully connected network with shared weights, which contains two fully connected layers and ReLU activation function to generate two channel attention feature maps;

[0044] Next, these two feature maps are stacked in the channel dimension and processed through a convolution layer with a convolution kernel size of 7×7 to capture contextual information in the spatial dimension;

[0045] Finally, the spatial attention map is generated by the Sigmoid activation function, and the attention map is element-wise multiplied with the original feature map, as shown in the following formula:

[0046] M s (F) = σ(f 7×7 ([AvgPool(F); MaxPool(F)]))

[0047] Where M s (F) represents the spatial attention output feature map, F represents the input feature map, f 7×7 () represents a convolution operation with a convolution kernel size of 7×7, AvgPool() and MaxPool() represent average pooling and maximum pooling operations, and σ represents the Sigmoid activation function.

[0048] Furthermore, the network structure of the FGTC module is as follows:

[0049] First, a random initialization is performed and input into the FGF module, and then after passing through a two-dimensional convolution layer, a ReLU activation function, Dropout, Padding and Maxpool, the final output is obtained; the two-dimensional convolution layer uses a 3×3 convolution kernel;

[0050] The FGTC module adopts a fractional-order Gabor filter;

[0051] The FAUM module uses sub-pixel convolution to process the high-level feature map F i Get F i+1 , so that it is consistent with the low-level feature map F iThe same size; then F i+1 and F i Perform splicing operations to integrate feature information; perform maximum pooling and average pooling operations on the spliced ​​feature maps to generate a two-dimensional feature map F Max and F Avg ;

[0052] Finally, the integrated features are activated to generate spatial weights, and the spatial weights are combined with the high-level features F output by sub-pixel convolution. i+1 Multiply and detect the output features of the image.

[0053] Furthermore, the LDyhead module is divided into three layers of input and output;

[0054] The first layer input passes through the ReLU function, 1×1 convolution, Saigmoid activation function, and the maximum pooling layer, and then enters the second layer through a weighted sum merger;

[0055] The second layer input passes through Offset, 3×3 convolution, Saigmoid activation function, Index and then output to the third layer;

[0056] The third layer input passes through the ReLU function, FC, maximum pooling layer, and Normalize, and finally the outputs of the above three layers are weighted summed.

[0057] Beneficial effects: Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects:

[0058] 1) The present invention combines CBAM with ELAN module by proposing a new attention feature extraction module (AFEM). ELAN module is an improved and efficient network structure for feature extraction. The module strengthens feature aggregation by strengthening the connection between different layers and adopts a path optimization strategy to improve the propagation efficiency of information flow in the network, thereby improving the feature extraction ability of the network in complex scenes. Based on channel and spatial attention, the CBAM module focuses on the important parts of the image and suppresses irrelevant background information, so that the network can more effectively identify and locate the target to be detected. The combination of the two greatly reduces the impact of the complex background in the image on the feature extraction of small targets.

[0059] 2) The present invention performs FGTC module processing on feature maps at different feature levels of the backbone network, significantly improving the network's ability to capture the texture, scale, and directional features of the target in the image. It not only enhances the model's ability to understand image details, but also provides more critical and richer feature information for the subsequent feature fusion process. At the same time, the feature aggregation upsampling module (FAUM) is used to reduce the loss of detail information during the upsampling process.

[0060] 3) By constructing the LDyhead detection head, the network can adaptively focus on the scale change and spatial position of the target and suppress irrelevant background information. After passing through the task-aware attention module, the feature map re-forms different activations according to the subsequent different task requirements, and its output can be accurately classified and regressed, which greatly improves the adaptability and accuracy of the network model in the face of multi-scale small targets and complex scenes.

[0061] In summary, the method proposed in the present invention further improves the accuracy of detecting small objects in images while reducing the impact of complex background on model detection performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is the structural diagram of AMGTNet.

[0063] Figure 2 This is the structural diagram of the AFEM module.

[0064] Figure 3 This is the structural diagram of ELAN.

[0065] Figure 4 This is the structural diagram of the CBAM module.

[0066] Figure 5 This is the structural diagram of the FGTC module.

[0067] Figure 6 This is the structural diagram of the FAUM module.

[0068] Figure 7 This is the structural diagram of LDyhead.

[0069] Figure 8 A visual qualitative comparison of AMGTNet with other methods on the DOTAv1 dataset. DETAILED DESCRIPTION

[0070] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0071] The embodiment of the present invention provides a target detection network (AMGTNet) based on attention mechanism and Gabor transform, and the network structure is as follows: Figure 1 As shown, the target detection method based on attention mechanism and Gabor transform includes the following steps:

[0072] Step 1: Get the DOTAv1 remote sensing image dataset, which contains 2806 remote sensing images with sizes between 800 and 4000. The dataset contains 15 categories with a total of 18882 instances, including airplanes, ships, vehicles, etc., and uses horizontal target annotation boxes as label annotations.

[0073] The preprocessing and data set division are as follows: the original remote sensing aircraft images and the corresponding label images in the DOTAV1 dataset are cropped into images of 640×640 pixels in order to avoid overlapping areas; a variety of image enhancement techniques are used, such as flipping, translation, scaling and cutting. Finally, the dataset is divided into a training set and a validation set, where the training set contains 15,749 randomly selected images and the validation set contains 5,297 randomly selected images.

[0074] Step 2: Build a multi-scale feature enhancement network as the target detection network, including the backbone network module, AFEM module (CB-ELAN), FGTC module, FAUM module and LDyhead module. The network contains three continuous attention mechanisms.

[0075] The steps to build a multi-scale feature enhancement network are:

[0076] Step 2.1, the processed remote sensing aircraft image is used as the input image to pass through the backbone network: the backbone network consists of the initial 4 CBS modules and 4 CB-ELAN modules (AFEM) and 3 transition modules.

[0077] The CBS module consists of a convolutional layer, normalization, and silu activation function, which is used to preliminarily extract image features and transform the image size. The convolution kernel size of the first and fourth CBS modules is 3 and the step size is 1. The convolution kernel size of the second and third CBS modules is 3 and the step size is 2. At this time, the feature map size becomes 160×160×128.

[0078] Then the attention feature extraction block, namely the CB-ELAN module (AFEM): the CB-ELAN module is composed of an Extended Efficient Layer Aggregation Network (ELAN) and a Convolutional Block Attention Module (CBAM), such as Figure 2 shown.

[0079] The transition module performs a Concat operation on the output through a parallel structure consisting of a maximum pooling layer, two CBS modules with a convolution kernel size of 1 and a CBS module with a convolution kernel size of 3. The input feature map is divided, and the maximum value of each divided area is selected as the output. In the backbone network, the original image undergoes the above operation so that each CB-ELAN module of the backbone network obtains multi-scale features as input features of the Neck network part.

[0080] The Concat operation refers to concatenating multiple feature maps.

[0081] like Figure 3 As shown, the ELAN structure is as follows:

[0082] The ELAN module strengthens feature aggregation by strengthening the connection between different layers, and adopts a path optimization strategy to improve the efficiency of information flow in the network, thereby improving the network's feature extraction capabilities in complex scenarios.

[0083] The input feature map passes through two parallel convolution modules to obtain outputs X1 and X2, then X2 is used as the input of two serial convolution modules to obtain output X3, then X3 is used as the input of another two serial convolution modules to obtain output X4, and finally the four outputs are concat operated to obtain the final output of the module. The calculation formula is defined as follows:

[0084] X1=X2=Conv 3×3 (F1) (1)

[0085] X3=Comv 3×3 (Conv 3×3 (X2)) (2)

[0086] X4=Conv 3×3 (Conv 3×3 (X3)) (3)

[0087] F0=Conv 1×1 (x1+x2+x3+x4) (4)

[0088] like Figure 4 As shown, the network structure of the CBAM module is as follows:

[0089] The CBAM module consists of two sub-modules, namely the channel attention module and the spatial attention module.

[0090] The channel attention is composed of an adaptive maximum pooling layer, an adaptive average pooling layer, and an MLP: the adaptive maximum pooling layer automatically processes the size of the input feature map and pools it into a fixed-size feature map. By default, the adaptive pooling layer pools the input feature map to a size of 1x1. A convolution operation with a convolution kernel size of 1×1 and a step size of 1 is performed. The MLP refers to a multilayer perceptron: the role of the MLP in CBAM is to learn feature maps for each channel. Through MLP, CBAM can model and learn the correlation between different channels to extract more useful features. MLP can be implemented by stacking multiple fully connected layers, each of which uses a nonlinear activation function.

[0091] The spatial attention module is composed of a maximum pooling layer with a convolution kernel size of 7 and an average pooling layer with a convolution kernel size of 7 connected in series, and the corresponding feature map is obtained through a convolution operation with a convolution kernel size of 7 and a step size of 3.

[0092] In step 2.2, at the end of the backbone network in step 2.1, the output feature map of the fourth CB-ELAN module (AFEM) is used as the input of the global pyramid pooling layer (spatial pooling pyramid) of the Neck network.

[0093] The global pyramid pooling layer is composed of 5 CBS with a convolution kernel size of 1, two CBS with a convolution kernel size of 3, a maximum pooling layer with a convolution kernel size of 5, a maximum pooling layer with a convolution kernel size of 9, and a maximum pooling layer with a convolution kernel size of 12. By using the spatial pooling pyramid, feature maps of different scales can be obtained, which helps to detect objects at different scales and extract more expressive features.

[0094] The Neck network consists of 1 global pyramid pooling layer, 2 transition modules, 4 upsampling, 4 Concat operations, 2 CBS modules, 2 FGTC modules and 4 CB-ELAN modules (AFEM).

[0095] The output feature maps of the CB-ELAN in the last three layers of the backbone network are adjusted through several CBS modules and upsampling to adjust the corresponding number of channels and sizes, and then input into the Neck part layer by layer for multi-scale feature fusion to reduce the loss in the feature transmission process, and then upsampling operation is performed, namely FAUM, as shown in Figure 6 shown.

[0096] After performing maximum pooling and average pooling operations on the spliced ​​feature maps, the feature maps are spliced ​​using sub-pixel convolution with a magnification factor of 2, and the spliced ​​features are integrated using a convolution module with a convolution kernel size of 3×3.

[0097] The FAUM upsampling refers to reconstructing the feature map to restore a higher resolution.

[0098] The FAUM module uses sub-pixel convolution (SPC) to process higher-level feature maps F. i Get F i+1 , so that it is consistent with the lower-level feature map F i The size is consistent, SPC can effectively alleviate the information loss caused by channel dimensionality reduction. i+1 and F i The splicing operation is performed to integrate the feature information. The spliced ​​feature maps are subjected to maximum pooling and average pooling operations to generate a two-dimensional feature map F Max and F Avg , specifically expressed as:

[0099] F Max =Map(Concat[SPC μ＝2 (F i+1 )F i ) (5)

[0100] F Avg =Avp(ConCat[SPC μ＝2 (F i+1 )],F i ]) (6)

[0101] Where Map(·) and Avp(·) represent the maximum pooling and average pooling operations respectively, SPC μ＝2 (·) represents a sub-pixel convolution with a magnification factor of 2. The feature maps are then concatenated and the concatenated features are integrated using a convolution module with a convolution kernel size of 3×3:

[0102] F = δ(B(Conv 3×3 (Concat[F Max , F Avg ]))) (7)

[0103] In the formula, B(·) is batch normalization, and δ(·) is the ReLU activation function. Finally, the integrated features are activated to generate spatial weights, and the spatial weights are multiplied by the high-level features output by sub-pixel convolution, so that the higher-level features contain richer detailed feature information. The final output feature F U It is expressed as:

[0104]

[0105] Where Sig(·) is the Sigmoid activation function.

[0106] Step 2.3, after step 2.2, there are three output feature maps at the Neck part corresponding to the inputs of three detection heads of three different sizes, such as Figure 7 As shown, the detection head end includes three parts: feature extraction and activation, feature adjustment and indexing, and feature aggregation and output adjustment.

[0107] The input of the first layer passes through the ReLU function, 1×1 convolution, Sigmoid, and the maximum pooling layer, and then enters the second layer through a weighted sum merger.

[0108] The second layer input passes through Offset, 3×3 convolution, Saigmoid activation function, Index, and then output to enter the third layer.

[0109] The third layer input passes through the ReLU function, FC, maximum pooling layer, and Normalize, and finally adjusts the contribution of different features by weighted sum of different outputs.

[0110] Step 2.4, after deploying steps 2.1, 2.2, and 2.3, the original image is pre-processed and input into the backbone network. After passing through the first four CBS modules, the first AFEM (AFEM structure diagram is shown in Figure 2 The second AFEM is called the second stage, the third AFEM is called the third stage, and the fourth AFEM is called the fourth stage.

[0111] After the second and third stages, the output features are put into the FGTC module (the FGTC module structure diagram is as follows Figure 5 As shown in the figure, the feature information is enriched.

[0112] The output of the fourth stage is used as the input of the global pyramid pooling layer of the Neck network. After the pooling pyramid global pyramid pooling layer, its output is upsampled by the CBS module and AFUM to transform the image size and extract features. The output features are fused with the features of the output of the third stage after FGTC processing as the first fusion.

[0113] After passing through the first AFEM module of the Neck part, the CBS with a convolution kernel size of 1 and the AFUM operation, it is fused with the output of the second stage through FGTC, which is called the second fusion. The output passes through the second AFEM module again, undergoes another CBS, and then passes through LDyhead for large-scale prediction.

[0114] At the same time, the output of the second AFEM is fused with the output of the second fusion after passing through the transition module, which is called the third fusion. The output is fused with the output after passing through the third AFEM module and a transition module and the output after passing through the global pyramid pooling layer, which is called the fourth fusion. The output is then passed through the AFEM module and a layer of CBS and then through LDyhead for small-scale prediction.

[0115] The output of the third AFEM passes through a layer of CBS and enters LDyhead for mesoscale prediction. After all the above processes, the accuracy and loss of each round of training are calculated.

[0116] The overall process of step 2.4 changes as follows:

[0117] After deploying step 21, step 22, and step 23, the processed image is taken as input and first passes through the first CBS module of the backbone network with a convolution kernel size of 3. At this time, the image size becomes 640×640×32, and then passes through the CBS module with a convolution kernel size of 3 and a step size of 2. At this time, the image size becomes 320×320×64.

[0118] Repeat the above two CBS module processing processes. At this time, the image size becomes 160×160×128. At this time, the output is used as the first stage input, that is, the first AFEM module input. The output image size is 160×160×256, which is used as the input of the first MP.

[0119] The transition module consists of a maximum pooling layer and a CBS with a convolution kernel size of 1 as the first branch, a CBS module with a convolution kernel size of 1, and a CBS module with a convolution kernel size of 3 and a step size of 2 as the second branch, which are merged through a Concat operation, and the output is 80×80×256.

[0120] After the second AFEM module, i.e. the second stage, the output is 80×80×512.

[0121] After the second transition module, the image size becomes 40×40×512, and then serves as the input of the third stage, i.e., the third AFEM. The image output size of the third stage is 40×40×1024.

[0122] After passing through the third transition module, the output size is 20×20×1024. After passing through the fourth stage, that is, the fourth AFEM module, the image output size is 20×20×1024 output feature map, which is transmitted to the Neck part global pyramid pooling layer module, and the output feature map size is 20×20×512. It is then used as the input of the next CBS module, and the output feature map size is 20×20×256. The feature map is then input into the AFUM module, and the output feature map size is 40×40×256.

[0123] Then the output of the third stage is processed by an FGTC to output the features. The two output features are concat-operated for feature fusion, and the output feature map with a size of 40×40×512 is used as the input of the first AFEM module of the fifth stage, i.e., the Neck part, for feature multi-branch processing.

[0124] At this time, the output feature map size of the fifth stage is 40×40×256. After the output feature map is adjusted to the corresponding number of channels by the CBS module with a convolution kernel size of 1, the output feature map size is 40×40×128. The second AFUM processing is performed, and the output feature map is fused with the output of the second stage through the output of the FGTC module. At this time, the output feature map size is 80×80×256, and then input into the second AFEM module of the Neck part. The output size becomes 80×80×128. The output passes through a CBS module and is input into the LDyhead module as the input feature of the large-size detection head.

[0125] The above output features are then used as input and transmitted to the fourth transition module. The output feature map size is 40×40×256, and then it is fused with the output of the first AFEM module in the Neck part. After the output passes through the third AFEM module, the output feature size is 40×40×512. The output feature map is then transmitted to LDyhead after CBS as the input feature of the second detection head.

[0126] The output of the third AFEM is then passed through the fifth transition module and concat-operated with the output features of the global pyramid pooling layer. The size becomes 20×20×1024. After passing through a CBS, it is input into LDyhead as an input feature for the third detection head to complete object prediction, accuracy calculation, loss back propagation, update network parameters, and minimize the loss function as the optimization goal. The optimal parameter model is obtained and saved.

[0127] Table 1 shows the ablation experiment results of AMGTNet provided in the embodiment on the DOTAv1 test set using YOLOv7 as the baseline method.

[0128] Table 1

[0129] Baseline Method AFEM FGTC LDyhead AP% Parameter quantity (M) √ 94.0 37.2 √ √ 95.3 37.5 √ √ 94.8 39.7 √ √ 95.1 34.5 √ √ √ 95.5 40.8 √ √ √ 95.8 35.4 √ √ √ 95.3 36.3 √ √ √ √ 96.2 36.9

[0130] It can be seen from Table 1 that this method improves the detection accuracy of the network to varying degrees when the model parameters do not change much. Among them, based on the AFEM feature extraction module, a new backbone network was built for feature extraction, and its detection accuracy was improved by 1.3% compared with the baseline method. This is due to the integration of the convolutional attention module in the feature extraction module. This module can enhance the network's ability to focus on important parts of the image and suppress irrelevant background feature information while increasing only a small amount of parameters, so that the network can more effectively identify and process key information, and to a certain extent reduce the interference of complex background in remote sensing images on feature extraction. When FGTC is used alone, the model's detection accuracy is improved by 0.8% with almost no change in the number of parameters. This is mainly due to the fact that before feature fusion, FGTC provides the network with richer detail features by strengthening the edge and texture information of aircraft targets in the feature map and enhancing the specific features of the image. When LDyhead is introduced alone as the network detection head, the model detection accuracy is improved by 1.1%. This is because LDyhea uses trainable dynamic weights and three continuous attention mechanisms, which help the model better understand the contextual information in the image and effectively integrate the relationship between features at different levels and target scales, and reduce background information interference, so as to achieve accurate detection of multi-scale aircraft targets. Moreover, Dyhead does not need to set a high number of channels, and the number of parameters is reduced by 2.7M. Finally, all the above-mentioned contents are integrated into the network, and the model achieves 96.2% detection accuracy for aircraft targets, which is 2.2% higher than the baseline method. This ablation experiment fully verifies the effectiveness of the module proposed in this invention. At the same time, this method has high accuracy in detecting aircraft targets in remote sensing images.

[0131] Table 2 is the parameter comparison results of each category of AMGTNet provided by the embodiment on the remote sensing aircraft image part of the DOTAv1 dataset.

[0132] Table 2

[0133] method Enter the image size Backbone network AP% Parameter quantity (M) SCRDet 800*800 ResNet101 90.2 34.1 Faster RCNN 800*600 VGG16 77.2 41.4 FMSSD 640*640 VGG16 89.1 99.7 ASSD 512*512 VGG16 89.3 30.2 S2Anet 416*416 R-50-FPN 88.9 38.6 YOLOv3 416*416 DarkNet53 79 61.5 YOLOv4 416*416 CSPDarkNet53 88.5 63.7 YOLOv5s 640*640 CSPDarkNet53 91.5 7.1 CSL 800*800 ResNet152 90.3 37.4 FAOD 1024*1024 R-101-FPN 90.2 40.2 STBY 640*640 CSPDarkNet53 93.4 38.4 DFECANet 640*640 DFEMS 95.2 13.4 SCFNet 640*640 CSPDarkNet53 94.9 125 YOLOv7 640*640 ELANS 94 37.2 Ours 640*640 AFEMS 96.2 36.9

[0134] In order to further demonstrate the good detection performance of the method of the present invention for aircraft targets in remote sensing images, the algorithms with excellent performance in the field of remote sensing image target detection in recent years are introduced for comparative experiments, including CSL, FAOD, STBY, and YOLOv7. As shown in Table 2, the remote sensing image aircraft target detection method based on the attention mechanism of the present method has the highest detection accuracy on the DOTA dataset, reaching 96.2%.

[0135] When the model parameters are similar, the CLS method with ResNet152 as the main framework achieves a detection accuracy of 90.3% for aircraft targets, which is 5.9% lower than this method. This method solves the problem of boundary discontinuity in the regression-based rotating frame detector by introducing circular smooth label technology. CLS has a significant effect when processing large-sized targets, but the detection accuracy is limited when processing aircraft targets with variable scales; the FAOD method with R-101-FPN as the main framework has a detection accuracy of 90.2%, which is 6.0% lower than this method. This method uses a rotating frame to reduce the background area in the detection frame, but for aircraft targets, the aspect ratio is often not obvious, especially for small-sized targets. Therefore, the rotating frame strategy has limited ability to reduce the interference of the background area near the aircraft target, and the detection performance needs to be improved; the STBY method based on CSPDarkNet53 introduces Transformer to retain global context information, and its detection accuracy reaches 93.4%, which is 2.8% lower than this method; in addition, the detection accuracy of the baseline algorithm YOLOv7 is 94%, which is 2.2% lower than this method; this method is 1.0% higher than the remote sensing image aircraft target detection method based on DFECANet, but the model parameter volume of this method is 36.9M, which is higher than DFECANet's 13.4M, so this method is more suitable for application on fixed equipment with higher detection accuracy requirements. It can be seen from the above comparison algorithms that the method proposed in this chapter has a relatively higher detection accuracy for aircraft targets in the DOTAv1 dataset.

Claims

1. A target detection method based on attention mechanism and Gabor transform, characterized in that: The following steps are involved: Step 1: Obtain a remote sensing road image dataset, pre-process the dataset, and divide it into a training set and a test set according to a set ratio; Step 2: Build an object detection network based on the improved attention mechanism and Gabor transform, including the backbone network, Neck network and LDyhead module; The backbone network consists of 4 CBS modules, 4 AFEM modules and 3 transition modules; the Neck network consists of 1 global pyramid pooling layer, 2 transition modules, 4 upsampling, 4 Concat operations, 2 CBS modules, 2 FGTC modules and 4 CB-ELAN modules; Step 3: Input the remote sensing images of the training set into the target detection network for training, calculate the loss function and perform back propagation, update the network parameters, and obtain the optimal parameter model; Step 4: Input the test set into the optimal parameter model trained in step 3 and output the detection map of the remote sensing image.

2. The object detection method based on attention mechanism and Gabor transform according to claim 1, characterized in that: In step 2, we build a target detection network, including: Build a feature extraction network. After processing by four CBS modules, introduce the attention feature extraction module AFEM to construct a feature extraction network in the form of a feature pyramid to obtain feature maps at different levels. At the same time, the network also contains three transition modules for dimensionality reduction and feature compression. Build a feature fusion network and perform downsampling. In the feature fusion part, use the FGTC module to enhance the feature information obtained in the feature extraction network, and use the FAUM module in the upsampling part. After the entire feature fusion network passes through the global pyramid pooling layer, transition module, and CB-ELAN module, it continuously outputs three feature maps to the LDyhead module through three output terminals; Build the LDyhead detection head. The output feature maps of the three output terminals correspond to the inputs of the three detection heads of three different sizes. The detection head includes a scale-aware attention module and a spatial-aware attention module. Through several convolution and maximum pooling layer operations, the features are extracted, adjusted and aggregated. Connect the feature extraction network, feature fusion network and LDyhead detection head; After the original image undergoes preliminary feature extraction through the CBS module of the feature extraction network, it is further enhanced through the AFEM module, pyramid pooling is performed through the global pyramid pooling layer module of the feature fusion network, and feature fusion and conversion are performed through the FGTC, FAUM, and transition modules. It then enters the LDyhead module for final prediction and outputs the location, category, and confidence of the detection box.

3. The target detection method based on attention mechanism and Gabor transform according to claim 2, characterized in that: The feature extraction network first passes through 4 CBS modules, then passes through 4 attention feature extraction modules AFEM, and then passes through 3 transition modules for output, specifically: The CBS module consists of a convolutional layer, normalization, and a silu activation function to perform preliminary extraction of image features and size transformation of the image; The convolution kernel size of the first CBS module and the fourth CBS module is 3, and the step size is 1. The convolution kernel size of the second CBS module and the third CBS module is 3, and the step size is 2. The attention feature extraction module consists of an ELAN module and a CBAM module; The ELAN module is an improved efficient network structure. The input feature map is passed through two parallel convolution modules to obtain outputs X1 and X2. X2 is used as the input of two serial convolution modules to obtain output X3. X3 is used as the input of another two serial convolution modules to obtain output X4. Finally, the four outputs are merged to obtain the final output F0 of the ELAN module. The calculation formula is defined as follows: X1=X2=Conv 3×3 (F1) X3=Conv 3×3 (Conv 3×3 (X2)) X4=Conv 3×3 (Conv 3×3 (X3)) F0=Conv 1×1 (X1+X2+X3+X4) The CBAM module includes two sub-modules, namely a channel attention module and a spatial attention module; The channel attention module is composed of an adaptive maximum pooling layer, an adaptive average pooling layer and an MLP. The adaptive maximum pooling layer automatically processes the size of the input feature map and pools it into a fixed-size feature map; The MLP is a multi-layer perceptron, implemented by stacking multiple fully connected layers, each of which uses a non-linear activation function; The spatial attention module is composed of a maximum pooling layer with a convolution kernel size of 7 and an average pooling layer with a convolution kernel size of 7 connected in series, and the corresponding feature map is obtained by a convolution operation with a convolution kernel size of 7 and a step size of 3; The transition module performs a merging operation by dividing the input feature map and selecting the maximum value of each divided area as the output. The transition module is composed of a maximum pooling layer, and a parallel structure consisting of two CBS modules with a convolution kernel size of 1 and a CBS module with a convolution kernel size of 3.

4. The object detection method based on attention mechanism and Gabor transform according to claim 3, characterized in that: The channel attention mechanism module includes the following operations: First, perform global average pooling and global maximum pooling operations on the collected feature maps to obtain two channel dimension feature vectors and obtain global feature information; Then, the acquired feature information is passed through a fully connected network with shared weights, which contains two fully connected layers and ReLU activation function to generate two channel attention feature maps; Finally, the two feature maps are element-wise added and passed through a Sigmoid activation function to obtain the final channel attention weight, which is used to adjust each channel of the original feature map, as shown in the following formula: M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F))) Where M c (F) represents the channel attention output feature map, F represents the input feature map, AvgPool() and MaxPool() represent average pooling and maximum pooling operations, MLP() represents the shared weight operation, and σ represents the Sigmoid activation function.

5. The target detection method based on attention mechanism and Gabor transform according to claim 3 or 4, characterized in that: The spatial attention mechanism module includes the following operations: First, use the global average pooling operation and the global maximum pooling operation to obtain two channel dimension feature vectors respectively to obtain global feature information; Then, the acquired feature information is passed through a fully connected network with shared weights, which contains two fully connected layers and ReLU activation function to generate two channel attention feature maps; Next, these two feature maps are stacked in the channel dimension and processed through a convolution layer with a convolution kernel size of 7×7 to capture contextual information in the spatial dimension; Finally, the spatial attention map is generated by the Sigmoid activation function, and the attention map is element-wise multiplied with the original feature map, as shown in the following formula: M s (F)=σ(f 7×7 ([AvgPool(F);MaxPool(F)])) Where M s (F) represents the spatial attention output feature map, F represents the input feature map, f 7×7 () represents a convolution operation with a convolution kernel size of 7×7, AvgPool() and MaxPool() represent average pooling and maximum pooling operations, and σ represents the Sigmoid activation function.

6. The object detection method based on attention mechanism and Gabor transform according to claim 2, characterized in that: The FGTC module has the following network structure: First, a random initialization is performed and input into the FGF module, and then after passing through a two-dimensional convolution layer, a ReLU activation function, Dropout, Padding and Maxpool, the final output is obtained; the two-dimensional convolution layer uses a 3×3 convolution kernel; The FGTC module adopts a fractional-order Gabor filter; The FAUM module uses sub-pixel convolution to process the high-level feature map F i Get F i+1 , so that it is consistent with the low-level feature map F i The size is the same; then F i+1 and F i Perform splicing operations to integrate feature information; perform maximum pooling and average pooling operations on the spliced ​​feature maps to generate a two-dimensional feature map F Max and F Avg ; Finally, the integrated features are activated to generate spatial weights, and the spatial weights are combined with the high-level features F output by sub-pixel convolution. i+1 Multiply and detect the output features of the image.

7. The object detection method based on attention mechanism and Gabor transform according to claim 2, characterized in that: The LDyhead module is divided into three layers of input and output; The first layer input passes through the ReLU function, 1×1 convolution, Saigmoid activation function, and the maximum pooling layer, and then enters the second layer through a weighted sum merger; The second layer input passes through Offset, 3×3 convolution, Saigmoid activation function, Index and then output to the third layer; The third layer input passes through the ReLU function, FC, maximum pooling layer, and Normalize, and finally the outputs of the above three layers are weighted summed.

Images