A method for aircraft target detection in remote sensing images based on asymmetric convolution module

Abstract

The present invention relates to the technical field of remote sensing image processing, and discloses a remote sensing image aircraft target detection method based on an asymmetric convolution module, comprising the following steps: S1: constructing a target detection model based on an asymmetric convolution module and training it; the asymmetric convolution module comprises a normalization module, a first multi-layer perception layer, a convolution module, a feature splicing module and a second multi-layer perception layer connected in sequence; the convolution module is used to extract features using a plurality of sub-convolution modules; and is used to extract the morphological features of the aircraft target. S2: using the trained target detection model to process the collected remote sensing image to obtain the aircraft target; the present invention introduces aircraft morphological prior information through an asymmetric convolution module, and further enhances the multi-scale feature extraction capability of the model, so as to correctly identify and locate potential aircraft targets in remote sensing images.

Description

Technical Field

[0001] The invention relates to the technical field of remote sensing image processing, and more particularly to a remote sensing image aircraft target detection method based on an asymmetric convolution module. Background Art

[0002] Remote sensing image target detection has broad application prospects in environmental monitoring, urban planning and other fields. The detection technology for remote sensing image aircraft targets has always been one of the research focuses in the field of remote sensing. The current mainstream target detection method is based on convolutional neural networks, which mainly include feature extraction network, feature fusion network and detection head. The feature extraction network extracts multiple feature maps of packets at different levels based on the convolution operator. The feature fusion network is used to integrate information from feature maps at different levels to assist prediction. The detection head decouples the feature map to output prediction information, and its feature processing capability greatly affects the accuracy of the prediction information.

[0003] Remote sensing images have complex imaging backgrounds and dramatic changes in target scale, which makes target detection tasks extremely difficult and challenging. The detection head structure of conventional target detection models is insufficient in feature refinement and cannot achieve high-quality feature decoupling to accurately output the model's prediction information on the location and category of aircraft targets in remote sensing images.

[0004] Therefore, how to improve the detection accuracy of aircraft targets in remote sensing images is an urgent problem that technicians in this field need to solve. Summary of the invention

[0005] In view of this, the present invention provides a remote sensing image aircraft target detection method based on an asymmetric convolution module, which introduces aircraft morphological prior information through the asymmetric convolution module, and further improves the multi-scale feature extraction capability of the model to correctly identify and locate potential aircraft targets in remote sensing images.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A remote sensing image aircraft target detection method based on an asymmetric convolution module comprises the following steps:

[0008] S1: Constructing a target detection model based on an asymmetric convolution module and performing training; the asymmetric convolution module comprises a normalization module, a first multi-layer perception layer, a convolution module, a feature splicing module and a second multi-layer perception layer connected in sequence; the convolution module is used for extracting features using multiple sub-convolution modules; and is used for extracting morphological features of the aircraft target.

[0009] S2: Use the trained target detection model to process the collected remote sensing images to obtain the aircraft target.

[0010] Preferably, the convolution module includes a cross convolution module and a slanted cross convolution module, and the cross convolution module and the slanted cross convolution module respectively convolve the feature map output by the first multi-layer perceptron to extract features that conform to the morphological characteristics of the aircraft.

[0011] Preferably, the convolution module also includes a common N×N convolution module and a point convolution module, which are used to extract features respectively.

[0012] Preferably, the target detection model includes a feature extraction network, a feature fusion network and a detection head network which are connected in sequence.

[0013] Preferably, the feature extraction network includes a basic convolution module and the asymmetric convolution module, and deep features are extracted by multiple basic convolution modules or asymmetric convolution modules.

[0014] Preferably, the feature fusion network is used to extract features of multiple levels in the feature extraction network, perform convolution, and then splice them with the original level features. When the original level features are lower-level features, after splicing with the original level features, they are also spliced ​​with the reshaped upper-level splicing results for a second time; and finally, the splicing features corresponding to each level are output.

[0015] Preferably, the feature fusion network is used to extract features of multiple levels in the feature extraction network, perform convolution and then splice them with the original level features; when the original level features are lower-level features, after splicing with the original level features, they are also spliced ​​again with the reshaped upper-level splicing results; finally, the splicing features corresponding to each level are output.

[0016] Preferably, the detection head network is used to process the features output by the feature fusion network and perform classification regression to generate target category and position information.

[0017] Preferably, the detection head network processes the features output by the feature fusion network, specifically including: performing basic convolution and asymmetric convolution on the original features respectively, splicing the convolution results, and inputting them into a multi-layer perceptron.

[0018] Preferably, when training the target detection model, the loss function is:

[0019]

[0020] in, represents the prediction box, represents the true label, Represents the focal loss, which is used to calculate the prediction box in each classification prediction branch The class probability and the true label The difference between Represents the rotation box IoU loss function, which is used to calculate the predicted box The target ground truth label corresponding to the input remote sensing image distance; Then represents the total loss used for network training.

[0021] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a remote sensing image aircraft target detection method based on an asymmetric convolution module, which uses an asymmetric convolution module to introduce morphological prior information to design a convolution structure, and further enhances the multi-scale feature extraction capability of the model to correctly identify and locate potential aircraft targets in remote sensing images. And it enables the detection head network to finely process the feature information of remote sensing image aircraft targets in the feature decoupling stage, fully tapping its potential local feature expression in the spatial dimension and channel dimension, effectively solving the problem of insufficient feature decoupling capability of the current aircraft target detection method, and thus realizing accurate remote sensing image aircraft target detection task recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0023] Figure 1 Schematic diagram of the structure of the target detection model in an embodiment of the present invention.

[0024] Figure 2 Schematic diagram of the structure of an asymmetric convolution module in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] like Figure 1 and Figure 2 The embodiment of the present invention discloses a remote sensing image aircraft target detection method based on an asymmetric convolution module, comprising the following steps:

[0027] S1: Construct a target detection model based on an asymmetric convolution module and perform training; the asymmetric convolution module includes a normalization module, a first multi-layer perception layer, a convolution module, a feature splicing module and a second multi-layer perception layer connected in sequence; the convolution module is used to extract features using multiple sub-convolution modules; and is used to extract morphological features of aircraft targets.

[0028] S2: Use the trained target detection model to process the collected remote sensing images to obtain the aircraft target.

[0029] In one implementation, in the target detection model, an asymmetric convolution module is used to extract features in the remote sensing image, and the aircraft target is obtained through a classification regression network. In which, deep feature extraction can be achieved through multiple overlapping asymmetric convolution modules, and feature enhancement can be achieved through feature processing, and then the enhanced features are used for classification regression tasks.

[0030] In order to further implement the above technical scheme, in order to better perceive the characteristics of aircraft targets in remote sensing images, and then perform accurate positioning and classification, the convolution structure in the asymmetric convolution module is designed based on the morphological characteristics of the aircraft. The convolution module includes a cross convolution module and an oblique cross convolution module. The cross convolution module and the oblique cross convolution module respectively convolve the feature map output by the first multi-layer perceptron to extract features that conform to the morphological characteristics of the aircraft. In addition, it also includes an ordinary N×N convolution module and a point convolution module to extract features respectively.

[0031] The feature extraction steps of the asymmetric convolution module are as follows:

[0032] First, the input feature map passes through the normalization module, and the multi-layer perceptron extracts the features of the input feature map. The calculation process can be formally expressed as:

[0033]

[0034] in, is the input feature map, is the output feature map. It represents the normalization module, which is used to preprocess the feature map data to normalize it to the standard feature space, thereby eliminating the adverse effects caused by singular sample data. 1 represents the No. 1 multilayer perceptron module, and the number of neurons in its input layer is Number of channels , the number of neurons in the hidden layer is 10× , the number of neurons in the output layer is 2× Then, the asymmetric combined convolution module extracts features of different dimensions from the feature map. Its purpose is to mine the feature information of the feature map in the spatial dimension and channel dimension. The calculation process can be formally expressed as:

[0035]

[0036]

[0037]

[0038]

[0039] in, Represents ordinary 3*3 convolution, Represents point convolution. Both are conventional convolution structures used to extract feature information of feature maps. represents cross convolution, Represents oblique cross convolution. Both are asymmetric convolution structures, which are designed based on the morphological characteristics of aircraft targets in remote sensing images and can better perceive and extract target features. , It is the feature map extracted by deep separation convolution, and then the normalization module is used to standardize the feature map for convenience of calculation. The activation function is used to introduce nonlinear information into the feature map to help the model distinguish the features of the potential target area. The calculation process can be formally expressed as:

[0040]

[0041]

[0042]

[0043]

[0044] , is the output feature map, represents the activation function, which is used to introduce nonlinear information into the feature map to help the model distinguish the features of the potential target area. Furthermore, the feature concatenation operation is used to integrate the feature information of different branches, and its calculation process can be formally expressed as:

[0045]

[0046] represents the feature normalization operation, Represents the activation function operation. Represents the feature map obtained after calculation, and its shape is the same as the input feature map Remain consistent, but the number of channels is 4 times. Furthermore, the multilayer perceptron is used to adjust the dimension of the output feature map to facilitate the subsequent feature fusion process. Its calculation process can be formally expressed as:

[0047]

[0048] in, Represents multilayer perceptron No. 2, whose input layer is neurons, and the output layer is A neuron. It is the feature map after feature extraction by the asymmetric convolution module.

[0049] In order to further implement the above technical solution, the target detection model includes a feature extraction network, a feature fusion network and a detection head network connected in sequence. The feature extraction network is used as the backbone network for deep feature extraction of remote sensing images. The traditional feature extraction network is composed of multiple stacked basic convolution modules connected in sequence. Each basic convolution module contains a basic convolution operator (Conv), a normalization module (Batch Normalization) and an activation function (Relu). The basic convolution operator calculates the channel information of the input remote sensing image at a pixel level to achieve the purpose of feature extraction. The normalization module normalizes the calculated channel results to the standard range to facilitate subsequent calculations while avoiding abnormal problems in gradient optimization. The activation function activates and inhibits the feature matrix of the potential target area to help the model distinguish between foreground and background information. The calculation process of the operation layer of each module can be formally expressed as:

[0050]

[0051] in represents the basic convolution operation, is a normalization module, which keeps the number of channels it receives and outputs features unchanged. To input the original remote sensing image, It is the output feature map obtained after feature extraction. The activation function is responsible for activating the potential target area of ​​the output feature map. Definition It is a convolution module that includes the basic convolution operator, normalization module and activation function. Therefore, the deep feature extraction process of the deep neural network can be formally expressed as:

[0052]

[0053] in represents the convolution operation of the nth layer, It is a multi-level feature map extracted by ordinary convolution operator. Based on this, we can obtain n feature maps of remote sensing images after deep feature extraction.

[0054] In this embodiment, the present invention replaces the basic convolution module in the traditional feature extraction network with the asymmetric convolution module in the present invention to improve the detection accuracy of the aircraft target.

[0055] Furthermore, the feature fusion network is used to receive the feature map of the backbone network for fusion to obtain the multi-scale characteristic information of the potential target. The feature fusion network can extract multiple levels of features in the feature extraction network, perform convolution, and then splice them with the original level features; when the original level features are lower-level features, after splicing with the original level features, they are also spliced ​​with the reshaped upper level splicing results for a second time; and output the splicing features corresponding to each level to the detection head network.

[0056] Specifically, the backbone feature extraction network The extracted feature map Pass it to the feature fusion network for feature fusion; the calculation process of the feature fusion network is expressed as:

[0057]

[0058] in, It is a feature fusion network used for feature fusion. It first receives the feature map of the backbone feature extraction network. Furthermore, the shape of the feature map is changed by the convolution operator to achieve fusion to obtain the fused feature map , the calculation process of the feature fusion network is expressed as:

[0059]

[0060]

[0061]

[0062] in, represents the feature concatenation operator, represents the convolution operator, Represents the feature reshaping operator, which is used to adjust the feature dimension in preparation for concatenation.

[0063] Furthermore, the detection head network is used to decouple the features output by the feature fusion network, and perform classification regression after processing the features output by the feature fusion network to generate target category and location information. After obtaining the original features output by the feature fusion network, the asymmetric convolution module is introduced again, and the original features are respectively subjected to basic convolution and asymmetric convolution, and the convolution results are spliced ​​and input into the multi-layer perceptron to achieve classification regression.

[0064] The specific steps include: First, the convolution module is used to receive the input remote sensing image features and process them in appropriate shapes and channel dimensions for subsequent calculations. The asymmetric convolution module is used to obtain high-quality feature information of aircraft targets in remote sensing images to assist prediction. Feature splicing is used to integrate feature maps from different branches, and its calculation process is expressed as:

[0065]

[0066] in, represents the asymmetric convolution module in step 1, Represents the output feature map. Then the multi-layer perceptron is used to adjust the number of channels of the feature map and extract high-value feature information.

[0067]

[0068] Represents multilayer perceptron No. 3, the number of neurons in its input layer is Number of channels 4 , the number of neurons in the hidden layer is 10× , the number of neurons in the output layer is Finally, the prediction function is used to predict the location of the potential target area of ​​the feature map, that is, the category information, and output the prediction result. The calculation process of the detection head is expressed as:

[0069]

[0070]

[0071] in, Classification information obtained for prediction; Location information obtained for prediction; The feature map output by the feature fusion network; is the classification function, is the rectangular box prediction function.

[0072] In order to further implement the above technical solution, after building a remote sensing image aircraft target detection model combined with an asymmetric convolution module, the network is trained for remote sensing image target detection tasks using focal loss and rotated IoU loss, which can be formally expressed as:

[0073]

[0074] in, represents the prediction box, represents the true label, Represents the focal loss, which is used to calculate the prediction box in each classification prediction branch The class probability and the true label The difference between Represents the rotation box IoU loss function, which is used to calculate the predicted box The target ground truth label corresponding to the input remote sensing image distance; It indicates the total loss used in network training. When the network is trained until the loss no longer decreases, it means that the network training has reached stability and the training process is completed, and the trained target detection model combined with the asymmetric convolution module is obtained.

[0075] Furthermore, the trained remote sensing image aircraft target detection model combined with the asymmetric convolution module is used to test the remote sensing image aircraft target data, which is formally expressed as:

[0076]

[0077] in, It represents the trained remote sensing image aircraft target detection model combined with asymmetric convolution module. and They respectively represent the remote sensing image aircraft target data to be tested and the corresponding target detection results.

[0078] In order to realize the data input of the target model, the calculation process of the data loader is expressed as:

[0079]

[0080] in, Indicates the size ,Input remote sensing image with 3 channels; Annotation information for the target in the image, including location and category information; It is a data loader function used to load data and data augmentation.

[0081] In order to further implement the above technical solution, a variety of general backbone feature extraction networks can be adapted through simple model replacement. Typical backbone feature extraction networks include: ResNet, VGG, EfficientNet, etc.

[0082] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0083] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A remote sensing image aircraft target detection method based on an asymmetric convolution module, characterized in that: The following steps are involved: S1: Build an object detection model based on an asymmetric convolution module and train it; The target detection model includes a feature extraction network, a feature fusion network and a detection head network connected in sequence; The feature extraction network includes a basic convolution module and an asymmetric convolution module, and a plurality of basic convolution modules or asymmetric convolution modules are used to extract deep features; the asymmetric convolution module includes a normalization module, a first multi-layer perceptron, a convolution module, a feature splicing module and a second multi-layer perceptron connected in sequence; the convolution module includes a cross convolution module and an oblique cross convolution module, and the cross convolution module and the oblique cross convolution module respectively convolve the feature map output by the first multi-layer perceptron to extract features that meet the morphological characteristics of the aircraft; The detection head network is used to perform basic convolution and asymmetric convolution on the features output by the feature fusion network, and then splice the convolution results and incorporate them into the multi-layer perceptron for classification and regression to generate target category and position information; S2: Use the trained target detection model to process the collected remote sensing images to obtain the aircraft target.

2. The method for detecting aircraft targets in remote sensing images based on an asymmetric convolution module according to claim 1, characterized in that: The convolution module also includes a common N×N convolution module and a point convolution module, which are used to extract features respectively.

3. The method for detecting aircraft targets in remote sensing images based on an asymmetric convolution module according to claim 1, characterized in that: The feature fusion network is used to extract features of multiple levels in the feature extraction network and then convolve them with the original level features; When the original level feature is a lower level feature, after being spliced ​​with the original level feature, it is also spliced ​​again with the reshaped upper level splicing result; Output the corresponding splicing features of each level.

4. The method for detecting aircraft targets in remote sensing images based on an asymmetric convolution module according to claim 1, characterized in that: When training the target detection model, the loss function is: Among them, y represents the prediction box, Represents the true label, FocalLoss represents the focal loss, which is used to calculate the category probability and true label of the prediction box y in each classification prediction branch The difference between them; RotatedIoULoss represents the rotation box IoU loss function, which is used to calculate the target truth label corresponding to the predicted box y and the input remote sensing image Loss represents the total loss used in network training.

Images