A multi-scene aircraft target detection method
By cropping and augmenting remote sensing image datasets, combining response spectrum analysis and multi-stage jump connection network, the problem of poor generalization ability of remote sensing aircraft target detection in different scenarios is solved, the recognition accuracy and robustness are improved, and the more Efficient aircraft target detection.
Patent Information
- Application Number
- CN202210395544.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-15
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-04-15
AI Technical Summary
Remote sensing aircraft target detection has poor generalization ability under different climates and lighting conditions, low recognition accuracy, and it is difficult to improve aircraft target detection capabilities.
By cropping high-resolution satellite remote sensing images, we can produce aircraft target data sets, perform geometric augmentation and response spectrum analysis, obtain channel increment prior knowledge, and combine multi-scene sample augmentation technology to train a deep-level residual network with multi-stage jump connections. , Bilinear interpolation and non-maximum suppression algorithm are used for aircraft target detection.
It improves the generalization ability and recognition accuracy of deep learning models in different scenarios, enhances the robustness and computing efficiency of aircraft target detection, and reduces computing overhead.
Smart Images

Figure CN114724050B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a multi-scene aircraft target detection method. Background Art
[0002] The detection of aircraft targets plays a vital role in the dynamic monitoring of airports and the search for aircraft. As people travel more and more frequently, aircraft, as the main carrier tool, have become an indispensable tool for long-distance travel and business trips. The positioning of aircraft generally relies on the positioning system in the black box. After a failure, there is a loss of connection. The positioning of aircraft targets based on satellite remote sensing technology has the characteristics of standing high and seeing far, and can be used as a means of assisting in tracking aircraft. The Gaofen-2 satellite is a satellite independently developed by my country. It has the characteristics of high resolution, multi-spectral bands, and wide coverage, which provides the possibility for accurate aircraft target detection.
[0003] In recent years, great progress has been made in the research of aircraft target detection based on remote sensing satellites. The saliency map is used to highlight the target, and the connected regions are found based on region growing and linear labeling algorithms to determine the number and location of candidate targets; the ratio of standard deviation and mean is combined to extract feature descriptors to evaluate the stability of features, and then the extracted features are combined into feature vectors to complete the recognition using support vector machines (SVM). Transfer learning methods can be used to effectively construct deep networks when labeled samples are scarce. Convolutional neural networks are used to detect and recognize aircraft targets in remote sensing images, and good results are achieved in the constructed small sample data set. Since deep learning methods have good generalization capabilities for massive data, they have become the mainstream method for extracting target features. However, remote sensing aircraft target detection is affected by climate, lighting, etc., and the target features vary greatly. Therefore, for aircraft remote sensing images, how to improve the generalization ability and recognition accuracy of deep learning methods in different scenarios, and then improve the ability to detect aircraft targets, is a technical problem that needs to be solved urgently. Summary of the invention
[0004] The purpose of the present invention is to provide a multi-scenario aircraft target detection method, which can improve the generalization ability and recognition accuracy of deep learning methods in different scenarios, thereby improving the aircraft target detection capability.
[0005] The present invention provides a multi-scenario aircraft target detection method, comprising: cutting aircraft target images according to high-resolution satellite remote sensing images to produce an aircraft target data set, and geometrically augmenting the aircraft target images in the data set to obtain multiple sample images; wherein the data set includes aircraft target coordinate information; obtaining a response spectrum of the center point of the aircraft target according to the multiple sample images and the aircraft target coordinate information; obtaining a maximum value and a minimum value of the response spectrum according to the response spectra of all sample images in the aircraft target data set, and calculating a channel increment prior knowledge of the maximum value and the minimum value of the response spectrum and the response spectrum of the center point of the aircraft target in each sample image according to the maximum value and the minimum value of the response spectrum; and obtaining a channel increment prior knowledge of the maximum value and the minimum value of the response spectrum according to the maximum value and the minimum value of the response spectrum according to the maximum value and the minimum value of the response spectrum. The sample expansion technology based on the prior knowledge of the aircraft center point is combined with the channel incremental prior knowledge and multiple sample images to perform multi-scene sample augmentation of the aircraft target, and obtain multiple aircraft target samples; the multiple aircraft target samples and the aircraft target coordinate information are input into the target detection model for training, wherein the target detection model uses a deep residual network with multi-level jump connections to fuse the multi-level extracted aircraft target features to obtain a feature map with high semantic relations; the feature map is intercepted by bilinear interpolation in combination with the aircraft target anchor frame, and the position of the aircraft target anchor frame is inferred through the neural network, and the overlapping aircraft target anchor frame position coordinates are eliminated in combination with the non-maximum suppression algorithm, and finally the coordinates of the aircraft target are drawn on the remote sensing image.
[0006] In a possible example, cropping an aircraft target image based on a high-resolution satellite image to produce an aircraft target data set includes: selecting multiple multi-scene high-resolution remote sensing images with different lighting and climate, dividing each remote sensing image into 416*416 grid blocks, calculating the grid block position corresponding to the aircraft target, cropping the grid block corresponding to the aircraft target, and recording the relative coordinates of the aircraft target on the grid block to generate a data set of aircraft target images.
[0007] In a possible example, cropping an aircraft target image according to a high-resolution satellite image to prepare an aircraft target data set further includes: normalizing the aircraft target image in the data set; wherein the normalization calculation formula is as follows:
[0008]
[0009] Where x represents the image, x′ represents the normalized image, min(x) represents the minimum value of the training image, and max(x) represents the maximum value of the training image.
[0010] In a possible example, after normalizing the data in the data set, the following further includes: enlarging the aircraft target image by 255 times, and converting the floating point data type into integer type storage.
[0011] In a possible example, geometrically augmenting an airplane target image in a data set to obtain a plurality of sample images includes: geometrically augmenting the airplane target image by rotating it at random angles, mirroring it horizontally and vertically, and randomly scaling it.
[0012] In a possible example, obtaining the response spectrum of the center point of the aircraft target according to multiple sample images and aircraft target coordinate information includes: obtaining multiple high-resolution remote sensing images of multiple scenes with different illumination and different climates, calculating the value of the image corresponding to the center of the aircraft target anchor frame according to the aircraft target position, and recording it as Where i is the image channel index and j is the image index.
[0013] In a possible example, the maximum and minimum values of the response spectrum are obtained according to the response spectrum of all sample images in the aircraft target data set, and the channel increment between the maximum and minimum values of the response spectrum and the response spectrum of the center point of the aircraft target in each sample image is calculated according to the maximum and minimum values of the response spectrum. Prior knowledge includes: statistics of all The maximum value of each channel i and the minimum value Min i , as prior knowledge, a training sample image I i The average value of each channel of all anchor box center points is recorded as V i ; The following augmentation formulas for different scenarios are used to calculate the channel increment prior knowledge:
[0014] A i =V i -Min i
[0015] B i =Max i -V i
[0016] I=I i +Rand(A i , B i )
[0017] Among them, A i is the difference between the minimum spectral response and the average of the training samples, B i is the difference between the maximum limit of spectral response and the average of training samples, and I is the training sample in A i With B i Random scene augmentation images between.
[0018] In a possible example, inputting multiple aircraft target samples and aircraft target coordinate information into a target detection model for training includes: the training process uses a gradient descent method of a neural network to solve the optimal solution for target classification and target anchor frame position regression tasks.
[0019] In a possible example, the target detection model is based on a deep residual network with multi-level skip connections to fuse the multi-level extracted aircraft target features to obtain a feature map with high semantic relationships, including: dividing the network model backbone network into 5 large convolution blocks, namely Layer_1, Layer_2, Layer_3, Layer_4, and Layer_5. The Layer_1 convolution block consists of a 7×7 convolution kernel, batch normalization and ReLU activation function, and a pooling layer. The remaining convolution blocks are composed of a 1×1 convolution kernel, a 3×3 convolution kernel, a 1×1 convolution kernel, and a stacked skip connection residual. Each convolution block is connected by a skip connection residual; the candidate area adopts an RPN network , extract features according to the backbone network, calculate the target candidate box, and use bilinear interpolation to obtain the aircraft features according to the candidate box position; the channel attention mechanism combines the aircraft feature F with the average pooling and the maximum pooling, and calculates the channel attention weight of each feature map according to the feature channel dimension, and applies the fused weight to the extracted feature map to obtain Fc; the spatial attention mechanism calculates the average pooling and the maximum pooling for each point corresponding to the horizontal channel of the feature map Fc from the spatial dimension, splices the calculated results, and uses a 1×1 convolution to obtain the spatial feature map attention weight, and applies the calculated spatial attention weight to the feature map Fc to obtain the feature map Fs.
[0020] In a possible example, the feature map is cropped, and the overlapping coordinate information is eliminated by combining the non-maximum suppression algorithm, and finally the coordinates of the aircraft target are drawn on the remote sensing image, including: using a sliding window method, setting an overlap value, and interlaced cropping of the remote sensing image; using the cropped remote sensing image as a model input to detect the longitude and latitude results; filtering the overlapping targets according to the non-maximum suppression algorithm, and finally drawing the longitude and latitude coordinates of the aircraft target in the remote sensing image.
[0021] The present invention provides a multi-scenario aircraft target detection method. Aiming at the problem that aircraft target detection is affected by climate and illumination, a method for augmenting aircraft target samples by combining prior knowledge of aircraft targets under different climate and illumination conditions is proposed. The method expands the aircraft target samples and improves the generalization ability of the deep learning model. In addition, the frequent scheduling between the computer disk and memory is reduced, and the scheduling overhead is reduced. In addition, based on the Faster RCNN model, the backbone network and target feature channel including deep residual jump connection and spatial attention are improved to enhance the ability to extract aircraft target features. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0023] Figure 1 A schematic diagram of a method flow chart of an embodiment of the present invention;
[0024] Figure 2 This is a schematic diagram of the sample augmentation process for different climate and lighting scenarios in an embodiment of the present invention;
[0025] Figure 3 This is a schematic diagram of a network structure improved based on Faster RCNN in an embodiment of the present invention;
[0026] Figure 4 Schematic diagram of the aircraft target detection effect in an embodiment of the present invention. DETAILED DESCRIPTION
[0027] In order to make the above and other features and advantages of the present invention more clear, the present invention is further described below in conjunction with the accompanying drawings. It should be understood that the specific embodiments given herein are for the purpose of explaining to those skilled in the art and are only exemplary and not restrictive.
[0028] In the following description, many specific details are set forth to provide a thorough understanding of the present invention. However, it will be apparent to one of ordinary skill in the art that specific details need not be adopted to practice the present invention. In other cases, well-known steps or operations are not described in detail to avoid obscuring the present invention.
[0029] The detection of aircraft targets plays a vital role in the dynamic monitoring of airports and the search for aircraft. In recent years, the remote sensing satellite aircraft target detection technology has also made great progress. Since deep learning methods have good generalization ability for massive data, they have also become the mainstream method for extracting target features. However, remote sensing aircraft target detection is affected by climate, lighting, etc., and the target features vary greatly. For aircraft remote sensing images, deep learning methods have poor generalization ability in different scenarios, low recognition accuracy, and relatively poor aircraft target detection capabilities.
[0030] Therefore, the present invention provides a multi-scenario aircraft target detection method, such as Figure 1 As shown, including:
[0031] Step S1, cropping aircraft target images according to high-resolution satellite remote sensing images to produce an aircraft target data set.
[0032] In one example, multiple high-resolution remote sensing images of multiple scenes with different lighting and climate are selected, each remote sensing image is divided into 416*416 grid blocks, the grid block position corresponding to the aircraft target is calculated, the grid block corresponding to the aircraft target is cropped, and the relative coordinates of the aircraft target on the grid block are recorded to generate a data set of aircraft target images.
[0033] Step S2, geometrically augmenting the aircraft target image in the data set to obtain multiple sample images; wherein the data set includes aircraft target coordinate information.
[0034] In one example, the aircraft target image in the data set is first normalized. Then the aircraft target image is enlarged 255 times and the floating point data type is converted to integer storage; the normalization calculation formula is as follows:
[0035]
[0036] Where x represents the image, x′ represents the normalized image, min(x) represents the minimum value of the training image, and max(x) represents the maximum value of the training image.
[0037] The geometric augmentation of the aircraft target image in the data set is performed by rotating the aircraft target image at random angles, mirroring it horizontally and vertically, and performing random scaling geometric transformations.
[0038] In a preferred embodiment, the step of preparing the aircraft target data set includes:
[0039] Step S11, preparation of training sets for different types of aircraft: 50 sets of Gaofen-2 satellite panchromatic and multispectral images from different regions are selected, and the panchromatic and multispectral images are fused by geometric correction and image fusion using the NNDiffuse Pan Sharpening method, and the civil aircraft sample data set is manually annotated. 80% of the data set is used as the training set and 20% as the test set.
[0040] Step S12, normalize the training graph, then magnify it by 255 times, and convert the floating point type into an integer type, which can speed up network calculation and reduce the calculation overhead of the system. The normalization calculation formula is as follows:
[0041]
[0042] Where x represents the image, x′ represents the normalized image, min(x) represents the minimum value of the training image, and max(x) represents the maximum value of the training image.
[0043] Step S13, improve the robustness of the network model: perform enhancement operations on the images segmented into 416×416 training images and their corresponding label images in the same way, including vertical and horizontal mirror flipping, 45°, 90°, 135°, 180°, 225°, 270°, 315°, 360° rotation, Gaussian noise processing, and expand the number of training sets. The expanded data can reach 60,000 groups.
[0044] Step S3, obtaining a response spectrum of the center point of the aircraft target according to the plurality of sample images and the aircraft target coordinate information. Preferably, the center point of the aircraft target may be the center point of the aircraft target anchor frame.
[0045] Step S4, obtaining the maximum and minimum values of the response spectrum according to the response spectra of all sample images in the aircraft target data set, and calculating the channel increment prior knowledge of the maximum and minimum values of the response spectrum and the response spectrum of the aircraft target center point in each sample image according to the maximum and minimum values of the response spectrum.
[0046] Specifically, by utilizing massive data of different lighting and climate conditions, we can obtain the maximum and minimum response values of each channel of the image, and then obtain the prior knowledge of the incremental range of each channel. Therefore, migration training can rely on a small number of samples and prior knowledge of the incremental range to achieve training migration between physical hosts, which can save massive data migration storage.
[0047] Step S5, based on the sample expansion technology of the aircraft center point prior knowledge under different lighting and climate conditions, the channel increment prior knowledge and multiple sample images are combined to perform multi-scene sample augmentation of the aircraft target to obtain multiple aircraft target samples.
[0048] In one embodiment, it is necessary to randomly augment the aircraft target samples within the interval of the incremental prior knowledge. The technology for augmenting the aircraft target samples includes the following steps:
[0049] Step S51, obtain the value of the Gaofen-2 satellite image at the center of the marked aircraft target anchor frame, denoted as M j i , where i is the image channel index, j is the image index, and the center point M of all aircraft targets j i Save to a text file.
[0050] Step S52: Count all the aircraft target center points in the text The maximum value of each channel Max i and the minimum value Min i , as prior knowledge, input a training sample image I i , the average value of each channel of all anchor box center points is recorded as V i .
[0051] Step S53, calculate the maximum Max i 、Minimum value i The average value V of the input image plane target i Based on the incremental prior knowledge of the channel, random multi-spectral samples with different climate and illumination scenes are augmented. The augmentation formulas for different scenes are as follows:
[0052] A i =V i -Min i
[0053] B i =Max i -Vi
[0054] I=I i +Rand(A i , B i )
[0055] Among them, A i is the difference between the minimum spectral response and the average of the training samples, B i is the difference between the maximum limit of spectral response and the average of training samples, and I is the training sample in A i With B i Random scene augmented images between, where I can be used as training sample augmented data.
[0056] like Figure 2 As shown in the example, multiple aircraft targets are obtained and the values at the center of the aircraft target anchor boxes are marked, which are recorded as Count all aircraft target center points The maximum value of each channel Max i and the minimum value Min i , based on multiple input sample images, the average value of the center point of the aircraft target anchor frame is obtained and recorded as V i ; Then according to the aircraft target center point The maximum value of each channel Max i 、Minimum value i The average value V of the aircraft target center point i Get an A i With B i , combined with the input sample image, different channel increments are randomly generated to generate multi-scene target samples.
[0057] In step S6, multiple aircraft target samples and aircraft target coordinate information are input into the target detection model for training, wherein the target detection model uses a deep residual network with multi-level skip connections to fuse the aircraft target features extracted at multiple levels to obtain a feature map with high semantic relationship.
[0058] It should be noted that the training process uses the gradient descent method of the neural network to solve the optimal solution for the target classification and target anchor box position regression tasks.
[0059] like Figure 3 As shown in the figure, in one example, the target detection model is improved based on the Faster RCNN (Faster-Regionbased Convolutional Neural Network) model, and a deep residual network with multi-level jump connections is used to fuse the multi-level extracted aircraft target features to obtain a feature map with high semantic relationship. The steps include:
[0060] Step S61, the backbone network of the network model is divided into 5 large convolution blocks, namely Layer_1, Layer_2, Layer_3, Layer_4, and Layer_5. The Layer_1 convolution block consists of a 7×7 convolution kernel, batch normalization and ReLU activation function, and a pooling layer. The remaining convolution blocks are composed of a 1×1 convolution kernel, a 3×3 convolution kernel, and a 1×1 convolution kernel, and are stacked with skip connection residuals. Each convolution block is connected by a skip connection residual.
[0061] This solution is mainly aimed at training across physical hosts, and the ability to detect aircraft can be improved with a small number of samples by connecting convolution blocks with residual skip connections.
[0062] In step S62, the RPN (Region Proposal Network) network is used as the candidate region network, features are extracted based on the backbone network, the target candidate box is calculated, and the aircraft features are obtained based on the candidate box position using bilinear interpolation.
[0063] Step S63, according to the channel attention mechanism, the aircraft feature F is combined with average pooling and maximum pooling, and the channel attention weight of each feature map is calculated according to the feature channel dimension, and the fused weight is applied to the extracted feature map to obtain F c ; The spatial attention mechanism converts the feature map F c From the spatial dimension, average pooling and maximum pooling are calculated for each point in the corresponding horizontal channel, and the results of the calculations are concatenated. A 1×1 convolution is used to obtain the attention weight of the feature map in space, and the calculated spatial attention weight is applied to the feature map F. c , and get the feature map F s .
[0064] Among them, the following formula is used to generate feature attention weights and enhance target features:
[0065] F c =σ(W1(W0(f a (F)))+W1(W0(f m (F))))
[0066] Where σ is the sigmoid function; f a is the average pooling, f m is the maximum pooling, F∈R C×H×W , W0∈R C / r×C ,W1∈R C×C / r , R z is the real number domain, z is the real number domain R space range set, W0 and W1 are the multi-layer perceptron weight vectors, C represents the size of the feature channel dimension, H represents the size of the feature high dimension, W represents the size of the feature wide dimension, r represents the reduction rate, F c It is the feature map after the channel attention mechanism.
[0067] F s =σ(f 7×7 ([f a (F c );f m (F c )]))
[0068] Among them, F s ∈R 1×H×W , F s is the spatial attention feature matrix, σ is the sigmoid function; f 7×7 It is a convolution operation with a convolution kernel of 7×7 dimensions.
[0069] In step S64, the augmented aircraft target samples and aircraft position coordinates are input into the model for training. The training process uses the gradient descent method of the neural network to solve the optimal solution for the target classification and detection anchor frame position regression tasks.
[0070] In step S7, the feature map is intercepted by bilinear interpolation in combination with the candidate frame, the position of the aircraft target anchor frame is inferred through a neural network, and the overlapping aircraft target anchor frame position coordinates are eliminated in combination with a non-maximum suppression algorithm, and finally the coordinates of the aircraft target are drawn on the remote sensing image.
[0071] In one example, the steps of combining the candidate frame with bilinear interpolation to extract the feature map, inferring the position of the aircraft target anchor frame through the neural network, and removing the overlapping aircraft target anchor frame position coordinates by combining the non-maximum suppression algorithm, and finally drawing the coordinates of the aircraft target on the remote sensing image include:
[0072] Step S71, using a sliding window method, setting an overlap value, and interlacingly cropping remote sensing images;
[0073] Step S72, using the cropped remote sensing image as a model input to detect the latitude and longitude results;
[0074] Step S73, filtering overlapping targets according to the non-maximum suppression algorithm, and finally drawing the longitude and latitude coordinates of the aircraft target in the remote sensing image.
[0075] like Figure 4 , showing the effect of aircraft target detection in a specific embodiment. The present invention proposes a method for augmenting climate and illumination remote sensing image data samples for the multi-spectral bands of the Gaofen-2 satellite, and integrates a deep residual network and attention mechanism with multi-level jump connections based on the Faster RCNN model to achieve aircraft target detection, thereby improving the generalization ability of the model for aircraft target detection in different regions, and further improving the aircraft target detection capability for airports in different regions.
[0076] The various technical features described above can be combined arbitrarily. Although all possible combinations of these technical features are not described, any combination of these technical features should be considered to be covered by this specification as long as there is no contradiction in such combination.
[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A multi-scenario aircraft target detection method, characterized in that: include: Crop aircraft target images based on high-resolution satellite remote sensing images to create aircraft target datasets; Performing geometric augmentation on the aircraft target image in the data set to obtain a plurality of sample images; wherein the data set includes aircraft target coordinate information; Acquire a response spectrum of the center point of the aircraft target according to the plurality of sample images and the aircraft target coordinate information; Obtaining the maximum and minimum values of the response spectrum according to the response spectra of all sample images in the aircraft target data set, and calculating the channel increment prior knowledge of the maximum and minimum values of the response spectrum and the response spectrum of the aircraft target center point in each sample image according to the maximum and minimum values of the response spectrum; According to the sample expansion technology of the aircraft center point prior knowledge under different lighting and climate conditions, the channel increment prior knowledge and the multiple sample images are combined to perform multi-scene sample augmentation of the aircraft target to obtain multiple aircraft target samples; Inputting the multiple aircraft target samples and the aircraft target coordinate information into a target detection model for training, wherein the target detection model uses a deep residual network with multi-level skip connections to fuse the aircraft target features extracted at multiple levels to obtain a feature map with high semantic relationship; Combined with the aircraft target anchor frame, bilinear interpolation is used to intercept the feature map, and the position of the aircraft target anchor frame is inferred through a neural network. The overlapping aircraft target anchor frame position coordinates are eliminated in combination with the non-maximum suppression algorithm, and finally the coordinates of the aircraft target are drawn on the remote sensing image.
2. The method according to claim 1, characterized in that The method of cutting out the aircraft target image according to the high-resolution satellite image to prepare the aircraft target data set includes: Multiple high-resolution remote sensing images of multiple scenes with different lighting and climate were selected, and each remote sensing image was divided into 416*416 grid blocks. The grid block position corresponding to the aircraft target was calculated, the grid block corresponding to the aircraft target was cropped, and the relative coordinates of the aircraft target on the grid block were recorded to generate a dataset of aircraft target images.
3. The method according to claim 2, characterized in that The step of cutting the aircraft target image according to the high-resolution satellite image to produce the aircraft target data set also includes: The aircraft target images in the data set are normalized; the normalization calculation formula is as follows: Where x represents the image, x′ represents the normalized image, min(x) represents the minimum value of the training image, and max(x) represents the maximum value of the training image.
4. The method according to claim 3, characterized in that After normalizing the data in the data set, the method further includes: Enlarge the aircraft target image by 255 times and convert the floating-point data type into integer storage.
5. The method according to claim 1, characterized in that The geometric augmentation of the aircraft target image in the data set to obtain a plurality of sample images includes: The aircraft target image is geometrically augmented by means of random angle rotation, horizontal and vertical mirroring, and random scaling.
6. The method according to claim 1, characterized in that The step of obtaining the aircraft target center point response spectrum according to the plurality of sample images and the aircraft target coordinate information comprises: Obtain multiple high-resolution remote sensing images of different lighting, different climate and multiple scenes, and calculate the value of the image corresponding to the center of the aircraft target anchor frame according to the aircraft target position, and record it as Where i is the image channel index and j is the image index.
7. The method according to claim 1, characterized in that The method of obtaining the maximum and minimum values of the response spectrum according to the response spectrum of all sample images in the aircraft target data set, and calculating the channel increment prior knowledge of the maximum and minimum values of the response spectrum and the response spectrum of the aircraft target center point in each sample image according to the maximum and minimum values of the response spectrum includes: Statistics all The maximum value of each channel i and the minimum value Min i , as prior knowledge, a training sample image I i The average value of each channel of all anchor box center points is recorded as V i ; The following augmentation formulas for different scenarios are used to calculate the channel increment prior knowledge: A i =V i -Min i B i =Max i -V i I=I i +Rand(A i ,B i ) Among them, A i is the difference between the minimum spectral response and the average of the training samples, B i is the difference between the maximum limit of spectral response and the average of training samples, and I is the training sample in A i With B i Random scene augmentation images between.
8. The method according to claim 1, characterized in that The inputting the plurality of aircraft target samples and the aircraft target coordinate information into the target detection model for training comprises: The training process uses the gradient descent method of the neural network to find the optimal solution for the target classification and target anchor box position regression tasks.
9. The method according to claim 8, characterized in that The target detection model is based on a deep residual network with multi-level skip connections to fuse the multi-level extracted aircraft target features to obtain a feature map with high semantic relationship, including: The backbone network of the network model is divided into five large convolution blocks, namely Layer_1, Layer_2, Layer_3, Layer_4, and Layer_5. The Layer_1 convolution block consists of a 7×7 convolution kernel, batch normalization and ReLU activation function, and a pooling layer. The remaining convolution blocks consist of a 1×1 convolution kernel, a 3×3 convolution kernel, and a 1×1 convolution kernel, and are stacked with skip connection residuals. Each convolution block is connected by a skip connection residual. The RPN network is used as the candidate area network. The features are extracted according to the backbone network, and the target candidate box is calculated. According to the position of the candidate box, the aircraft features are obtained by bilinear interpolation. The channel attention mechanism combines the aircraft feature F with the average pooling and the maximum pooling, calculates the attention weight of each feature map channel according to the feature channel dimension, and applies the fused weight to the extracted feature map to obtain Fc; the spatial attention mechanism calculates the average pooling and the maximum pooling for each point corresponding to the horizontal channel of the feature map Fc from the spatial dimension, concatenates the calculated results, uses a 1×1 convolution to obtain the spatial feature map attention weight, and applies the calculated spatial attention weight to the feature map Fc to obtain the feature map Fs.
10. The method according to claim 1, characterized in that The feature map is cropped, and the overlapping coordinate information is eliminated by combining the non-maximum suppression algorithm, and finally the coordinates of the aircraft target are drawn on the remote sensing image, including: The sliding window method is used to set the overlap value and interleave the remote sensing images; The cropped remote sensing image is used as the model input to detect the latitude and longitude results; The overlapping targets are filtered according to the non-maximum suppression algorithm, and finally the latitude and longitude coordinates of the aircraft target are drawn in the remote sensing image.
Citation Information
Patent Citations
Remote sensing image target detection method and system based on multi-module fusion
CN112800964A
Remote sensing image aircraft identification method based on deep learning and part prior
CN112966555A
Optical remote sensing image aircraft detection method and device based on cascade regression correction
CN113537017A
General target detection method for adaptive attention guidance mechanism
WO2021139069A1
