A Remote Sensing Image Aircraft Detection Method and System Based on a Local Similarity Offset Generator

Abstract

The present invention discloses a method and system for detecting airplanes in remote sensing images based on a local similarity offset generator, which relates to the technical field of remote sensing image processing. The method includes: obtaining a to-be-detected remote sensing image and inputting it into the local similarity offset generator to obtain a plurality of different feature map groups composed of hierarchical feature maps and corresponding resampled feature maps; inputting all the feature map groups into a feature pyramid network to obtain a final fused feature map; performing resampling processing on the final fused feature map to obtain a final output feature map; inputting the final output feature map into a detection network to obtain a final detection result. The detection accuracy and robustness of airplanes in remote sensing images are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and more specifically, to a method and system for detecting airplanes in remote sensing images based on a local similarity offset generator. Background Art

[0002] Today, with the rapid development of remote sensing technology, the application of remote sensing images has penetrated into many fields, including agricultural monitoring, urban planning, environmental protection, and military reconnaissance. Among them, the object detection in remote sensing images, especially the detection of airplanes, has attracted much attention due to its great significance for security and monitoring. With the continuous increase in the number of unmanned aerial vehicles and civil aviation airplanes, how to accurately and efficiently detect and identify airplanes from remote sensing images has become an urgent technical problem to be solved.

[0003] Traditional airplane detection methods mainly rely on feature-based detection algorithms, which usually analyze by extracting features such as edges, corners, and textures in images. However, these methods have strong limitations. For example, in complex backgrounds, feature extraction is inaccurate, resulting in a significant reduction in the detection rate of airplanes. In addition, with the improvement of the resolution of remote sensing images, airplanes appear smaller and smaller in the images, making it difficult for traditional methods to accurately detect airplanes.

[0004] In recent years, the rapid development of deep learning technology has brought new opportunities for object detection in remote sensing images. Especially the emergence of convolutional neural networks (CNNs) has made feature extraction and representation more automated and efficient. Network architectures represented by EfficientNet have gradually become important tools in remote sensing image processing due to their superior performance in parameter efficiency and accuracy. However, when dealing with airplanes with complex backgrounds, different scales, and pose changes, the robustness and accuracy of these deep learning models still need to be improved.

[0005] Therefore, how to improve the detection accuracy and robustness of airplanes in remote sensing images is an urgent problem for those skilled in the art to solve. Summary of the Invention

[0006] In view of this, the present invention provides a method and system for detecting airplanes in remote sensing images based on a local similarity offset generator, which improves the detection accuracy and robustness of airplanes in remote sensing images by optimizing the feature extraction and fusion mechanism.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] A method for detecting airplanes in remote sensing images based on a local similarity offset generator, comprising:

[0009] Obtain the remote sensing image to be measured and input it into the local similarity offset generator to obtain multiple different groups of feature maps composed of hierarchical feature maps and corresponding resampled feature maps;

[0010] Input all the groups of feature maps into the feature pyramid network to obtain the final fused feature map;

[0011] Perform resampling processing on the final fused feature map to obtain the final output feature map;

[0012] Input the final output feature map into the detection network to obtain the final detection result.

[0013] Preferably, the method for obtaining the group of feature maps is as follows:

[0014] Input the remote sensing image to be measured into the feature extraction module to obtain multiple hierarchical feature maps with different levels;

[0015] Input all the hierarchical feature maps into the local similarity calculation module to obtain the corresponding hierarchical similarity;

[0016] Input the hierarchical feature maps and the corresponding hierarchical similarity into the offset generator to obtain the corresponding predicted offset;

[0017] Input the hierarchical feature maps and the corresponding predicted offset into the resampling module to obtain the resampled feature map;

[0018] Compose the group of feature maps based on the hierarchical feature maps and the corresponding resampled feature maps.

[0019] Preferably, the method for obtaining the hierarchical similarity is as follows:

[0020] Calculate the cosine similarity between each pixel in the hierarchical feature map and the pixels in its neighborhood to obtain the corresponding pixel similarity;

[0021] Obtain the corresponding hierarchical similarity based on all the pixel similarities.

[0022] Preferably, the calculation formula of the pixel similarity is as follows:

[0023]

[0024] Among them, represents the pixel similarity at the position (i, j) of the l-th hierarchical feature map, represents the pixel at the position (i, j) in the l-th hierarchical feature map, m and n respectively represent different offsets around the pixel (i, j), and k represents the range size of the neighborhood, represents the extended feature point, Represents the eigenvalue of the relevant position.

[0025] Preferably, the offset generator includes: a splicing unit, a first branch, a second branch, and a fusion unit;

[0026] The first branch includes a first convolutional layer;

[0027] The second branch includes a second convolutional layer and an activation function layer;

[0028] The hierarchical feature map and the corresponding hierarchical similarity are input into the splicing unit to obtain the corresponding spliced feature;

[0029] The spliced feature is input into the first convolutional layer to obtain the offset direction;

[0030] The spliced feature is sequentially input into the second convolutional layer and the activation function layer to obtain the offset scale;

[0031] Based on the offset direction and the offset scale input into the fusion unit, the predicted offset is obtained.

[0032] Preferably, obtaining the final fusion feature map specifically includes:

[0033] Each group of feature maps is respectively input into different layers of the feature pyramid network. Based on the top-down path, the hierarchical feature map in this layer is fused with the resampled feature map in the next layer to obtain the layer fusion feature map of this layer, and so on, to obtain the layer fusion feature maps of all layers;

[0034] Based on the layer fusion feature map and the resampled feature map in the corresponding layer, the layer scale feature map is obtained;

[0035] Based on the integration of all the layer scale feature maps, the final fusion feature map is obtained.

[0036] Preferably, obtaining the final output feature map specifically includes:

[0037] Calculate the final offset based on each pixel in the final fusion feature map;

[0038] Based on the final offset and the final fusion feature map, resampling is performed to obtain the final output feature map.

[0039] Preferably, the detection network includes: a region proposal network, a classifier, and a bounding box regression module;

[0040] The final output feature map is input into the region proposal network to obtain multiple candidate boxes and the corresponding confidence scores. The candidate boxes and the corresponding confidence scores together form the candidate region features;

[0041] The candidate region features are input into the classifier to obtain the class probabilities of the candidate regions;

[0042] The candidate region features are input into the bounding box regression module to obtain the localization results of the bounding boxes;

[0043] Based on the class probabilities and the localization results, the final detection results are obtained.

[0044] Preferably, the local similarity offset generator, the feature pyramid network, and the detection network together constitute an aircraft detection model;

[0045] An annotated remote sensing image dataset is obtained as the training set;

[0046] Based on the training set and the comprehensive loss function, the aircraft detection model is trained to obtain a trained aircraft detection model;

[0047] The comprehensive loss function L is:

[0048] L = L cls + λL boundary

[0049] where L cls represents the classification loss, L boundary represents the bounding loss, and λ represents the weight coefficient for balancing the two loss terms.

[0050] A remote sensing image aircraft detection system based on a local similarity offset generator, comprising: a feature map group generation module, a feature fusion module, a preprocessing module, and a result output module;

[0051] The feature map group generation module is configured to obtain a to-be-detected remote sensing image and input it into the local similarity offset generator to obtain a plurality of different feature map groups each composed of hierarchical feature maps and corresponding resampled feature maps;

[0052] The feature fusion module is configured to input all the feature map groups into the feature pyramid network to obtain a final fused feature map;

[0053] The preprocessing module is configured to perform resampling processing on the final fused feature map to obtain a final output feature map;

[0054] The result output module is configured to input the final output feature map into the detection network to obtain a final detection result.

[0055] Through the above technical solutions, compared with the prior art, the present invention discloses a remote sensing image aircraft detection method and system based on a local similarity offset generator, having the following beneficial effects:

[0056] 1. Improve the accuracy and robustness of aircraft detection: By introducing a local similarity offset generator, the present invention can effectively utilize the spatial information in remote sensing images, thereby better maintaining the consistency and accuracy of features when detecting aircraft; compared with traditional detection methods, the present invention can more accurately identify the boundaries and positions of aircraft, especially under complex backgrounds and various interference conditions, showing higher robustness; the method of the present invention can stably perform in different types of remote sensing images, reducing the false detection rate and missed detection rate.

[0057] 2. Optimize the feature fusion mechanism and multi-scale feature expression: By adopting the Feature Pyramid Network (FPN) structure, the present invention can effectively fuse features at different levels, ensuring the organic combination of high-level semantic information and low-level detail information; through this fusion mechanism, the detection ability of the model for multi-scale aircraft targets is enhanced, and the recognition effect for aircraft of various sizes is improved; in addition, the design of the local similarity offset generator enables better retention of high-similarity regions during the feature map resampling process, thereby enhancing the overall feature expression ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on the provided drawings.

[0059] Figure 1 It is a flowchart of a method for detecting aircraft in remote sensing images based on a local similarity offset generator provided by the present invention.

[0060] Figure 2 It is a flowchart of a method for obtaining a feature map group provided by the present invention.

[0061] Figure 3 It is a schematic structural diagram of a system for detecting aircraft in remote sensing images based on a local similarity offset generator provided by the present invention.

[0062] Figure 4 It is a block diagram of the structure of a computer device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0063] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0064] Example 1

[0065] As Figure 1 shown, an aircraft detection method for remote sensing images based on a local similarity offset generator disclosed in an embodiment of the present invention includes:

[0066] Obtain the remote sensing image to be measured and input it into the local similarity offset generator to obtain multiple different groups of feature maps composed of hierarchical feature maps and corresponding resampled feature maps;

[0067] Input all the groups of feature maps into the feature pyramid network to obtain the final fused feature map;

[0068] Perform resampling processing on the final fused feature map to obtain the final output feature map;

[0069] Input the final output feature map into the detection network to obtain the final detection result.

[0070] Example 2

[0071] An aircraft detection method for remote sensing images based on a local similarity offset generator disclosed in an embodiment of the present invention includes:

[0072] Obtain the remote sensing image to be measured and input it into the local similarity offset generator to obtain multiple different groups of feature maps composed of hierarchical feature maps and corresponding resampled feature maps:

[0073] Preferably, as Figure 2 shown, the method for obtaining the group of feature maps is:

[0074] Input the remote sensing image to be measured into the feature extraction module to obtain multiple hierarchical feature maps with different levels;

[0075] Input all the hierarchical feature maps into the local similarity calculation module to obtain the corresponding hierarchical similarity;

[0076] Input the hierarchical feature maps and the corresponding hierarchical similarity into the offset generator to obtain the corresponding predicted offset;

[0077] Input the hierarchical feature maps and the corresponding predicted offset into the resampling module to obtain the resampled feature map;

[0078] Form a group of feature maps based on the hierarchical feature maps and the corresponding resampled feature maps.

[0079] Preferably, the feature extraction module in this embodiment adopts a pre-trained EfficientNet model, and inputs the remote sensing image to be measured through I into the pre-trained EfficientNet model to obtain a series of hierarchical feature maps {X 1 , X2 ,..., X L} where L represents the number of layers of the hierarchical feature map.

[0080] Preferably, the method for obtaining the hierarchical similarity is as follows:

[0081] Based on the hierarchical feature map X l (l = 1, 2, 3,..., L), calculate the cosine similarity between each pixel (i, j) and the pixels in its neighborhood in the hierarchical feature map X to obtain the corresponding pixel similarity;

[0082] Based on all the pixel similarities, obtain the corresponding hierarchical similarity.

[0083] Preferably, the calculation formula for the pixel similarity is as follows:

[0084]

[0085] where, represents the pixel similarity at the position (i, j) in the l-th hierarchical feature map, represents the pixel at the position (i, j) in the l-th hierarchical feature map, m and n respectively represent different offsets around the pixel (i, j) for traversing its neighboring pixels, k represents the range size of the neighborhood, which is a fixed integer to control the window size of the neighborhood, represents the extended feature point, represents the feature value at the relevant position, and their relationship reveals that by increasing the translation dimension or displacement dimension, feature changes can be analyzed from a broader perspective, especially for capturing more complex image characteristics in a multi-dimensional space.

[0086] Preferably, the offset generator includes: a splicing unit, a first branch, a second branch, and a fusion unit;

[0087] The first branch includes a first convolutional layer;

[0088] The second branch includes a second convolutional layer and an activation function layer;

[0089] The hierarchical feature map and the corresponding hierarchical similarity are input into the splicing unit to obtain the corresponding spliced feature;

[0090] The spliced feature is input into the first convolutional layer to obtain the offset direction;

[0091] The spliced feature is successively input into the second convolutional layer and the activation function layer to obtain the offset scale;

[0092] Based on the offset direction and the offset scale, input them into the fusion unit to obtain the predicted offset.

[0093] Preferably, the first convolutional layer uses a 3×3 convolutional layer, Xl The corresponding offset direction D l is:

[0094] D l = Conv 3×3 (Concat(X l , S l ));

[0095] where Conv 3×3 represents a 3×3 convolution operation, Concat represents feature concatenation, X l represents the hierarchical feature map of the l-th layer, S l represents the hierarchical similarity of the l-th layer, D l ∈R 2G×H×W , where G represents the number of offset groups, and H and W represent height and width respectively.

[0096] Preferably, the second convolutional layer uses a 3×3 convolutional layer, and the activation function layer uses a Sigmoid activation function. The corresponding offset scale A l is: l is:

[0097] A l = Sigmoid(Conv 3×3 (Concat(X l , S l )));

[0098] where Sigmoid represents the activation function, and A l ∈R 2G×H×W controls the size of the offset.

[0099] Preferably, X l corresponding predicted offset O l = D l ·A l .

[0100] Preferably, the present invention divides the features into different groups, assigns unique spatial offsets for finer-grained resampling, allows resampling of features with high intra-class similarity to replace features with low intra-class similarity. In this way, the offset generator can handle a large area of inconsistent features and refine the boundaries.

[0101] Preferably, resampling is performed on the corresponding hierarchical feature map based on the predicted offset to generate a feature map with improved local consistency, that is, a resampled feature map is obtained.

[0102] Based on all the feature map groups being input into the feature pyramid network, the final fused feature map is obtained:

[0103] Preferably, obtaining the final fused feature map specifically includes:

[0104] Each group of feature maps is respectively input into different layers of the feature pyramid network. Based on the top-down path, the hierarchical feature map of this layer is fused with the resampled feature map in the next layer to obtain the layer-fused feature map of this layer. And so on, the layer-fused feature maps of all layers are obtained;

[0105] Based on the layer-fused feature map and the resampled feature map in the corresponding layer, the layer-scale feature map is obtained;

[0106] Based on the integration of all the layer-scale feature maps, the final fused feature map is obtained.

[0107] Preferably, the hierarchical feature map X a (a = 1, 2, 3,..., L) in the highest layer is upsampled and then fused with the resampled feature map in the next layer to obtain the layer-fused feature map U a :

[0108]

[0109] wherein, Upsample() represents the upsampling operation;

[0110] And so on until the layer-fused feature map of the bottom layer is output, and the layer-fused feature maps of all layers are obtained.

[0111] Preferably, horizontal connection is performed. The layer-fused feature map obtained from the top-down path is fused with the resampled feature map in the same layer to obtain the fused layer-scale feature map. For example, the layer-fused feature map U a in the highest layer is fused with the resampled feature map in the highest layer to obtain the layer-scale feature map F a :

[0112]

[0113] And so on until the layer-scale feature map of the bottom layer is output, and the layer-scale feature maps of all layers are obtained.

[0114] Preferably, based on the integration of all the layer-scale feature maps, the final fused feature map F is obtained:

[0115] F = Concat(F 1 , F 2 ,..., F L )

[0116] wherein, Concat() represents the concatenation operation along the feature dimension.

[0117] Perform resampling on the basis of the final fused feature map to obtain the final output feature map:

[0118] Preferably, obtaining the final output feature map specifically includes:

[0119] Calculate the final offset for each pixel in the final fused feature map;

[0120] Perform resampling based on the final offset and the final fused feature map to obtain the final output feature map.

[0121] Preferably, input the final fused feature map into the above-mentioned offset generator to correspondingly obtain the final offset direction D and the final offset scale A, and obtain the final offset O based on the final offset direction D and the final offset scale A: O = D·A; perform resampling on the final fused feature map F based on the final offset O to obtain the final output feature map X final : X final = Resample(F, O), where Resample represents the resampling operation, and spatially adjust the final fused feature map F according to the final offset O.

[0122] Input the final output feature map into the detection network to obtain the final detection result.

[0123] Preferably, the detection network includes: a region proposal network, a classifier, and a bounding box regression module;

[0124] Input the final output feature map into the region proposal network to obtain multiple candidate boxes and the corresponding confidence scores, and the candidate boxes and the corresponding confidence scores together form the candidate region features;

[0125] Input the candidate region features into the classifier to obtain the class probabilities of the candidate regions;

[0126] Input the candidate region features into the bounding box regression module to obtain the localization results of the bounding boxes;

[0127] Obtain the final detection result based on the class probabilities and the localization results.

[0128] Preferably, use the high-level semantic information in the final output feature map X final to generate candidate target regions, use the Region Proposal Network (RPN) to identify regions that may contain an aircraft, and the region proposal network generates multiple candidate boxes for each position in the final output feature map X final and assign a confidence score to each candidate box to indicate whether the box is likely to contain the target, and the candidate boxes and the corresponding confidence scores together form the candidate region features.

[0129] Preferably, each candidate region feature is further classified to determine whether it contains an aircraft target: based on the candidate region feature being input into a classifier, it is determined whether the region is an aircraft.

[0130] Preferably, the classifier includes an ROI pooling layer, a Flatten layer, a first fully connected layer, a second fully connected layer, and a ReLU activation function connected in sequence; the candidate region features are sequentially input into the ROI pooling layer, the Flatten layer, the first fully connected layer, the second fully connected layer, and the ReLU activation function for processing to obtain the class probabilities of the candidate regions.

[0131] Preferably, for each candidate region, bounding box regression is performed to further accurately locate the boundary of the aircraft. The candidate region features are input into a bounding box regression module, and the bounding box regression module uses the position information in the feature map to adjust each candidate box so that it more accurately encloses the aircraft target, outputs the four coordinate offset values of the bounding box, and applies them to the initial candidate box to obtain the adjusted accurate position, that is, the localization result of the bounding box.

[0132] Preferably, post-processing is performed on the output results based on the class probabilities and the localization results. The non-maximum suppression (NMS) method is used to remove duplicate detection boxes, and only the boxes with higher confidence are retained. Based on the non-maximum suppression method, the confidence scores of multiple overlapping detection boxes are calculated, the detection box with the highest score is retained, and other boxes with a large overlap with it are suppressed, so as to obtain a more accurate detection result, that is, the final detection result.

[0133] Preferably, the local similarity offset generator, the feature pyramid network, and the detection network together constitute an aircraft detection model;

[0134] An annotated remote sensing image dataset is obtained as the training set;

[0135] Based on the training set and the comprehensive loss function, the aircraft detection model is trained to obtain a trained aircraft detection model;

[0136] The comprehensive loss function L is:

[0137] L = L cls + λL boundary

[0138] where L cls represents the classification loss, L boundary represents the bounding loss, and λ represents the weight coefficient for balancing the two loss terms.

[0139] Preferably, the goal of training is to optimize the model parameters of the aircraft detection model by constructing a comprehensive loss function that includes a classification loss and a boundary loss, ensuring that the aircraft detection model can accurately identify the aircraft category and precisely locate the aircraft boundary.

[0140] Preferably, the classification loss L cls is used to measure the prediction accuracy of the model for the aircraft category, and the cross-entropy loss function is used to calculate the classification loss L cls :

[0141]

[0142] where N represents the number of samples, C represents the number of categories, and y ic represents the true label of whether the i-th sample belongs to category C, and p ic represents the probability that the model predicts the i-th sample belongs to category C.

[0143] Preferably, the boundary loss L boundary is used to measure the accuracy of the model for aircraft boundary localization, and the IoU (Intersection over Union) loss function is used to calculate the boundary loss L boundary :

[0144]

[0145] where P represents the aircraft boundary region predicted by the model, and T represents the annotated true aircraft boundary region.

[0146] Preferably, an annotated remote sensing image dataset is used as the training set to train the model. The training set contains the category labels of the aircraft and their precise boundaries. The model parameters are optimized by minimizing the comprehensive loss function L:

[0147]

[0148] where θ represents the model parameters, and θ* represents the optimal model parameters.

[0149] The gradient descent algorithm is used for backpropagation to calculate the gradient of the loss function L with respect to the model parameters θ and update the model parameters, and finally a trained aircraft detection model is obtained.

[0150] Example 3

[0151] As Figure 3 shown, a remote sensing image aircraft detection system based on a local similarity offset generator includes: a feature map group generation module, a feature fusion module, a preprocessing module, and a result output module;

[0152] A feature map group generation module, configured to obtain a remote sensing image to be measured and input it into a local similarity offset generator, so as to obtain a plurality of different feature map groups each composed of a hierarchical feature map and a corresponding resampled feature map;

[0153] A feature fusion module, configured to input all the feature map groups into a feature pyramid network to obtain a final fused feature map;

[0154] A preprocessing module, configured to perform resampling processing on the final fused feature map to obtain a final output feature map;

[0155] A result output module, configured to input the final output feature map into a detection network to obtain a final detection result.

[0156] Preferably, the processes implemented by the above system modules of the present invention correspond one by one to the corresponding method parts above, and will not be elaborated here one by one.

[0157] Embodiment 4

[0158] Based on the same inventive concept, the present invention further provides a computer device, including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus;

[0159] The memory is used to store a computer program;

[0160] When the processor is used to execute the program stored in the memory, it can implement a remote sensing image aircraft detection method based on a local similarity offset generator as described in Embodiment 1 or 2.

[0161] As Figure 4 shown, the electronic device may include: a processor 41, a communication interface 42, a memory 43, and a communication bus 44. Among them, the processor 41, the communication interface 42, and the memory 43 complete mutual communication through the communication bus 44. The processor 41 can call the logical instructions in the memory 43 to execute a remote sensing image aircraft detection method based on a local similarity offset generator as described in Embodiment 1 or 2.

[0162] In addition, when the logical instructions in the above-mentioned memory 43 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0163] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a remote sensing image aircraft detection method and system based on a local similarity offset generator, which has the following beneficial effects:

[0164] 1. Improved accuracy and robustness of aircraft detection: By introducing a local similarity offset generator, the present invention can effectively utilize the spatial information in remote sensing images, thereby better maintaining the consistency and accuracy of features when detecting aircraft; compared with traditional detection methods, the present invention can more accurately identify the boundaries and positions of aircraft, especially under complex backgrounds and various interference conditions, showing higher robustness; the method of the present invention can stably perform in different types of remote sensing images, reducing the false detection rate and missed detection rate.

[0165] 2. Optimized feature fusion mechanism and multi-scale feature expression: By adopting the feature pyramid network (FPN) structure, the present invention can effectively fuse features at different levels, ensuring the organic combination of high-level semantic information and low-level detail information; through this fusion mechanism, the detection ability of the model for multi-scale aircraft targets is enhanced, and the recognition effect of various sizes of aircraft is improved; in addition, the design of the local similarity offset generator enables better retention of high-similarity regions during the feature map resampling process, thereby enhancing the overall feature expression ability.

[0166] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0167] The foregoing description of the disclosed embodiments enables those skilled in the art to practice or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A remote sensing image aircraft detection method based on a local similarity offset generator, characterized in that: include: Obtain a remote sensing image to be tested and input it into a local similarity shift generator to obtain a plurality of different feature map groups consisting of hierarchical feature maps and corresponding resampled feature maps; The method for obtaining the feature map group is: The remote sensing image to be measured is input into a feature extraction module to obtain multiple hierarchical feature maps at different levels; All of the hierarchical feature maps are input into a local similarity calculation module to obtain corresponding hierarchical similarities; Based on the hierarchical feature map and the corresponding hierarchical similarity, the offset generator is input to obtain a corresponding predicted offset; Based on the hierarchical feature map and the corresponding predicted offset, the hierarchical feature map is input into a resampling module to obtain a resampled feature map; Forming the feature map group based on the hierarchical feature map and the corresponding resampled feature map; Based on all the feature map groups, input into the feature pyramid network to obtain the final fused feature map; The final fusion feature map is obtained, including: Each of the feature map groups is input into different layers of the feature pyramid network respectively, and the hierarchical feature map in the current layer is fused with the resampled feature map in the next layer based on a top-down path to obtain a layer fusion feature map of the current layer, and so on to obtain the layer fusion feature maps of all layers; Based on the fusion of the layer fusion feature map and the resampled feature map in the corresponding layer, a layer scale feature map is obtained; Integrate all the layer-scale feature maps to obtain the final fused feature map; Perform resampling processing based on the final fusion feature map to obtain a final output feature map; The final output feature map is input into the detection network to obtain the final detection result.

2. The method for detecting aircraft in remote sensing images based on a local similarity offset generator according to claim 1, characterized in that: The hierarchical similarity acquisition method is: Based on each pixel in the hierarchical feature map, the cosine similarity between the pixel and the pixels in its neighborhood is calculated to obtain the corresponding pixel similarity; The corresponding hierarchical similarity is obtained based on all the pixel similarities.

3. The method for detecting aircraft in remote sensing images based on a local similarity offset generator according to claim 1, characterized in that: The offset generator comprises: a splicing unit, a first branch, a second branch and a fusion unit; The first branch includes a first convolutional layer; The second branch includes a second convolutional layer and an activation function layer; The hierarchical feature map and the corresponding hierarchical similarity are input into the splicing unit to obtain corresponding splicing features; The splicing feature is input into the first convolutional layer to obtain the offset direction; The concatenated features are sequentially input into the second convolutional layer and the activation function layer to obtain an offset scale; The offset direction and the offset scale are input into the fusion unit to obtain the predicted offset.

4. The method for detecting aircraft in remote sensing images based on a local similarity offset generator according to claim 1, characterized in that: The final output feature map is obtained, including: Calculate a final offset based on each pixel in the final fused feature map; Resampling is performed based on the final offset and the final fused feature map to obtain the final output feature map.

5. The method for detecting aircraft in remote sensing images based on a local similarity offset generator according to claim 1, characterized in that: The detection network includes: a region proposal network, a classifier and a bounding box regression module; The final output feature map is input into the region proposal network to obtain a plurality of candidate boxes and corresponding confidence scores, wherein the candidate boxes and corresponding confidence scores together constitute candidate region features; The candidate region features are input into the classifier to obtain the category probability of the candidate region; The candidate region features are input into the bounding box regression module to obtain a positioning result of the bounding box; The final detection result is obtained based on the category probability and the positioning result.

6. The method for detecting aircraft in remote sensing images based on a local similarity offset generator according to claim 1, characterized in that: The local similarity offset generator, the feature pyramid network and the detection network together constitute an aircraft detection model; Obtain annotated remote sensing image dataset as a training set; Training the aircraft detection model based on the training set and the comprehensive loss function to obtain a trained aircraft detection model; The comprehensive loss function L is: L=L cls +λL boundary Among them, L cls represents the classification loss, L boundary represents the boundary loss, and λ represents the weight coefficient for balancing the two loss terms.

7. A remote sensing image aircraft detection system based on a local similarity offset generator, applied to a remote sensing image aircraft detection method based on a local similarity offset generator as claimed in any one of claims 1 to 6, characterized in that: include: Feature map group generation module, feature fusion module, preprocessing module and result output module; The feature map group generation module is used to obtain the remote sensing image to be tested and input it into the local similarity shift generator to obtain a plurality of different feature map groups consisting of hierarchical feature maps and corresponding resampled feature maps; The feature fusion module is used to input all the feature map groups into a feature pyramid network to obtain a final fused feature map; The preprocessing module is used to perform resampling processing based on the final fusion feature map to obtain a final output feature map; The result output module is used to input the final output feature map into the detection network to obtain the final detection result.

Images