Remote sensing image aircraft detection method and system based on local similarity migration generator

Abstract

The invention discloses a remote sensing image aircraft detection method and system based on a local similarity offset generator, and relates to the technical field of remote sensing image processing. The method comprises the following steps: acquiring a remote sensing image to be detected and inputting the remote sensing image to the local similarity offset generator; obtaining a plurality of different feature map groups consisting of hierarchical feature maps and corresponding resampling feature maps; inputting all the feature map groups into the feature pyramid network to obtain a final fusion feature map; performing resampling processing based on the final fusion feature map to obtain a final output feature map; and inputting the final output feature map into a detection network to obtain a final detection result. And the detection accuracy and robustness of the airplane in the remote sensing image are improved.

Description

Technical Field

[0001] The invention relates to the technical field of remote sensing image processing, and more particularly to a remote sensing image aircraft detection method and system based on a local similarity offset generator. Background Art

[0002] With the rapid development of remote sensing technology, the application of remote sensing images has penetrated into many fields, including agricultural monitoring, urban planning, environmental protection and military reconnaissance. Among them, target detection in remote sensing images, especially aircraft detection, has attracted much attention due to its great significance for security and monitoring. With the increasing number of drones and civil aircraft, how to accurately and efficiently detect and identify aircraft from remote sensing images has become a technical problem that needs to be solved urgently.

[0003] Traditional aircraft detection methods mainly rely on feature-based detection algorithms, which usually analyze features such as edges, corners, and textures in images. However, these methods have strong limitations. For example, in complex backgrounds, feature extraction is inaccurate, resulting in a significant reduction in the detection rate of aircraft. In addition, as the resolution of remote sensing images increases, aircraft appear smaller and smaller in images, making it difficult for traditional methods to accurately detect aircraft.

[0004] In recent years, the rapid development of deep learning technology has brought new opportunities for target detection in remote sensing images. In particular, the emergence of convolutional neural networks (CNNs) has made feature extraction and representation more automated and efficient. Network architectures represented by EfficientNet have gradually become important tools in remote sensing image processing due to their superior performance in parameter efficiency and accuracy. However, the robustness and accuracy of these deep learning models still need to be improved when dealing with aircraft with complex backgrounds, different scales, and attitude changes.

[0005] Therefore, how to improve the detection accuracy and robustness of aircraft in remote sensing images is an urgent problem that technicians in this field need to solve. Summary of the invention

[0006] In view of this, the present invention provides a remote sensing image aircraft detection method and system based on a local similarity offset generator, which improves the detection accuracy and robustness of aircraft in remote sensing images by optimizing feature extraction and fusion mechanism.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A remote sensing image aircraft detection method based on a local similarity offset generator, comprising:

[0009] Obtain a remote sensing image to be tested and input it into a local similarity shift generator to obtain a plurality of different feature map groups consisting of hierarchical feature maps and corresponding resampled feature maps;

[0010] Based on all the feature map groups, input into the feature pyramid network to obtain the final fused feature map;

[0011] Perform resampling processing based on the final fusion feature map to obtain a final output feature map;

[0012] The final output feature map is input into the detection network to obtain the final detection result.

[0013] Preferably, the method for acquiring the feature graph group is:

[0014] The remote sensing image to be measured is input into a feature extraction module to obtain multiple hierarchical feature maps at different levels;

[0015] All of the hierarchical feature maps are input into a local similarity calculation module to obtain corresponding hierarchical similarities;

[0016] Based on the hierarchical feature map and the corresponding hierarchical similarity, the offset generator is input to obtain a corresponding predicted offset;

[0017] Based on the hierarchical feature map and the corresponding predicted offset, the hierarchical feature map is input into a resampling module to obtain a resampled feature map;

[0018] The feature map group is formed based on the hierarchical feature map and the corresponding resampled feature map.

[0019] Preferably, the hierarchical similarity acquisition method is:

[0020] Based on each pixel in the hierarchical feature map, the cosine similarity between the pixel and the pixels in its neighborhood is calculated to obtain the corresponding pixel similarity;

[0021] The corresponding hierarchical similarity is obtained based on all the pixel similarities.

[0022] Preferably, the calculation formula of the pixel similarity is as follows:

[0023]

[0024] in, Represents the pixel similarity of the l-th level feature map at position (i, j), represents the pixel at position (i, j) in the l-th level feature map, m and n represent the different offsets around the pixel (i, j), and k represents the size of the neighborhood. represents the extended feature points, Represents the eigenvalue of the relevant position.

[0025] Preferably, the offset generator comprises: a splicing unit, a first branch, a second branch and a fusion unit;

[0026] The first branch includes a first convolutional layer;

[0027] The second branch includes a second convolutional layer and an activation function layer;

[0028] The hierarchical feature map and the corresponding hierarchical similarity are input into the splicing unit to obtain corresponding splicing features;

[0029] The splicing feature is input into the first convolutional layer to obtain the offset direction;

[0030] The concatenated features are sequentially input into the second convolutional layer and the activation function layer to obtain an offset scale;

[0031] The offset direction and the offset scale are input into the fusion unit to obtain the predicted offset.

[0032] Preferably, the final fusion feature map is obtained, specifically including:

[0033] Each of the feature map groups is input into different layers of the feature pyramid network respectively, and the hierarchical feature map in the current layer is fused with the resampled feature map in the next layer based on a top-down path to obtain a layer fusion feature map of the current layer, and so on to obtain the layer fusion feature maps of all layers;

[0034] Based on the fusion of the layer fusion feature map and the resampled feature map in the corresponding layer, a layer scale feature map is obtained;

[0035] Based on the integration of all the layer-scale feature maps, the final fused feature map is obtained.

[0036] Preferably, obtaining the final output feature map specifically includes:

[0037] Calculate a final offset based on each pixel in the final fused feature map;

[0038] Resampling is performed based on the final offset and the final fused feature map to obtain the final output feature map.

[0039] Preferably, the detection network comprises: a region proposal network, a classifier and a bounding box regression module;

[0040] The final output feature map is input into the region proposal network to obtain a plurality of candidate boxes and corresponding confidence scores, wherein the candidate boxes and corresponding confidence scores together constitute candidate region features;

[0041] The candidate region features are input into the classifier to obtain the category probability of the candidate region;

[0042] The candidate region features are input into the bounding box regression module to obtain a positioning result of the bounding box;

[0043] The final detection result is obtained based on the category probability and the positioning result.

[0044] Preferably, the local similarity offset generator, the feature pyramid network and the detection network together constitute an aircraft detection model;

[0045] Obtain annotated remote sensing image dataset as a training set;

[0046] Training the aircraft detection model based on the training set and the comprehensive loss function to obtain a trained aircraft detection model;

[0047] The comprehensive loss function L is:

[0048] L=L cls +λL boundary

[0049] Among them, L cls represents the classification loss, L boundary represents the boundary loss, and λ represents the weight coefficient for balancing the two loss terms.

[0050] A remote sensing image aircraft detection system based on a local similarity offset generator comprises: a feature map group generation module, a feature fusion module, a preprocessing module and a result output module;

[0051] The feature map group generation module is used to obtain the remote sensing image to be tested and input it into the local similarity shift generator to obtain a plurality of different feature map groups consisting of hierarchical feature maps and corresponding resampled feature maps;

[0052] The feature fusion module is used to input all the feature map groups into a feature pyramid network to obtain a final fused feature map;

[0053] The preprocessing module is used to perform resampling processing based on the final fused feature map to obtain a final output feature map;

[0054] The result output module is used to input the final output feature map into the detection network to obtain the final detection result.

[0055] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a remote sensing image aircraft detection method and system based on a local similarity offset generator, which has the following beneficial effects:

[0056] 1. Improved accuracy and robustness of aircraft detection: By introducing a local similarity offset generator, the present invention can effectively utilize spatial information in remote sensing images, thereby better maintaining feature consistency and accuracy when detecting aircraft; compared with traditional detection methods, the present invention can more accurately identify aircraft boundaries and positions, especially under complex backgrounds and multiple interference conditions, and exhibits higher robustness; the method of the present invention can stably perform performance in different types of remote sensing images, reducing false detection rates and missed detection rates.

[0057] 2. Optimize feature fusion mechanism and multi-scale feature expression: The present invention adopts a feature pyramid network (FPN) structure to effectively fuse features at different levels, ensuring the organic combination of high-level semantic information and low-level detail information; through this fusion mechanism, the model's detection capability for multi-scale aircraft targets is enhanced, and the recognition effect for aircraft of various sizes is improved; in addition, the design of the local similarity offset generator enables better retention of high-similarity areas during feature map resampling, thereby enhancing the overall feature expression capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0059] Figure 1 A flow chart of a remote sensing image aircraft detection method based on a local similarity offset generator provided by the present invention.

[0060] Figure 2 This is a flow chart of the feature graph group acquisition method provided by the present invention.

[0061] Figure 3 A schematic diagram of the structure of a remote sensing image aircraft detection system based on a local similarity offset generator provided by the present invention.

[0062] Figure 4 This is a structural block diagram of the computer device provided by the present invention. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] Example 1

[0065] like Figure 1 As shown, the embodiment of the present invention discloses a remote sensing image aircraft detection method based on a local similarity offset generator, comprising:

[0066] Obtain a remote sensing image to be tested and input it into a local similarity shift generator to obtain a plurality of different feature map groups consisting of hierarchical feature maps and corresponding resampled feature maps;

[0067] Based on all feature map groups, input them into the feature pyramid network to obtain the final fused feature map;

[0068] Resampling is performed based on the final fusion feature map to obtain the final output feature map;

[0069] Based on the final output feature map, it is input into the detection network to obtain the final detection result.

[0070] Example 2

[0071] The embodiment of the present invention discloses a remote sensing image aircraft detection method based on a local similarity offset generator, comprising:

[0072] Get the remote sensing image to be tested and input it into the local similarity offset generator to obtain multiple different feature map groups consisting of hierarchical feature maps and corresponding resampled feature maps:

[0073] Preferably, Figure 2 As shown, the method for obtaining the feature map group is:

[0074] The remote sensing image to be measured is input into the feature extraction module to obtain multiple hierarchical feature maps at different levels;

[0075] All hierarchical feature maps are input into the local similarity calculation module to obtain the corresponding hierarchical similarity;

[0076] Based on the hierarchical feature map and the corresponding hierarchical similarity, the offset generator is input to obtain the corresponding predicted offset;

[0077] Based on the hierarchical feature map and the corresponding prediction offset, the resampling module is input to obtain a resampled feature map;

[0078] A feature map group is formed based on the hierarchical feature maps and the corresponding resampled feature maps.

[0079] Preferably, the feature extraction module of this embodiment adopts a pre-trained EfficientNet model, and the remote sensing image to be measured is input into the pre-trained EfficientNet model through I to obtain a series of hierarchical feature maps {X 1 , X2 , ..., X L}, where L represents the number of layers of the hierarchical feature map.

[0080] Preferably, the hierarchical similarity acquisition method is:

[0081] Based on the hierarchical feature map X l For each pixel (i, j) in (l = 1, 2, 3, ..., L), calculate the cosine similarity between the pixel (i, j) and the pixels in its neighborhood to obtain the corresponding pixel similarity;

[0082] Based on all pixel similarities, the corresponding hierarchical similarity is obtained.

[0083] Preferably, the calculation formula of pixel similarity is as follows:

[0084]

[0085] in, Represents the pixel similarity of the l-th level feature map at position (i, j), Represents the pixel at position (i, j) in the l-th level feature map. m and n represent different offsets around the pixel (i, j), which are used to traverse its neighborhood pixels. k represents the size of the neighborhood, which is a fixed integer and controls the window size of the neighborhood. represents the extended feature points, The feature values ​​representing related positions, their relationship reveals that by adding translation dimension or displacement dimension, feature changes can be analyzed from a wider perspective, especially capturing more complex image characteristics in multi-dimensional space.

[0086] Preferably, the offset generator comprises: a splicing unit, a first branch, a second branch and a fusion unit;

[0087] The first branch includes a first convolutional layer;

[0088] The second branch includes a second convolutional layer and an activation function layer;

[0089] The hierarchical feature map and the corresponding hierarchical similarity are input into the splicing unit to obtain the corresponding splicing features;

[0090] The concatenated features are input to the first convolutional layer to obtain the offset direction;

[0091] The concatenated features are sequentially input into the second convolutional layer and the activation function layer to obtain the offset scale;

[0092] Based on the offset direction and offset scale, the offset is input to the fusion unit to obtain the predicted offset.

[0093] Preferably, the first convolutional layer uses a 3×3 convolutional layer, Xl The corresponding offset direction D l for:

[0094] D l =Conv 3×3 (Concat(X l ,S l ));

[0095] Among them, Conv 3×3 represents a 3×3 convolution operation, Concat represents feature concatenation, and X l represents the hierarchical feature map of the lth layer, S l represents the hierarchical similarity of the lth layer, D l ∈R 2G×H×W , where G represents the number of offset groups, and H and W represent the height and width respectively.

[0096] Preferably, the second convolution layer uses a 3×3 convolution layer, and the activation function layer uses a Sigmoid activation function. l The corresponding offset scale A l for:

[0097] A l =Sigmoid(Conv 3×3 (Concat(X l ,S l )));

[0098] Among them, Sigmoid represents the activation function, A l ∈R 2G×H×W Controls the size of the offset.

[0099] Preferably, X l The corresponding prediction offset O l =D l ·A l .

[0100] Preferably, the present invention divides features into different groups and assigns unique spatial offsets for finer-grained resampling, allowing features with high intra-class similarity to be resampled to replace features with low intra-class similarity, so that the offset generator can handle large areas of inconsistent features and refine boundaries.

[0101] Preferably, the corresponding hierarchical feature map is resampled based on the predicted offset to generate a feature map with improved local consistency, that is, a resampled feature map.

[0102] Based on all feature map groups input into the feature pyramid network, the final fused feature map is obtained:

[0103] Preferably, the final fusion feature map is obtained, specifically including:

[0104] Each feature map group is input into different layers of the feature pyramid network respectively. Based on the top-down path, the hierarchical feature map of this layer is fused with the resampled feature map of the next layer to obtain the layer fusion feature map of this layer. And so on, the layer fusion feature maps of all layers are obtained.

[0105] Based on the fusion of the layer fusion feature map and the resampled feature map in the corresponding layer, a layer scale feature map is obtained;

[0106] Based on the integration of all layer-scale feature maps, the final fused feature map is obtained.

[0107] Preferably, the hierarchical feature map X in the highest layer a (a=1,2,3,...,L) after upsampling operation and the resampled feature map of the next layer Fusion is performed to obtain the layer fusion feature map U of this layer a :

[0108]

[0109] Among them, Upsample() represents the upsampling operation;

[0110] And so on, until the layer fusion feature map of the bottom layer is output, and the layer fusion feature maps of all layers are obtained.

[0111] Preferably, a horizontal connection is performed to fuse the layer fusion feature map obtained from the top-down path with the resampled feature map of the same layer to obtain a fused layer scale feature map, such as the layer fusion feature map U of the highest layer. a With the resampled feature map of the highest layer Fusion is performed to obtain the highest-level layer-scale feature map F a :

[0112]

[0113] And so on, until the layer-scale feature map of the bottom layer is output, and the layer-scale feature maps of all layers are obtained.

[0114] Preferably, all layer-scale feature maps are integrated to obtain the final fused feature map F:

[0115] F = Concat(F 1 ,F 2 ,...,F L )

[0116] Among them, Concat() represents the connection operation along the feature dimension.

[0117] Resampling is performed based on the final fusion feature map to obtain the final output feature map:

[0118] Preferably, obtaining the final output feature map specifically includes:

[0119] Calculate the final offset based on each pixel in the final fused feature map;

[0120] Resampling is performed based on the final offset and the final fused feature map to obtain the final output feature map.

[0121] Preferably, the final fused feature map is input into the above-mentioned offset generator, and the final offset direction D and the final offset scale A are obtained accordingly. The final offset O is obtained based on the final offset direction D and the final offset scale A: O=D·A; the final fused feature map F is resampled based on the final offset O to obtain the final output feature map X final :X final =Resample(F,O), where Resample represents a resampling operation, and the final fused feature map F is spatially adjusted according to the final offset O.

[0122] Based on the final output feature map, it is input into the detection network to obtain the final detection result.

[0123] Preferably, the detection network includes: a region proposal network, a classifier, and a bounding box regression module;

[0124] The final output feature map is input into the region proposal network to obtain multiple candidate boxes and corresponding confidence scores. The candidate boxes and corresponding confidence scores together constitute the candidate region features;

[0125] The candidate region features are input into the classifier to obtain the category probability of the candidate region;

[0126] The candidate region features are input into the bounding box regression module to obtain the positioning result of the bounding box;

[0127] The final detection result is obtained based on the category probability and positioning results.

[0128] Preferably, the final output feature map X is used final The high-level semantic information in the image generates candidate target regions, and the Region Proposal Network (RPN) is used to identify regions that may contain aircraft. The RPN is used to output the feature map X final Multiple candidate boxes are generated for each position in , and a confidence score is assigned to each candidate box to indicate whether the box may contain the target. The candidate box and the corresponding confidence score together constitute the candidate region feature.

[0129] Preferably, each candidate region feature is further classified to determine whether it contains an aircraft target: based on the candidate region feature input to a classifier, it is determined whether the region is an aircraft.

[0130] Preferably, the classifier includes a ROI pooling layer, a Flatten layer, a first fully connected layer, a second fully connected layer and an activation function ReLU connected in sequence; the candidate region features are sequentially input into the ROI pooling layer, the Flatten layer, the first fully connected layer, the second fully connected layer and the activation function ReLU for processing to obtain the category probability of the candidate region.

[0131] Preferably, for each candidate area, a bounding box regression is performed to further accurately locate the boundary of the aircraft, and the candidate area features are input into a bounding box regression module. The bounding box regression module uses the position information in the feature map to adjust each candidate box so that it more accurately surrounds the aircraft target, and outputs the four coordinate offset values ​​of the bounding box, which are applied to the initial candidate box to obtain the adjusted precise position, i.e., the positioning result of the bounding box.

[0132] Preferably, the output result is post-processed based on the category probability and the positioning result. The non-maximum suppression (NMS) method is used to remove duplicate detection frames, and only the frames with higher confidence are retained. The confidence scores of multiple overlapping detection frames are calculated based on the non-maximum suppression method, the detection frame with the highest score is retained, and other frames with a large degree of overlap are suppressed, so as to obtain a more accurate detection result, that is, the final detection result.

[0133] Preferably, the local similarity offset generator, the feature pyramid network and the detection network together constitute an aircraft detection model;

[0134] Obtain annotated remote sensing image dataset as a training set;

[0135] The aircraft detection model is trained based on the training set and the comprehensive loss function to obtain a trained aircraft detection model;

[0136] The comprehensive loss function L is:

[0137] L=L cls +λL boundary

[0138] Among them, L cls represents the classification loss, L boundary represents the boundary loss, and λ represents the weight coefficient for balancing the two loss terms.

[0139] Preferably, the goal of training is to optimize the model parameters of the aircraft detection model by constructing a comprehensive loss function including classification loss and boundary loss, so as to ensure that the aircraft detection model can accurately identify the aircraft category and precisely locate the aircraft boundary.

[0140] Preferably, the classification loss L cls Used to measure the model's prediction accuracy for aircraft categories, the cross entropy loss function is used to calculate the classification loss L cls :

[0141]

[0142] Among them, N represents the number of samples, C represents the number of categories, and y ic Indicates whether the i-th sample belongs to the true label of category C, p ic It represents the probability that the model predicts that the i-th sample belongs to category C.

[0143] Preferably, the boundary loss L boundary It is used to measure the accuracy of the model in locating the boundary of the aircraft. The IoU (intersection over union) loss function is used to calculate the boundary loss L boundary :

[0144]

[0145] Among them, P represents the aircraft boundary area predicted by the model, and T represents the marked actual aircraft boundary area.

[0146] Preferably, the model is trained using a labeled remote sensing image dataset as a training set, which contains the category labels of the aircraft and their precise boundaries. The model parameters are optimized by minimizing the comprehensive loss function L:

[0147]

[0148] Among them, θ represents the model parameters and θ* represents the optimal model parameters.

[0149] The gradient descent algorithm is used for back propagation to calculate the gradient of the loss function L with respect to the model parameters θ, and the model parameters are updated to finally obtain a trained aircraft detection model.

[0150] Example 3

[0151] like Figure 3 As shown, a remote sensing image aircraft detection system based on a local similarity offset generator includes: a feature map group generation module, a feature fusion module, a preprocessing module and a result output module;

[0152] A feature map group generation module is used to obtain the remote sensing image to be tested and input it into the local similarity shift generator to obtain a plurality of different feature map groups consisting of hierarchical feature maps and corresponding resampled feature maps;

[0153] The feature fusion module is used to input all feature map groups into the feature pyramid network to obtain the final fused feature map;

[0154] A preprocessing module is used to perform resampling processing based on the final fusion feature map to obtain the final output feature map;

[0155] The result output module is used to input the final output feature map into the detection network to obtain the final detection result.

[0156] Preferably, the process implemented by the above-mentioned system modules of the present invention corresponds one-to-one with the above-mentioned corresponding method parts, and will not be described one by one here.

[0157] Example 4

[0158] Based on the same inventive concept, the present invention also provides a computer device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0159] Memory, used to store computer programs;

[0160] The processor, when used to execute the program stored in the memory, can implement a remote sensing image aircraft detection method based on a local similarity offset generator as in embodiment 1 or 2.

[0161] like Figure 4 As shown, the electronic device may include: a processor 41, a communication interface 42, a memory 43 and a communication bus 44, wherein the processor 41, the communication interface 42 and the memory 43 communicate with each other via the communication bus 44. The processor 41 may call the logic instructions in the memory 43 to execute a remote sensing image aircraft detection method based on a local similarity offset generator in Embodiment 1 or 2.

[0162] In addition, the logic instructions in the above-mentioned memory 43 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0163] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a remote sensing image aircraft detection method and system based on a local similarity offset generator, which has the following beneficial effects:

[0164] 1. Improved accuracy and robustness of aircraft detection: By introducing a local similarity offset generator, the present invention can effectively utilize spatial information in remote sensing images, thereby better maintaining feature consistency and accuracy when detecting aircraft; compared with traditional detection methods, the present invention can more accurately identify aircraft boundaries and positions, especially under complex backgrounds and multiple interference conditions, and exhibits higher robustness; the method of the present invention can stably perform performance in different types of remote sensing images, reducing false detection rates and missed detection rates.

[0165] 2. Optimize feature fusion mechanism and multi-scale feature expression: The present invention adopts a feature pyramid network (FPN) structure to effectively fuse features at different levels, ensuring the organic combination of high-level semantic information and low-level detail information; through this fusion mechanism, the model's detection capability for multi-scale aircraft targets is enhanced, and the recognition effect for aircraft of various sizes is improved; in addition, the design of the local similarity offset generator enables better retention of high-similarity areas during feature map resampling, thereby enhancing the overall feature expression capability.

[0166] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0167] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A remote sensing image aircraft detection method based on a local similarity offset generator, characterized in that: include: Obtain a remote sensing image to be tested and input it into a local similarity shift generator to obtain a plurality of different feature map groups consisting of hierarchical feature maps and corresponding resampled feature maps; Based on all the feature map groups, input into the feature pyramid network to obtain the final fused feature map; Perform resampling processing based on the final fusion feature map to obtain a final output feature map; The final output feature map is input into the detection network to obtain the final detection result.

2. The method for detecting aircraft in remote sensing images based on a local similarity offset generator according to claim 1, characterized in that: The method for obtaining the feature map group is: The remote sensing image to be measured is input into a feature extraction module to obtain multiple hierarchical feature maps at different levels; All of the hierarchical feature maps are input into a local similarity calculation module to obtain corresponding hierarchical similarities; Based on the hierarchical feature map and the corresponding hierarchical similarity, the offset generator is input to obtain a corresponding predicted offset; Based on the hierarchical feature map and the corresponding predicted offset, the hierarchical feature map is input into a resampling module to obtain a resampled feature map; The feature map group is formed based on the hierarchical feature map and the corresponding resampled feature map.

3. The method for detecting aircraft in remote sensing images based on a local similarity offset generator according to claim 2, characterized in that: The hierarchical similarity acquisition method is: Based on each pixel in the hierarchical feature map, the cosine similarity between the pixel and the pixels in its neighborhood is calculated to obtain the corresponding pixel similarity; The corresponding hierarchical similarity is obtained based on all the pixel similarities.

4. The method for detecting aircraft in remote sensing images based on a local similarity offset generator according to claim 3, characterized in that: The calculation formula of the pixel similarity is as follows: in, Represents the pixel similarity of the l-th level feature map at position (i, j), represents the pixel at position (i, j) in the feature map of the first level, m and n represent the different offsets around the pixel (i, j), and k represents the size of the neighborhood. represents the extended feature points, Represents the eigenvalue of the relevant position.

5. The method for detecting aircraft in remote sensing images based on a local similarity offset generator according to claim 2, characterized in that: The offset generator comprises: a splicing unit, a first branch, a second branch and a fusion unit; The first branch includes a first convolutional layer; The second branch includes a second convolutional layer and an activation function layer; The hierarchical feature map and the corresponding hierarchical similarity are input into the splicing unit to obtain corresponding splicing features; The splicing feature is input into the first convolutional layer to obtain the offset direction; The concatenated features are sequentially input into the second convolutional layer and the activation function layer to obtain an offset scale; The offset direction and the offset scale are input into the fusion unit to obtain the predicted offset.

6. The method for detecting aircraft in remote sensing images based on a local similarity offset generator according to claim 1, characterized in that: The final fusion feature map is obtained, including: Each of the feature map groups is input into different layers of the feature pyramid network respectively, and the hierarchical feature map in the current layer is fused with the resampled feature map in the next layer based on a top-down path to obtain a layer fusion feature map of the current layer, and so on to obtain the layer fusion feature maps of all layers; Based on the fusion of the layer fusion feature map and the resampled feature map in the corresponding layer, a layer scale feature map is obtained; Based on the integration of all the layer-scale feature maps, the final fused feature map is obtained.

7. The method for detecting aircraft in remote sensing images based on a local similarity offset generator according to claim 1, characterized in that: The final output feature map is obtained, including: Calculate a final offset based on each pixel in the final fused feature map; Resampling is performed based on the final offset and the final fused feature map to obtain the final output feature map.

8. The method for detecting aircraft in remote sensing images based on a local similarity offset generator according to claim 1, characterized in that: The detection network includes: a region proposal network, a classifier and a bounding box regression module; The final output feature map is input into the region proposal network to obtain a plurality of candidate boxes and corresponding confidence scores, wherein the candidate boxes and corresponding confidence scores together constitute candidate region features; The candidate region features are input into the classifier to obtain the category probability of the candidate region; The candidate region features are input into the bounding box regression module to obtain a positioning result of the bounding box; The final detection result is obtained based on the category probability and the positioning result.

9. The method for detecting aircraft in remote sensing images based on a local similarity offset generator according to claim 1, characterized in that: The local similarity offset generator, the feature pyramid network and the detection network together constitute an aircraft detection model; Obtain annotated remote sensing image dataset as a training set; Training the aircraft detection model based on the training set and the comprehensive loss function to obtain a trained aircraft detection model; The comprehensive loss function L is: L=L cls +λL boundary Among them, L cls represents the classification loss, L boundary represents the boundary loss, and λ represents the weight coefficient for balancing the two loss terms.

10. A remote sensing image aircraft detection system based on a local similarity offset generator, applied to a remote sensing image aircraft detection method based on a local similarity offset generator as claimed in any one of claims 1 to 9, characterized in that: include: Feature map group generation module, feature fusion module, preprocessing module and result output module; The feature map group generation module is used to obtain the remote sensing image to be tested and input it into the local similarity shift generator to obtain a plurality of different feature map groups consisting of hierarchical feature maps and corresponding resampled feature maps; The feature fusion module is used to input all the feature map groups into a feature pyramid network to obtain a final fused feature map; The preprocessing module is used to perform resampling processing based on the final fusion feature map to obtain a final output feature map; The result output module is used to input the final output feature map into the detection network to obtain the final detection result.

Images