Remote sensing image aerial aircraft detection method based on ambiguity guided region screening

Through the method of ambiguity-guided area screening, combined with multi-layer perceptron and Transformer models, the problem of inaccurate detection of airplanes in remote sensing images is solved, the detection accuracy and model attention is improved, and more efficient airplane detection is achieved .

CN119068179AActive Publication Date: 2024-12-03NAT UNIV OF DEFENSE TECH
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411282303.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-13
Publication Date
2024-12-03
Estimated Expiration
2044-09-13

AI Technical Summary

Technical Problem

The current remote sensing image is inaccurate in the airplane detection, and the general target detection algorithm fails to fully consider the characteristics of high-speed target detection, resulting in a decrease in detection accuracy.

Method used

Using a method of guiding area screening based on ambiguity, the blurring and image features of the remote sensing image are fused, and the target area is screened using a multi-layer perceptron and Transformer model, considering the blurring phenomenon and background changes caused by airplane movement, and improving detection accuracy.

Benefits of technology

Through ambiguity-guided area screening, irrelevant background is effectively removed, the accuracy of airplane detection and model attention is improved, and the detector's ability to identify aircraft areas is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068179A_ABST
    Figure CN119068179A_ABST
Patent Text Reader

Abstract

The invention discloses a remote sensing image aerial aircraft detection method based on ambiguity guided area screening, which is applied to the field of target detection and comprises the following steps: acquiring a remote sensing image of an aerial aircraft; calculating the ambiguity of the remote sensing image, and extracting the image features of the remote sensing image through the backbone network; fusing the ambiguity and the image features to obtain fused features; screening each image region in the remote sensing image based on the fusion features through the trained multi-layer perceptron to obtain a target image region; and obtaining an aerial aircraft position based on the image features of the target image area. According to the method, the motion blur phenomenon generated in the aerial aircraft motion process and the characteristic that the background of the aerial aircraft has arbitrariness due to aircraft motion are considered, the blur degree of the remote sensing image of the aerial aircraft is estimated, the target area is screened according to the blur degree, and the irrelevant background is screened out; and the attention of the detector on the airplane area is improved, so that the detection accuracy of the model is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the field of target detection, and in particular to a remote sensing image aerial aircraft detection method based on fuzziness-guided region screening. Background Art

[0002] Since aircraft detection is of great significance in both civil and military fields, the problem of aircraft detection in remote sensing images has been widely studied and many achievements have been made. However, current research mainly focuses on stationary targets in airports, and relatively few studies have been conducted on the detection of flying aircraft in remote sensing images. Deep learning (DL) technology has made significant progress in the detection of stationary aircraft in remote sensing images. Through carefully designed and trained feature extractors, such as convolutional neural networks (CNNs), target detection algorithms can cleverly capture robust target features. Therefore, in the field of stationary aircraft detection, DL-based methods have shown impressive performance and accuracy.

[0003] At present, the aircraft detection model in remote sensing images is usually based on the general target detection model, which is mainly used in static or low-speed target detection scenarios, and does not fully consider the characteristics of high-speed target detection, resulting in a decrease in the accuracy of the general target detection algorithm. Therefore, how to detect aircraft in the air and how to ensure the accuracy of detection are currently urgent issues that need to be solved. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a method for detecting aircraft in the air from remote sensing images based on ambiguity-guided region screening, which solves the problem of inaccurate detection of aircraft in the air in the prior art.

[0005] In order to solve the above technical problems, the present invention provides a method for detecting aircraft in the air from remote sensing images based on fuzziness-guided region screening, comprising:

[0006] Acquire remote sensing images from aerial aircraft;

[0007] Calculating the blur of the remote sensing image and extracting image features of the remote sensing image through a backbone network;

[0008] Fusing the blur and the image feature to obtain a fused feature;

[0009] By using a trained multi-layer perceptron and based on the fusion features, each image region in the remote sensing image is screened to obtain a target image region;

[0010] The image features of the target image area are input into the trained Transformer encoder and decoder to obtain the position of the aircraft in the air.

[0011] Optionally, calculating the blurriness of the remote sensing image and extracting image features of the remote sensing image through a backbone network includes:

[0012] Extracting multi-scale image features of the remote sensing image through a Swin-Tranformer backbone network;

[0013] The remote sensing image is divided into a plurality of image regions, and the blur degree of each image region is calculated using an improved edge gradient algorithm to obtain a blur degree map corresponding to the remote sensing image.

[0014] Optionally, dividing the remote sensing image into a plurality of image regions, and using an improved edge gradient algorithm to calculate the blur degree of each image region to obtain a blur degree map corresponding to the remote sensing image includes:

[0015] Performing Sobel operator convolution on each of the image regions in the horizontal direction and the vertical direction to obtain a horizontal gradient component and a vertical gradient component;

[0016] Calculate an average gradient component of the target image based on the horizontal gradient component and the vertical gradient component;

[0017] Based on the average gradient component, the clear background area and the pure color background area are suppressed by an extreme value suppression filter to obtain the blur degree map.

[0018] Optionally, fusing the blur and the image feature to obtain a fused feature includes:

[0019] Upsampling the image features and the blurriness to obtain image features and blurriness of uniform scale;

[0020] Assigning a weight matrix to each of the uniformly scaled image features and blur; the weight matrix is ​​adaptively updated according to the image region evaluation result of the multi-layer perceptron;

[0021] The fusion feature is calculated based on the weight matrix, the image features of the unified scale and the blur.

[0022] Optionally, the training process of the multilayer perceptron includes:

[0023] Obtain training sample images and corresponding training fusion features;

[0024] While keeping the center point of the target object bounding box on the training fusion feature unchanged, the length and width of the target object bounding box are enlarged to obtain an enlarged target object bounding box;

[0025] When the coordinates of the pixel point on the training fusion feature are within the enlarged target object boundary box, it is marked as a target area label;

[0026] Otherwise, it is marked as a background area label;

[0027] The multilayer perceptron is trained based on the training fusion features, the target area label and the background area label, and when the training end condition is reached, the trained multilayer perceptron is obtained.

[0028] Optionally, the loss function in the training process of the multilayer perceptron is a Focal Loss loss function.

[0029] Optionally, the method of screening each image region in the remote sensing image based on the fusion feature by using a trained multi-layer perceptron to obtain a target image region includes:

[0030] Inputting the fused features into the trained multi-layer perceptron to obtain predicted probability values ​​of each image region based on the fused features;

[0031] The image region whose predicted probability value is greater than a preset threshold is used as the target image region.

[0032] It can be seen that the present invention obtains a remote sensing image of an airplane in the air; calculates the blurriness of the remote sensing image, and extracts the image features of the remote sensing image through a backbone network; fuses the blurriness and image features to obtain fused features; uses a trained multi-layer perceptron and screens each image area in the remote sensing image based on the fused features to obtain a target image area; inputs the image features of the target image area into the encoder and decoder of the trained Transformer to obtain the position of the airplane in the air. The present invention takes into account the motion blur phenomenon generated during the movement of the airplane in the air and the fact that the background of the airplane in the air is arbitrary due to the movement of the airplane. Therefore, the blurriness of the remote sensing image of the airplane in the air is estimated, and the target area is screened according to the different blurriness, and irrelevant background is screened out, thereby increasing the detector's attention to the airplane area, thereby improving the detection accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0034] Figure 1A flowchart of a method for detecting aircraft in the air from remote sensing images based on fuzziness-guided region screening provided by an embodiment of the present invention;

[0035] Figure 2 An example diagram of a remote sensing image of an aircraft in the air provided by an embodiment of the present invention;

[0036] Figure 3 An example diagram of an aircraft detection result in the air provided by an embodiment of the present invention;

[0037] Figure 4 An example flow chart of a method for detecting aircraft in the air from remote sensing images based on fuzziness-guided region screening provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] The following is an introduction to the Transformer neural network model:

[0040] Transformer is a deep learning architecture based on the self-attention mechanism, which has achieved great success in fields such as natural language processing (NLP). The core concepts of Transformer include attention mechanism, encoder-decoder architecture, and multi-head attention. Attention mechanism is an important concept in neural networks, which allows the model to focus on certain parts of the input sequence while ignoring other parts. In Transformer, self-attention mechanism, also known as internal attention, is particularly used. The self-attention mechanism enables the model to automatically highlight important words or phrases to understand the meaning of the sentence. Multi-head attention is a key innovation in Transformer, which allows the model to learn attention distribution in different representation subspaces. This means that the model can pay attention to different positions of the input sequence at the same time, thereby capturing richer contextual information. Transformer has efficient parallel computing capabilities. Since the calculation of the attention mechanism can be performed in parallel, the Transformer model has extremely high computing efficiency and parallel processing capabilities. In addition, with powerful representation capabilities, the Transformer model can effectively capture the global information of the input data, so it has achieved significant performance improvements in the field of natural language processing, such as language modeling, translation and other tasks. The Transformer model has been widely used in various tasks, including but not limited to: language translation, text generation, dialogue systems, language models, image processing, recommendation systems, etc. In summary, Transformer is a powerful deep learning model, and its emergence has greatly promoted the development of natural language processing and other fields.

[0041] Since aircraft detection is of great significance in both civil and military fields, the problem of aircraft detection in remote sensing images has been widely studied and many achievements have been made. In the civil field, aircraft detection in the air can assist civil aviation positioning and air traffic control; in the military field, it can provide early battlefield situation awareness, so its practical significance exceeds that of stationary aircraft detection. However, there are relatively few studies on the detection of flying aircraft in remote sensing images, because previous studies have mainly focused on stationary targets in airports. Therefore, more and more researchers have begun to explore methods to detect moving aircraft in remote sensing images.

[0042] Deep learning (DL) technology has made significant progress in the detection of stationary aircraft in remote sensing images. Through carefully designed and trained feature extractors, such as convolutional neural networks (CNNs), object detection algorithms can cleverly capture robust target features. Therefore, in the field of stationary aircraft detection, DL-based methods have shown impressive performance and accuracy. Aircraft detection models in remote sensing images are usually built on the basis of general target detection models, which are mainly applied to static or low-speed target detection scenarios, and do not fully consider the characteristics of high-speed target detection. Studies have shown that motion blur in images can lead to a decrease in the accuracy of general target detection algorithms, and the background of moving aircraft is also changeable, which also affects the detector's extraction of target features.

[0043] Based on the phenomenon that the blur degree is different due to the difference in the motion state of the aircraft and the background, the present invention models the blur degree of the image, and then selects the image area that is more likely to contain the aircraft and removes the irrelevant background. For details, please refer to Figure 1 , Figure 1 A flowchart of a method for detecting aircraft in remote sensing images based on fuzziness-guided region screening is provided in an embodiment of the present invention. The method may include:

[0044] S101: Acquire remote sensing images of aircraft in the air.

[0045] The execution subject of this embodiment is a terminal. This embodiment does not limit the type of terminal, as long as it can complete the operation of detecting an aircraft in the air. The remote sensing image of the aircraft in the air can be referenced Figure 2 .

[0046] S102: Calculate the blur of the remote sensing image and extract image features of the remote sensing image through the backbone network.

[0047] This embodiment also uses a feature extraction algorithm to extract image features of remote sensing images. It should be noted that the image features are multi-scale features. This embodiment does not limit the feature extraction algorithm, that is, it does not limit the backbone network, as long as image feature extraction can be achieved.

[0048] Image blur is a very important factor in image degradation. There are many factors that may cause image blur during image acquisition, transmission, and processing. For example, when acquiring an image, incorrect focusing will cause out-of-focus blur, the relative movement of the scene and the camera will cause motion blur, and the blur caused by high-frequency loss after image compression. Blur reduces the clarity of the image, seriously affects the image quality, and leads to difficulties or even failures in image analysis, processing, and reception. Therefore, effective fuzzy evaluation methods must be used to control the use of fuzzy images to improve the overall performance of the system. Fuzziness is a concept that describes the degree of image blur. The clearer the image, the higher the quality and the smaller the blur; the blurrier the image, the lower the quality and the greater the blur. Therefore, fuzziness can be used to describe the degree of blur of an image. This embodiment does not limit the evaluation algorithm of the blurriness of remote sensing images. For example, blurriness evaluation algorithms can be divided into several major categories: (1) pixel-based technology, including analyzing the statistical characteristics of pixel grayscale values ​​and the correlation between pixels; (2) transform domain-based technology, which uses the principle that the more high-frequency components in the transform domain, the clearer the image, and the fewer high-frequency components, the blurrier the image; (3) image gradient-based technology, which uses the gradient of the image edge to measure the blurriness of the image, and the larger the gradient, the clearer the image. In addition, there are also fewer algorithms based on histograms.

[0049] Further, in order to improve the accuracy of fuzziness assessment, the above-mentioned calculation of fuzziness of remote sensing images and extraction of image features of remote sensing images through the backbone network may include the following steps, which may specifically include:

[0050] Step 21: Extract multi-scale image features of remote sensing images through the Swin-Tranformer backbone network.

[0051] Swin-Transformer is a deep learning model for computer vision tasks. It is based on the Transformer architecture, but it has been improved to adapt to computer vision tasks. The main innovation of Swin-Transformer lies in the design of its self-attention mechanism. The traditional Transformer model uses a global self-attention mechanism, which means that it considers all elements in the input at once. However, in computer vision tasks, local information is usually more important than global information. Therefore, Swin Transformer introduces a hierarchical self-attention mechanism, which allows the model to focus on different parts of the input at different levels. This embodiment uses the Swin-Tranformer backbone network to extract multi-scale image features. Based on the hierarchical design of Swin-Transformer, features of different scales can be extracted from different levels, which is helpful for processing visual scenes with large changes in target size. A self-attention mechanism based on a moving window is introduced, which can better capture local features in the image, so it has an efficient and powerful feature extraction effect.

[0052] Furthermore, in order to further adapt to the actual application scenarios of the present invention and improve the accuracy of fuzziness assessment, the above-mentioned remote sensing image is divided into multiple image regions, and the fuzziness of each image region is calculated using the improved edge gradient algorithm to obtain the fuzziness degree map corresponding to the remote sensing image, which may include the following steps, which may specifically include:

[0053] Step 22: Divide the remote sensing image into multiple image regions, use the improved edge gradient algorithm to calculate the blur degree of each image region, and obtain a blur degree map corresponding to the remote sensing image.

[0054] When the target object is motion blurred, the change of its image edge will become smoother, resulting in a decrease in the gradient value. On the contrary, a decrease in the gradient value indicates that motion blur has occurred. Therefore, this embodiment evaluates the blur degree of the image based on the image gradient value. And further optimization and improvement are made based on the traditional edge gradient algorithm to make it more suitable for the application scenario of the aerial aircraft detection of this embodiment. This embodiment divides the remote sensing image into 16x16 image blocks, that is, each image block represents an image area. The blur degree of each image block is calculated using the improved edge gradient algorithm, so as to obtain the entire blur degree map corresponding to the remote sensing image.

[0055] Furthermore, the above-mentioned dividing the remote sensing image into a plurality of image regions, using the improved edge gradient algorithm to calculate the blur degree of each image region, and obtaining the blur degree map corresponding to the remote sensing image may include the following steps:

[0056] Step 221: performing Sobel operator convolution on each image region in the horizontal direction and the vertical direction respectively to obtain a horizontal gradient component and a vertical gradient component;

[0057] Step 222: Calculate the average gradient component of the target image based on the horizontal gradient component and the vertical gradient component;

[0058] Step 223: Based on the average gradient component, the clear background area and the pure color background area are suppressed by an extreme value suppression filter to obtain a blur degree map.

[0059] In the airborne aircraft detection scene, the gradient of the edge of the airborne aircraft is smoother and smaller than that of the static background (such as clouds). On the contrary, in the pure color image area (lack of edge changes), the gradient of the airborne aircraft is larger. Therefore, the Gaussian function can be used as an extreme value suppression filter to suppress the clear background area with extremely large gradients and the pure color background area with extremely small gradients. Its mean parameter is updated in the model process to enhance the ability of the improved edge gradient algorithm to distinguish different backgrounds and effectively highlight the potential aircraft area. The improved edge gradient algorithm of this embodiment has good sensitivity and accuracy. The steps of the improved edge gradient algorithm are as follows:

[0060] (1) Convolve the image block with the horizontal and vertical Sobel operators respectively to obtain the horizontal gradient component and the vertical gradient component The Sobel horizontal and vertical operators are as follows:

[0061] .

[0062] (2) The average gradient component of the image block is calculated by the following formula: , where n is the total number of pixels:

[0063] .

[0064] (3) Through extreme value suppression filter The final image block blur is obtained, where x is the input gradient and u is the mean gradient of the entire image.

[0065] (4) Perform the above operation on all image blocks to obtain the blur degree map of the entire image.

[0066] S103: Fusion the blur and the image features to obtain fusion features.

[0067] Specifically, this embodiment adaptively fuses multi-scale image features and introduces a blur degree map into the multi-scale features to obtain final fused features.

[0068] Further, in order to ensure the fusion effect, the above blur and image features are fused to obtain fusion features, which may include the following steps, which may specifically include:

[0069] Step 31: up-sample the image features and blur to obtain image features and blur of uniform scale;

[0070] Step 32: assigning a weight matrix to each uniform-scale image feature and blur; the weight matrix is ​​adaptively updated according to the image region evaluation result of the multi-layer perceptron;

[0071] Step 33: The fusion feature is obtained based on the weight matrix, the image features of uniform scale and the blur calculation.

[0072] The multi-scale image features in this embodiment contain feature information at different levels, and fusing them can better integrate features at different scales. In addition, the blur level map is fused to introduce the motion information of the aircraft in the air, thereby obtaining more targeted fusion features. The specific steps are as follows:

[0073] (1) Upsample the multi-scale image features and blur level maps to a uniform scale.

[0074] (2) Assign a weight matrix to each feature (including the blur level map). With the weight matrix Multiply and add to get the fusion feature , the formula is:

[0075] .

[0076] (3) The fusion operation conforms to the differential characteristics, and the weight matrix can be adaptively updated according to the aircraft area scoring results in the subsequent multi-layer perceptron training.

[0077] S104: By using the trained multi-layer perceptron and based on the fusion features, each image region in the remote sensing image is screened to obtain a target image region.

[0078] Unlike static target detection, the background of an airplane in the air in a remote sensing image is arbitrary due to the movement of the airplane, and has very low semantic relevance to the target. When predicting the target bounding box on the feature map obtained based on the image features, the network may incorrectly associate the background with the target. Therefore, the presence of redundant and irrelevant background elements will significantly affect the accuracy of target positioning. To address this issue, this embodiment uses MLP (multi-layer perceptron) to evaluate the probability of the existence of the target (i.e., the airplane in the air) on the fused features. Based on this, the irrelevant background is eliminated as much as possible, and the model's attention to the aircraft area is enhanced.

[0079] Specifically, the training process of the multilayer perceptron may include the following steps:

[0080] Step 41: Obtain training sample images and corresponding training fusion features;

[0081] Step 42: While keeping the center point of the target object bounding box on the training fusion feature unchanged, the length and width of the target object bounding box are enlarged to obtain an enlarged target object bounding box;

[0082] Step 43: When the coordinates of the pixel point on the training fusion feature are within the enlarged target object boundary box, it is marked as the target area label;

[0083] Step 44: Otherwise, mark it as a background area label;

[0084] Step 45: The multi-layer perceptron is trained based on the training fusion features, the target region labels, and the background region labels. When the training end condition is reached, a trained multi-layer perceptron is obtained.

[0085] While keeping the center point of the bounding box on the fused feature unchanged, its length and width can be expanded to twice its original size to alleviate the boundary effect. After that, if the coordinate (i, j) on the fused feature falls within the expanded bounding box, it is marked as the aircraft area and marked as 1; otherwise, it will be marked as the background area 0. This is how the training samples are labeled.

[0086] Considering that there is a significant difference in the ratio of the aircraft area to the background area in the training sample data set, and the aircraft area only accounts for a very small part, Focal Loss is used as the loss function for the multi-layer perceptron training to solve the sample imbalance problem. The training focus can be placed on the area of ​​the aircraft sample, thereby improving the accuracy of predicting the probability that the area contains an aircraft.

[0087] Furthermore, the above-mentioned method of obtaining the target image area by screening each image area in the remote sensing image based on the fusion features through the trained multi-layer perceptron may include the following steps, which may specifically include:

[0088] Step 51: Input the fused features into the trained multi-layer perceptron to obtain the predicted probability value of each image region based on the fused features;

[0089] Step 52: Taking the image area whose predicted probability value is greater than the preset threshold as the target image area

[0090] The trained multi-layer perception system can predict the score of the image area with fused features, that is, the probability value of the existence of the target, and take the image area with the probability value exceeding the preset threshold as the target image area; or it can also sort the image areas based on the prediction score and take the top 40% of the image areas as the target image areas.

[0091] S105: Input the image features of the target image area into the trained Transformer encoder and decoder to obtain the position of the aircraft in the air.

[0092] The target detection model of this embodiment adopts the Transformer architecture, which is mainly composed of two parts: an encoder and a decoder. The encoder is responsible for processing the input sequence, while the decoder generates the target sequence based on the output of the encoder. Each encoder and decoder is composed of multiple identical layers (blocks) stacked together, usually 6 layers. In this embodiment, the target image area screened in step S104, that is, the area where the target is most likely to exist, is further updated in the encoder-decoder structure, while the background part remains unchanged, so as to train the encoder and decoder of the Transformer. This embodiment obtains the image features of the target image, and inputs the image features of the area into the target detection model of the Transformer architecture to obtain the position of the aircraft in the air, such as Figure 3 Furthermore, this embodiment can further continue to identify and classify the located aircraft in the air, that is, the type of the aircraft can be identified.

[0093] The method for detecting aircraft in a remote sensing image based on fuzziness-guided area screening provided by the embodiment of the present invention is applied, by acquiring a remote sensing image of an aircraft in the air; calculating the fuzziness of the remote sensing image, and extracting the image features of the remote sensing image through a backbone network; fusing the fuzziness and the image features to obtain fusion features; using a trained multi-layer perceptron, and based on the fusion features, screening each image region in the remote sensing image to obtain a target image region; and obtaining the position of the aircraft in the air based on the image features of the target image region. The present invention takes into account the motion blur phenomenon generated during the movement of the aircraft in the air and the fact that the background of the aircraft in the air is arbitrary due to the movement of the aircraft, estimates the degree of blur for the remote sensing image of the aircraft in the air, screens the target region according to the degree of blur, and screens out irrelevant background, thereby increasing the detector's attention to the aircraft region, and thus improving the detection accuracy of the model.

[0094] In order to make the present invention easier to understand, please refer to Figure 4 , Figure 4 An example flow chart of a method for detecting aircraft in a remote sensing image based on fuzziness-guided region screening provided by an embodiment of the present invention may specifically include:

[0095] Firstly, the image features of the remote sensing image of the backbone network are used and the fuzziness is calculated using the improved edge gradient algorithm. Then, the fuzziness is introduced into the image features through the adaptive feature fusion module to obtain the fused features. Finally, the fused features are scored through the aircraft area scoring module. The image areas with the top 40% scores are taken as the target image areas. The image features of the target image areas continue to be updated in the encoder and decoder, while other areas remain unchanged. Finally, the aircraft position and category are output.

[0096] Step 1: The backbone network uses Swin-Tranformer to extract the multi-scale features of the image (i.e., image features). At the same time, the remote sensing image is divided into 16x16 image blocks, and the blur degree is calculated on the image blocks using the improved edge gradient algorithm to obtain the image blur degree map.

[0097] Step 2: The adaptive feature fusion module adaptively fuses the multi-scale features and introduces the blur degree map into the multi-scale features to obtain the final fused features.

[0098] Step 3: The aircraft area scoring module scores the fused features, and the image area before 40% of the scoring area is taken as the target image area.

[0099] Step 4: The encoder will focus on the filtered target image area and update the image features, while the background area remains unchanged. The decoder processes the encoder output to obtain the final aircraft position.

[0100] Object detection technology has been developed for more than 20 years. It provides an important research direction in the field of computer vision and has been widely used in fields such as pedestrian detection, face recognition, text extraction and remote sensing. Traditional object detection methods usually include steps such as region selection, manual feature extraction and classifier classification. However, the most critical step in manual feature extraction is difficult to adapt to the diversity of different objects and scenes, resulting in poor performance of traditional object detection methods in complex scenes. Due to the outstanding performance of convolutional neural networks in object detection, a large number of object detection algorithms based on deep learning have emerged rapidly. This embodiment implements various object detection methods on the established aerial aircraft simulation data set. The results are shown in Table 1. Compared with the current mainstream object detection model, the detection rate and recall rate of this method are higher.

[0101] Table 1 Results comparison table

[0102]

[0103] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0104] Finally, it should be noted that, in this article, relationships such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0105] The above is a detailed introduction to the remote sensing image aerial aircraft detection method based on ambiguity-guided area screening provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for detecting aircraft in remote sensing images based on fuzziness-guided region screening. It is characterized in that include: Acquire remote sensing images from aerial aircraft; Calculating the blur of the remote sensing image and extracting image features of the remote sensing image through a backbone network; Fusing the blur and the image feature to obtain a fused feature; By using a trained multi-layer perceptron and based on the fusion features, each image region in the remote sensing image is screened to obtain a target image region; The image features of the target image area are input into the trained Transformer encoder and decoder to obtain the position of the aircraft in the air.

2. The method for detecting aircraft in the air from remote sensing images based on fuzziness-guided region screening according to claim 1, It is characterized in that Calculating the blur of the remote sensing image and extracting image features of the remote sensing image through a backbone network, including: Extracting multi-scale image features of the remote sensing image through a Swin-Tranformer backbone network; The remote sensing image is divided into a plurality of image regions, and the blur degree of each image region is calculated using an improved edge gradient algorithm to obtain a blur degree map corresponding to the remote sensing image.

3. The method for detecting aircraft in the air from remote sensing images based on fuzziness-guided region screening according to claim 2, It is characterized in that The remote sensing image is divided into a plurality of image regions, and the fuzziness of each image region is calculated using an improved edge gradient algorithm to obtain a fuzziness degree map corresponding to the remote sensing image, including: Performing Sobel operator convolution on each of the image regions in the horizontal direction and the vertical direction to obtain a horizontal gradient component and a vertical gradient component; Calculate an average gradient component of the target image based on the horizontal gradient component and the vertical gradient component; Based on the average gradient component, the clear background area and the pure color background area are suppressed by an extreme value suppression filter to obtain the blur degree map.

4. The method for detecting aircraft in the air from remote sensing images based on fuzziness-guided region screening according to claim 1, It is characterized in that The blur and the image feature are fused to obtain a fused feature, including: Upsampling the image features and the blurriness to obtain image features and blurriness of uniform scale; Assigning a weight matrix to each of the uniform-scale image features and blur; the weight matrix is ​​adaptively updated according to the image region evaluation result of the multi-layer perceptron; The fusion feature is calculated based on the weight matrix, the image features of the unified scale and the blur.

5. The method for detecting aircraft in the air from remote sensing images based on ambiguity-guided region screening according to any one of claims 1 to 4, It is characterized in that The training process of the multilayer perceptron includes: Obtain training sample images and corresponding training fusion features; While keeping the center point of the target object bounding box on the training fusion feature unchanged, the length and width of the target object bounding box are enlarged to obtain an enlarged target object bounding box; When the coordinates of the pixel point on the training fusion feature are within the enlarged target object boundary box, it is marked as a target area label; Otherwise, it is marked as a background area label; The multilayer perceptron is trained based on the training fusion features, the target area label and the background area label, and when the training end condition is reached, the trained multilayer perceptron is obtained.

6. The method for detecting aircraft in the air from remote sensing images based on fuzziness-guided region screening according to claim 1, It is characterized in that By using a trained multi-layer perceptron and based on the fusion features, each image region in the remote sensing image is screened to obtain a target image region, including: Inputting the fused features into the trained multi-layer perceptron to obtain a predicted probability value of each image region based on the fused features; The image region whose predicted probability value is greater than a preset threshold is used as the target image region.

7. The method for detecting aircraft in the air from remote sensing images based on fuzziness-guided region screening according to claim 1, It is characterized in that The loss function in the training process of the multilayer perceptron is the Focal Loss loss function.

Citation Information

Patent Citations

  • Image definition evaluation method and device

    CN110717922A

  • Test paper surface texture defect detection method based on gray scale gradient clustering

    CN111179225A

  • Contextual visual-based SAR target detection method and apparatus, and storage medium

    US20230184927A1