Synthetic aperture radar image aircraft detection method based on computer vision
By combining diffusion model and attention mechanism in synthetic aperture radar images, the problems of multi-size targets and background interference are solved, and high accuracy and high efficiency of aircraft target detection are achieved. The experimental results show that the AP value is significantly improved.
Patent Information
- Application Number
- CN202411123698.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2024-11-15
AI Technical Summary
The prior art has problems in aircraft target detection in synthetic aperture radar images, such as difficulty in identifying multi-size targets, complex scattering characteristics and strong background interference, resulting in low detection accuracy and efficiency, especially in fixed receptive field technology, which is difficult to identify small size or backward direction. Aircraft targets with weak scattering information.
Using a combination of diffusion model and attention mechanism, multi-scale features are extracted through the encoder, and the dynamic gradient bidirectional attention module calculates the spatial attention weight. The decoder generates aircraft targets and models through the diffusion method, avoiding adjacent attention scores. The problem of large differences and the attention score matrix showing stripes.
The detection accuracy of aircraft targets and categories was improved, and the detection efficiency was maintained. The experimental results showed that the AP value on the SAR-AIRcraft-1.0 data set reached 62.164, the AP0.5 value was 94.87, and the AP0.75 value was 68.62. , proves the effectiveness of the method.
Smart Images

Figure FT_1 
Figure FT_2 
Figure FT_3
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer vision and image processing technology, and in particular relates to real-time detection of aircraft targets in synthetic aperture radar images by computer vision methods. Specifically, a method based on a combination of a diffusion model and an attention mechanism is used to generate the position and model of an aircraft in an image. Background Art
[0002] Synthetic aperture radar is a microwave remote sensing imaging technology with the ability to collect images in all weather conditions. This technology has been widely used in various fields such as topographic mapping, disaster risk monitoring and ecology. In these applications, the aircraft target detection process is a basic component. The information acquisition of aircraft targets can effectively assist in dynamic monitoring, situation analysis and emergency rescue of key areas. Compared with targets such as vehicles and ships, aircraft targets have complex structures and complex scattering mechanisms, which is a major challenge for accurate detection. Aircraft in synthetic aperture radar images have a variety of target sizes. Different types of aircraft targets and images with different resolutions will cause changes in target size. This multi-size target problem will cause problems when using fixed receptive field technology for feature extraction. As the network goes deeper, it is easy to miss aircraft targets with small size or weak backscattering information. At the same time, another difficulty is that image targets are usually discrete scattering points, and the scattering characteristics of aircraft targets are discrete and discontinuous. Structural incompleteness and weak correlation between components make it difficult for the classic network model structure to achieve accurate prediction. In addition, complex scenes and strong background interference are often encountered during the error detection process. In real scenes, buildings, vehicles and certain metal structures around the aircraft exhibit strong scattering characteristics similar to those of aircraft targets. Therefore, conventional detection methods usually have difficulty detecting aircraft targets in synthetic aperture radar images.
[0003] At present, methods based on deep learning with powerful feature learning capabilities have made significant progress in synthetic aperture radar image target detection. Existing methods are usually based on convolutional neural networks and Transformer structures. In the process of generating target detection boxes, a common practice is to use FPN or prior boxes to generate region proposals and bounding boxes, but this design requires prior knowledge and is usually combined with a complex network structure. The detection speed usually cannot meet the requirements of real-time detection. Summary of the invention
[0004] To solve the above technical problems, the present invention proposes an aircraft detection method (Aircraft Diffusion Det) for synthetic aperture radar images by combining a diffusion model with an attention mechanism. The method can improve the detection accuracy of aircraft targets and categories and maintain a high detection efficiency.
[0005] The present invention applies the method combining diffusion model and attention mechanism to the aircraft detection task in synthetic aperture radar image for the first time. The Aircraft Diffusion Det detection process includes the following steps: Step 1: extracting multi-scale features of the original image through the encoder; Step 2: calculating the spatial attention weights and obtaining weighted image features through the dynamic gradient bidirectional attention (DGBA) module; Step 3: inputting the features obtained in step 2 into the detection decoder, and the decoder generates aircraft targets and models through the diffusion method.
[0006] Among them, the dynamic gradient bidirectional attention module is a part of the present invention. This module is specifically used to extract attention information from synthetic aperture radar images. It is an improved calculation process based on the CA attention mechanism. Compared with the CA module attention module, the dynamic gradient bidirectional attention module avoids the problem of large differences in adjacent attention scores and the striped appearance of the attention score matrix in space. BRIEF DESCRIPTION OF THE DRAWINGS
[0007] The accompanying drawings, which constitute a part of this application, are used to provide a further understanding of the present invention. In the accompanying drawings: Figure 1 The overall flow chart of the aircraft detection method Figure 2 This is the structure diagram of the dynamic gradient bidirectional attention module Figure 3 Decoder structure diagram Specific implementation methods
[0008] In order to facilitate those skilled in the art to understand and implement the present invention, the present invention is further described in detail below with reference to the accompanying drawings.
[0009] The following is a detailed description of the overall process of Aircraft Diffusion Det. Figure 1, the aircraft detection method includes three steps, namely: Step 1: Encoder extracts features; Step 2: Dynamic gradient bidirectional attention mechanism; Step 3: Decoder outputs target box and category; The Encoder in Step 1 represents the encoder, which can be any neural network model that can extract multi-scale features (such as ResNet, Swin, etc.), and outputs feature tensors of four scales, with height and width of 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the illustrated image, respectively, and then these feature tensors will be input into FPN for fusion. The encoder model corresponding to the experimental data of this description uses ResNet50+FPN. Step 2 takes the feature tensors of four scales output by Step 1 as input and inputs them into the dynamic gradient bidirectional attention module, which can calculate the feature importance of different regions in space, so as to filter certain background information and highlight the aircraft target information. This description refers to the tensor output by this module as the weighted feature tensor (Features). The detailed structure of the dynamic gradient bidirectional attention module will be described in detail later. Step 3 contains two inputs, one is the weighted feature tensor output by step 2, and the other is the noise box (NoisedBoxes). The generation process of the noise box is different in the model training and model inference stages. In the training stage, the noise box is formed by filling the ground truth box and adding Gaussian noise. In the inference stage, the noise box is a simple Gaussian noise. The decoder will analyze the weighted feature tensor to obtain the aircraft bounding box (Boxes) and aircraft model (Classes) matrix. As the result of the entire detection method, the two matrices can identify several aircraft target positions and corresponding aircraft models in the image. Among them, the decoder is a Denoising Diffusion ImplicitModels (DDIM). Its forward propagation is a denoising process that iterates S times (the value of S can be adjusted according to needs). Each round of iteration will input the bounding box matrix generated by the previous round of iteration, and denoise it after predicting the noise to generate a bounding box matrix with less noise, and so on. The specific process of the decoder will be described in detail later.
[0010] Step 1 is described in detail below. Step 1 is designed to extract multi-scale features of the original image. It includes an encoder for extracting image features and an FPN for fusing multi-scale information. The encoder can use any structure that can receive image data and output feature maps of at least 4 scales (1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the original image width and height) (such as ResNet, Swin Transformer, etc.). The encoder model corresponding to the experimental data in this description uses ResNet50+FPN.
[0011] Step 2 is described in detail below. Step 2 is designed to extract the attention weights of image features, thereby filtering out certain background information and highlighting the aircraft target information. This step contains a dynamic gradient bidirectional attention module, the detailed structure of which is as follows: Figure 2 As shown in the figure. The blue rectangle on the far right of the figure represents the feature map of a certain scale of the model input, and the far right represents the corresponding output feature map. At the same time, in order to facilitate readers to further understand the calculation process of the module, each more complex calculation module in the figure is marked with the output shape of the module. The following will explain the dynamic gradient bidirectional attention module step by step.
[0012] The first step includes a round of pooling layers, which performs four pooling operations on the input feature map. In the upper half, X maxpooling and X avg pooling perform global maximum and global average pooling on the input feature map in the X direction (width direction), respectively. Similarly, in the lower half, Y max pooling and Y avg pooling perform global maximum and global average pooling on the input feature map in the Y direction (height direction), respectively. The output shapes of the upper and lower half routes are (C, H, 1) and (C, 1, W), respectively.
[0013] The second step is to merge the four feature maps into two feature maps through channel stitching. In the upper half of the route, the two feature maps with shapes (C, H, 1) are spliced and transposed, and the output shape is (2C, 1, H). In the lower half of the route, the two feature maps with shapes (C, 1, W) are only spliced through channels, and the output shape is (2C, 1, W). The third to sixth steps to be described in detail below are the same in the upper and lower half of the route, and the parameters are isolated, so only the upper half of the route is used as an example for explanation.
[0014] The third step includes two one-dimensional convolution operations, where the convolution kernel size is 3, the number of convolution kernels is 2C / r, and the output shape is (2C / r, 1, H).
[0015] The fourth step includes a downsampling module, which is used to reduce the size of the feature map in the width direction. The downsampling module contains several convolution operations (which can be adjusted according to the input image resolution. This experimental process uses two one-dimensional convolution operations), and the output shape is (2C / r, 1, H / k).
[0016] The fifth step includes a dynamic convolution operation with a convolution kernel size of 1 and a number of convolutions of C. The convolution kernel parameters of this step are not obtained through network learning, but are obtained by deforming the feature map generated in the second step.
[0017] The sixth step includes a one-dimensional convolution operation, with a convolution kernel size of 1, a convolution kernel number of 3, and an output shape of (3,1, H / k). In the first half of the route, the output feature map of the sixth step is split on the channel to obtain three vectors of dimension H / k, and these three vectors are defined as distribution weight vectors respectively. , distribution mean bias vector , the distribution variance vector Similarly, the lower half of the route can be a vector with three dimensions W / k , and .
[0018] The seventh step is to generate a Gaussian distribution using the above vector. In the first half of the process, the output H / k one-dimensional Gaussian distributions are defined as ~ , for every one-dimensional Gaussian distribution , its mean Pick , whose variance Pick Similarly, in the second half of the process, the output W / k one-dimensional Gaussian distributions are defined as ~ , for every one-dimensional Gaussian distribution , its mean Pick , whose variance Pick .
[0019] The eighth step is to obtain the attention score matrix based on the distributions and distribution weight vectors obtained in the previous step. In the first half of the route, let The probability density function of , then define For vertical distribution Probability density function. Similarly, in the second half of the route, let The probability density function of , then define For horizontal distribution Probability density function. So far, two one-dimensional distributions have been obtained and The probability density function of , then we can define is the joint distribution density function of the two. Finally, let is the attention score matrix, and The joint density function is The mean and variance in the feature map space are set as The value of row i and column j . So far, the attention score matrix The calculation is complete.
[0020] Step 9: The attention score matrix calculated in the previous step The weighted feature tensor is obtained by bitwise multiplication with the matrix of each channel of the input feature map as the output of the dynamic gradient bidirectional attention module.
[0021] The following is a detailed description of step 3. Step 3 is used to generate the aircraft target location (Boxes) and model (Classes) by decoding the features obtained in step 2. Figure 3 , the decoding process of the decoder is a process of several iterations. For the Tth iteration, the input of the decoder is the encoded image features and the target box (Boxes (T)) at time T, where the shape of the target box is (N, 4), N represents the number of target boxes, and N is a hyperparameter that does not participate in training (N is 20 in the experimental process of this description). Then, the fixed-size feature information of the corresponding position of the target box is extracted from the image features (Featuers) by the RoI Align method to obtain a target feature map with a shape of (N, 49), where N represents the number of targets, and 49 represents that the feature map corresponding to each target box is a 49-dimensional vector. The target feature map is then input into the Sparse R-CNN head to obtain the first round of predicted target boxes, and then the first round of predicted target boxes is input into the next round of RoI Align method, which is cycled 6 times in sequence. Finally, the target box (Boxes (T-1)) and model at time T-1 can be obtained, and one round of iteration ends. In particular, in the first iteration, the input target frame is completely generated by Gaussian noise. When the iteration is greater than or equal to the second time, the input target frame is sampled by the predicted target frame at the previous moment according to the DDIM method.
[0022] The following is a description of the method experiment.
[0023] This experiment uses SAR-AIRcraft-1.0 high-resolution radar images as training data sets. The data set contains 4368 images and 16463 aircraft target instances, covering 7 categories including A220, A320 / 321, A330, ARJ21, Boeing737, Boeing787 and others. The data set contains images of multiple civil airports at different times. These images cover a large area, and the background includes facilities such as terminal buildings, vehicles, and buildings. At the same time, a slice image may contain multiple aircraft targets, which are densely distributed in most cases, and there is mutual interference between targets, which is challenging for model capabilities. The following table compares the experimental results of different models on this data set, where the first column corresponds to the experimental model, and the other columns correspond to the AP0.75 of the model on different models of aircraft. As can be seen from the table, Aircraft Diffusion Det has a greater AP0.75 than other models in the four models, but its performance on A320 / 321, A330 and ARJ21 is lower than Cascade R-CNN and FasterR-CNN. Overall, Aircraft Diffusion Det performs well in the aircraft detection task.
[0024] Model A220 A320 / 321 A330 ARJ21 Boeing 737 Boeing 787 other Faster R-CNN 58.7 87.7 85.0 55.2 42.8 60.5 45.4 Cascade R-CNN 49.1 73.9 87.4 59.0 39.1 57.6 46.1 Reppoints 49.4 84.9 66.4 50.9 36.6 41.8 43.1 SKG-Net 29.8 49.6 66.4 37.7 48.7 51.6 41.1 Aircraft Diffusion Det 62.2 60.1 63.0 54.3 66.7 70.0 59.2
[0025] In summary, the present invention proposes a synthetic aperture radar image aircraft detection method (Aircraft Diffusion Det) using a diffusion model combined with an attention mechanism. The method can improve the detection accuracy of aircraft targets and categories and maintain a high detection efficiency. The present invention applies the method combining the diffusion model and the attention mechanism to the synthetic aperture radar image aircraft detection task for the first time. The Aircraft Diffusion Det detection process includes the following steps: Step 1, extracting multi-scale features of the original image through the encoder, so that the model can adapt to multi-scale targets to detect aircraft of different sizes in actual use. Step 2, through the dynamic gradient bidirectional attention (DGBA) module, calculate the spatial attention weight and obtain the weighted image features, so that the decoder can analyze the target pixel features separately and exclude some background information. Step 3, input the features obtained in step 2 into the detection decoder, and the decoder generates aircraft targets and models through the diffusion method. Through experiments on the SAR-AIRcraft-1.0 dataset, Aircraft Diffusion Det achieved an AP value of 62.164, an AP0.5 value of 94.87, and an AP0.75 value of 68.62. Experimental results show that Aircraft DiffusionDet can improve detection accuracy without reducing detection efficiency, which is in line with the expected effect.
Claims
1. A method for aircraft detection in synthetic aperture radar images based on computer vision, characterized by: The following steps are involved: Step 1: Extract multi-scale features of the original image through the encoder; Step 2: Calculate the spatial attention weights and obtain weighted image features through the dynamic gradient bidirectional attention (DGBA) module; Step 3: Input the features obtained in step 2 into the detection decoder, which generates aircraft targets and models through the diffusion method.
2. The method for detecting aircraft in synthetic aperture radar images based on computer vision according to claim 1, characterized in that: The step 1 includes an encoder for extracting image features and an FPN for fusing multi-scale information. The encoder can use any structure (such as ResNet, Swin Transformer, etc.) that can receive image data and output feature maps of at least 4 scales (1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the original image width and height).
3. The method for detecting aircraft in synthetic aperture radar images based on computer vision according to claim 1, characterized in that: The step 2 includes a dynamic gradient bidirectional attention module, which includes the following 9 steps. The first step includes a round of pooling layer, which performs four pooling operations on the input feature map. In the upper half route, X max pooling and X avg pooling perform global maximum and global average pooling on the input feature map in the X direction (width direction), respectively. Similarly, in the lower half route, Y max pooling and Y avg pooling perform global maximum and global average pooling on the input feature map in the Y direction (height direction), respectively. The output shapes of the upper half route and the lower half route are (C, H, 1) and (C, 1, W), respectively. The second step is used to merge the four feature maps into two feature maps through channel splicing. In the upper half route, channel splicing and transposition operations are performed on the two feature maps with shapes of (C, H, 1), and the output shape is (2C, 1, H). In the lower route, only channel concatenation is performed on the two feature maps of shape (C, 1, W), and the output shape is (2C, 1, W). The third to sixth steps detailed below are the same in the upper and lower routes, and the parameters are isolated, so only the upper route is used as an example for explanation. The third step includes two one-dimensional convolution operations, where the convolution kernel size is 3, the number of convolution kernels is 2C / r, and the output shape is (2C / r, 1, H). The fourth step includes a downsampling module, which is used to reduce the size of the feature map in the width direction. The downsampling module includes several convolution operations (which can be adjusted according to the input image resolution), and the output shape is (2C / r, 1, H / k). The fifth step includes a dynamic convolution operation, where the convolution kernel size is 1 and the number of convolutions is C. The convolution kernel parameters of this step are not obtained through network learning, but are obtained by deforming the feature map generated in the second step. The sixth step includes a one-dimensional convolution operation, where the convolution kernel size is 1, the number of convolution kernels is 3, and the output shape is (3, 1, H / k). In the upper half of the route, the feature map output in the sixth step is split on the channel to obtain three vectors of dimension H / k, and these three vectors are defined as distribution weight vectors respectively. , distribution mean bias vector , the distribution variance vector Similarly, the lower half of the route can be a vector with three dimensions W / k , and The seventh step is to generate a Gaussian distribution using the above vector. In the first half of the process, the output H / k one-dimensional Gaussian distributions are defined as ~ , for every one-dimensional Gaussian distribution , its mean Pick , whose variance Pick Similarly, in the second half of the process, the output W / k one-dimensional Gaussian distributions are defined as ~ , for every one-dimensional Gaussian distribution , its mean Pick , whose variance Pick The eighth step is to obtain the attention score matrix based on the distributions and distribution weight vectors obtained in the previous step. In the first half of the route, let The probability density function of , then define For vertical distribution Probability density function. Similarly, in the second half of the route, let The probability density function of , then define For horizontal distribution Probability density function. So far, two one-dimensional distributions have been obtained and The probability density function of is the joint distribution density function of the two. Finally, let is the attention score matrix, and The joint density function is The mean and variance in the feature map space are set as The value of row i and column j . So far, the attention score matrix The calculation is completed. Step 9: Add the attention score matrix calculated in the previous step The output of the dynamic gradient bidirectional attention module is obtained by bitwise multiplication with the matrix of each channel of the input feature map.
4. The method for detecting aircraft in synthetic aperture radar images based on computer vision according to claim 1, characterized in that: The decoding process of the decoder in step 3 is a process of several iterations. For the Tth iteration, the input of the decoder is the encoded image features and the target box (Boxes (T)) at time T, where the shape of the target box is (N, 4), N represents the number of target boxes, and N is a hyperparameter that does not participate in training. Then, the fixed-size feature information of the corresponding position of the target box is extracted from the image features (Featuers) by the RoI Align method to obtain a target feature map with a shape of (N, 49), where N represents the number of targets, and 49 represents that the feature map corresponding to each target box is a 49-dimensional vector, and then the target feature map is input into the Sparse R-CNN head to obtain the first round of predicted target boxes, and then the first round of predicted target boxes are input into the next round of RoI Align method, and the cycle is repeated 6 times in sequence, and finally the target box (Boxes (T-1)) and model at time T-1 can be obtained, and at this point, one round of iteration ends. In particular, in the first iteration, the input target frame is completely generated by Gaussian noise. When the iteration is greater than or equal to the second time, the input target frame is sampled by the predicted target frame at the previous moment according to the DDIM method.
