A SAR Aircraft Target Detection and Recognition Algorithm Based on Salient Features

Abstract

The invention discloses a SAR aircraft target detection and recognition algorithm based on significant features, characterized in that it comprises the following steps: pre-processing a SAR image, obtaining a mask image according to the location of the target as the input of a subsequent detection model; training and fine-tuning the target detection model, optimizing the feature extractor of the model by generating a network module, and then mining pixel information in the data to learn the different features of the aircraft target and the clutter background in the SAR image; training and fine-tuning the mask image and the target label in step (1) as the input of the aircraft target detection and recognition algorithm model, and obtaining the detection result of the target. The invention can focus on learning foreground targets, and can better learn the difference between foreground and background clutter; only a small amount of new class training samples are needed to achieve good accuracy; the model's ability to distinguish between targets and backgrounds is enhanced, the target detection accuracy is improved, and false alarms and missed detections are reduced.

Description

Technical Field

[0001] The invention relates to the field of radar image processing and target detection, and in particular to a SAR aircraft target detection and recognition algorithm based on significant features and applicable to a small number of samples. Background Art

[0002] Synthetic Aperture Radar (SAR) has the advantage of working around the clock and in all weather conditions, so it is widely used in both civil and military fields. Aircraft targets are important military targets and means of transportation. High-precision detection and identification of aircraft targets helps to obtain military intelligence such as combat effectiveness in real time, which is of great significance in combat decision-making and civil aviation scheduling.

[0003] SAR images have great potential in target detection and recognition tasks. SAR images contain rich information, such as the amplitude and phase of the echo signal, which can provide richer and more accurate target features. The emergence of deep learning technology has greatly improved the accuracy and robustness of SAR target detection. However, the annotation of SAR image datasets requires a lot of manual labor. The target detection method based on few samples has been successfully applied to many fields because it can detect targets with less data and achieve high accuracy.

[0004] Few-shot object detection refers to object detection in the target detection task when there are only a small number of labeled samples available for training. The core idea is to use a limited number of labeled samples for training to classify and locate objects in the image. By designing effective training strategies, model architectures, and loss functions, a detection model with a certain degree of generalization ability is obtained to achieve effective detection in complex environments. Inspired by human learning patterns, the concept of few-shot learning was first proposed in 2003 to solve the problem of model overfitting caused by a small number of training samples. In 2020, a two-stage fine-tuning approach (TFA) was proposed. When fine-tuning the few-shot class, TFA only fine-tunes the last layer of the detection model, and the parameters of the remaining layers are all frozen, which effectively improves the model's migration ability. Then, a low-shot transfer detector (LSTD) was proposed. LSTD designed two mechanisms: background suppression regularization (BD) and knowledge transfer regularization (TK). However, there are problems such as small correlation within the same category and small differences between different categories, resulting in low target detection accuracy. Summary of the invention

[0005] Purpose of the invention: The purpose of the present invention is to provide a SAR aircraft target detection and recognition algorithm based on significant features suitable for small sample conditions.

[0006] Technical solution: The present invention comprises the following steps:

[0007] (1) Preprocess the SAR image and obtain a mask map based on the target location, which serves as the input of the subsequent few-shot SAR aircraft target detection model;

[0008] (2) Train and fine-tune the salient feature-based few-shot SAR aircraft target detection model, optimize the model’s feature extractor by generating network modules, and then mine pixel information in the data to learn the different features of aircraft targets and clutter background in SAR images;

[0009] (3) The mask image and target label in step (1) are used as inputs of a SAR aircraft target detection and recognition algorithm model based on salient features and suitable for few-sample conditions to train and fine-tune the model to obtain the target detection result.

[0010] Furthermore, the step of extracting the mask image in step (1) includes:

[0011] (1.1) Convert the input image into a (0, 1) tensor, denoted as Image_Tensor;

[0012] (1.2) Create an all-zero tensor, denoted as modified_image;

[0013] (1.3) For each pair of images denoted as Image_Tensor and annotations denoted as Target, traverse each bounding box in the annotation information, convert the bounding box coordinates into integers in a loop, extract the coordinates of the bounding box, and finally copy the pixel values ​​of the corresponding area in the original image to the modified image.

[0014] Furthermore, the step (2) comprises:

[0015] (2.1) In the basic training stage, the Faster RCNN loss and generative network module loss training strategies are used to alternately train and optimize the target detection model.

[0016] (2.2) In the few-shot fine-tuning stage, randomly initialized weights are assigned to the predictors of new classes, the fully connected layer-based classifier is replaced with a cosine similarity-based classifier, and the parameters of the feature extraction module, RPN module, generative network module, and RoIPooling module are frozen.

[0017] Furthermore, the Faster RCNN loss in step (2.1) adopts the standard joint loss function in Faster RCNN and the proposed generative network module loss function, expressed as:

[0018] Loss = L RPN +L RCNN

[0019] Among them, the first item is the RPN loss and the second item is the RCNN loss.

[0020] Furthermore, the RPN loss includes classifier loss and regression loss, and the RPN loss calculation formula is:

[0021]

[0022] Among them, p i represents the probability that the i-th anchor point is predicted to be the true label, When it is a positive sample, it is 1, and when it is a negative sample, it is 0. i represents the bounding box regression parameters for predicting the i-th anchor point, represents the true label bounding box corresponding to the i-th anchor point, N cls Represents the number of mini-batch training samples, N reg Indicates the number of anchor positions.

[0023] The classifier loss uses binary cross entropy loss, namely:

[0024]

[0025] Among them, y i is the true label of sample i, is the predicted probability of the sample.

[0026] The regression loss is calculated using the smooth L1 loss function, expressed as:

[0027]

[0028] The smooth L1 function is defined as follows:

[0029]

[0030] Furthermore, the RCNN loss includes classifier loss and regression loss, where the classifier loss uses the softmax multi-classification cross entropy loss, and the regression loss is still the smooth L1 loss, that is:

[0031] L(p,u,t u , v)=L cls (p,u)+λ[u≥1]L reg (tu , v)

[0032] Where p is the softmax probability distribution predicted by the classifier, u is the encoding of the corresponding target true category label, u = (u0, ..., u k ), [u≥1] is the Iverson bracket, and the multi-classification cross entropy loss is expressed as:

[0033]

[0034] Furthermore, the step (2.1) of performing alternating training optimization on the target detection model includes:

[0035] The mask image obtained in step (1) is used as the supervised label. The feature extraction network inputs the obtained feature image into the generation network module to obtain the fake image, and the MSE loss is calculated by the two to optimize the generation network module and the feature extractor module. The loss function is:

[0036] L g =MSELoss(fake, label)

[0037] Among them, fake is the image generated by the network module, label is the label image obtained in step (1),

[0038]

[0039] is the generated image, Y is the real image, n is the number of pixels, is the predicted value of the i-th pixel, y i is the true value of the i-th pixel.

[0040] Furthermore, the classifier based on cosine similarity in step (2.2) is expressed as:

[0041]

[0042]

[0043] in, represents the predicted probability that the i-th RoI feature belongs to class j, w j represents the weight vector of the jth class, α is the scale factor, and F(x) is the input feature.

[0044] Furthermore, the step (3) comprises:

[0045] (3.1) Perform basic training on all images in the dataset that do not contain any new class objects, and randomly select a very small training set containing the base class and the new class for fine-tuning;

[0046] (3.2) The evaluation index used is used to evaluate the few-shot object detection algorithm and calculate the mAP of new classes with different K values, namely K-shot mAP, which is defined as follows:

[0047]

[0048] Where n is the number of new classes, AP K is the average precision of the K-shot model, and the calculation formula is as follows:

[0049]

[0050] Among them, r is the recall rate, P K is the accuracy of the K-shot model.

[0051] Furthermore, the evaluation indicators used in step (3.2) include precision and recall, where precision refers to the probability of correct detection among all detected targets, and recall refers to the probability of correct identification among all positive samples;

[0052]

[0053]

[0054] Among them, True Positives (TP) represents the number of samples correctly classified as positive examples; False Positives (FP) represents the number of samples incorrectly classified as positive examples; False Negatives (FN) represents the number of samples incorrectly classified as negative examples; True Negatives (TN) represents the number of samples correctly classified as negative examples.

[0055] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: a generative network module is used to optimize the feature extraction network, so that it can focus on learning foreground targets and better learn the differences between foreground and background clutter; a complete small sample transfer learning framework is constructed, and only a small number of new class training samples are needed to achieve good accuracy; the fully connected classifier is replaced by a cosine similarity-based classifier to enhance the correlation within the same category and the difference between different categories, thereby enhancing the model's ability to distinguish between targets and backgrounds, improving target detection accuracy, and reducing false alarms and missed detections. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a flow chart of the present invention;

[0057] Figure 2 It is a model framework diagram of the present invention;

[0058] Figure 3A step-by-step diagram of the image resolution generated by the generative network model.

[0059] Figure 4 This is the transition diagram from 16×16 images to 32×32 images using the generative network model;

[0060] Figure 5 Result diagram of target detection in SAR aircraft image. DETAILED DESCRIPTION

[0061] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0062] like Figure 1 As shown, the present invention comprises the following steps:

[0063] (1) Preprocess the SAR image and obtain a mask map based on the target location, which serves as the input of the subsequent few-shot SAR aircraft target detection model;

[0064] Based on the SAR aircraft amplitude image, the target mask is extracted according to the dataset annotation information as the supervision information of the generative network module in the model:

[0065]

[0066]

[0067] (2) Train and fine-tune the salient feature-based few-shot SAR aircraft target detection model, optimize the model’s feature extractor by generating network modules, and then mine pixel information in the data to learn the different features of aircraft targets and clutter background in SAR images;

[0068] The proposed model of SAR aircraft target detection and recognition algorithm based on salient features suitable for few-sample conditions is as follows Figure 2 shown.

[0069] (2.1) In the basic training stage, training is performed on the base class dataset to establish prior knowledge. The training strategy uses the Faster RCNN loss and the generative network module loss to alternately train and optimize. In this stage, the standard joint loss function in Faster RCNN and the proposed generative network module loss function are used to train all parameters of the base detector on the base class dataset, where the standard joint loss function in Faster RCNN is:

[0070] Loss = L RPN +L RCNN

[0071] Among them, the first term is RPN loss, and the second term is RCNN loss. The RPN loss includes classifier loss and regression loss, and the RPN loss calculation formula is:

[0072]

[0073] Among them, p i represents the probability that the i-th anchor point is predicted to be the true label, When it is a positive sample, it is 1, and when it is a negative sample, it is 0. i represents the bounding box regression parameters for predicting the i-th anchor point, represents the true label bounding box corresponding to the i-th anchor point, N cls Represents the number of mini-batch training samples, N reg Indicates the number of anchor positions.

[0074] Each anchor point in RPN only distinguishes between foreground and background, and the classifier loss uses binary cross entropy loss, namely:

[0075]

[0076] Among them, y i is the true label of sample i, is the predicted probability of the sample. The regression loss is calculated using the smooth L1 loss function, expressed as:

[0077]

[0078] The smooth L1 function is defined as follows:

[0079]

[0080] RCNN loss consists of two parts, classifier loss and box regression loss. The classifier loss uses softmax multi-classification cross entropy loss, and the regression loss is still smooth L1 loss, that is:

[0081] L(p,u,t u , v)=L cls (p,u)+λ[u≥1]L reg (t u , v)

[0082] p is the classifier prediction softmax probability distribution p = (p0, ..., p k ), u corresponds to the encoding of the target real category label u = (u0, ..., u k ), [u≥1] is the Iverson bracket, and the multi-classification cross entropy loss is defined as follows

[0083]

[0084] like Figure 3 As shown in , the generative network module starts with a low-resolution image and then gradually increases the resolution by adding layers to the network. The resolution of the generated image is gradually increased from 16×16 pixels to 1024×1024 pixels. Figure 4 The figure shows the transition from a 16×16 pixel image to a 32×32 pixel image. During the transition, the higher resolution layer is treated as a residual block, and its weight α increases linearly from 0 to 1. Here, 2× means that the image resolution is doubled using the nearest neighbor filter, and toRGB means that the feature vector is projected to the RGB color layer. This incremental nature allows training to first discover the large-scale structure of the image distribution, and then shift attention to increasingly fine scale details without having to learn all scales at the same time. When adding new layers to the network, they are added smoothly, which avoids sudden impacts on the already trained small resolution layers.

[0085] During training, the mask image obtained in step (1) is used as the supervised label. The feature extraction network inputs the obtained feature image into the generation network module to obtain a fake image, and the MSE loss is calculated by the two to optimize the generation network module and the feature extractor module. The loss function is:

[0086] L g =MSELoss(fake, label)

[0087] fake is the image generated by the network module, label is the label image obtained in step S1, where:

[0088]

[0089] is the generated image, Y is the real image, n is the number of pixels, is the predicted value of the i-th pixel, y i is the true value of the i-th pixel.

[0090] (2.2) In the few-shot fine-tuning stage, randomly initialized weights are first assigned to the predictors of the new classes, which enables the predictors trained in the fine-tuning stage to predict the new classes. In addition, the classifier based on the fully connected layer is replaced by a classifier based on cosine similarity:

[0091]

[0092]

[0093] in, represents the predicted probability that the i-th RoI feature belongs to class j, s i，j represents the cosine similarity of the i-th candidate box corresponding to the j-th category, wi represents the weight vector of the jth class, α is the scale factor, and F(x) is the input feature.

[0094] During the few-shot fine-tuning training, the parameters of the feature extraction module, RPN module, generation network module, and RoIPooling module are frozen, that is, the parameters of these parts remain unchanged, and the parameters of the rest of the model are updated. The model obtained in the basic category training phase has achieved good performance, enabling it to focus on the foreground target during the transfer learning process and better distinguish the foreground from the background, thereby improving the detection accuracy of the few-shot category.

[0095] (3) The mask image and target label in step (1) are used as inputs of a SAR aircraft target detection and recognition algorithm model based on salient features and suitable for few-sample conditions to train and fine-tune the model to obtain the target detection result.

[0096] (3.1) We perform base training on all images that do not contain any new class objects on the dataset, and then randomly select a very small training set containing base classes and new classes for fine-tuning, where each class has only K annotated images, and K is 5, 10, 30. We test our method on all images that contain at least one new class object.

[0097] (3.2) The evaluation indicators used are as follows.

[0098] True Positives (TP): the number of samples correctly classified as positive examples;

[0099] False Positives (FP): The number of samples that are mistakenly classified as positive examples;

[0100] False Negatives (FN): The number of samples that are mistakenly classified as negative examples;

[0101] True Negatives (TN): The number of samples correctly classified as negative examples;

[0102]

[0103]

[0104] Precision refers to the probability of detecting the correct target among all detected targets, and recall refers to the probability of correctly identifying all positive samples. In addition, in order to better evaluate the few-shot target detection algorithm, mAP of new classes with different K values ​​is calculated and called K-shot mAP. Similar to the traditional mAP, the formula of K-shot mAP is defined as follows:

[0105]

[0106] Where n is the number of new classes, AP K is the average precision of the K-shot model, and the calculation formula is as follows:

[0107]

[0108] Among them, r is the recall rate, P K is the accuracy of the K-shot model.

[0109] The following is a verification of the SAR aircraft target detection and recognition algorithm based on significant features and applicable to few-sample conditions through actual data, taking the aircraft images of the SAR-AIRcraft-1.0 dataset as an example, as shown in Table 1.

[0110] Table 1: Specific parameters of each network layer in the generative network model used in the present invention

[0111]

[0112] like Figure 5 The figure shows the experimental results of target detection of SAR aircraft images using the few-sample target detection algorithm proposed in the present invention. The few-sample target detection network proposed in the present invention successfully detects targets using only 10 images for each new category, achieving a high accuracy rate and a low false alarm rate for target detection.

Claims

1. A SAR aircraft target detection and recognition algorithm based on salient features, characterized in that: The steps include: (1) Preprocess the SAR image and obtain a mask map based on the target location, which serves as the input of the subsequent few-shot SAR aircraft target detection model; (2) Train and fine-tune the salient feature-based few-shot SAR aircraft target detection model, optimize the model’s feature extractor by generating network modules, and then mine pixel information in the data to learn the different features of aircraft targets and clutter background in SAR images; (3) The mask image in step (1) is used as the input of a SAR aircraft target detection and recognition algorithm model based on salient features and suitable for few-sample conditions to train and fine-tune the target detection result. The step (1) of extracting the mask image comprises: (1.1) Convert the input image into a (0, 1) tensor, denoted as Image_Tensor; (1.2) Create an all-zero tensor, denoted as modified_image; (1.3) The image pair includes the original image and the annotation. For each pair of images, record them as Image_Tensor and the annotations as Target. Traverse each bounding box in the annotation information, convert the bounding box coordinates into integers in a loop, extract the coordinates of the bounding box, and finally copy the pixel values ​​of the corresponding area in the original image to modified_image. The step (2) comprises: (2.1) In the basic training stage, the Faster RCNN loss and the generative network module loss training strategy are used to alternately train and optimize the target detection model; (2.2) In the few-shot fine-tuning phase, randomly initialized weights are assigned to the predictors of the new classes, the fully connected layer-based classifier is replaced with a cosine similarity-based classifier, and the parameters of the feature extraction module, RPN module, generative network module, and RoIPooling module are frozen. The Faster RCNN loss in step (2.1) adopts the standard joint loss function in Faster RCNN and the proposed generative network module loss function, which is expressed as: Loss=L RPN +L RCNN Among them, the first item is RPN loss, and the second item is RCNN loss. The RPN loss includes classifier loss and regression loss. The RPN loss calculation formula is: Among them, p i represents the probability that the i-th anchor point is predicted to be the true label, When it is a positive sample, it is 1, and when it is a negative sample, it is 0. i represents the bounding box regression parameters for predicting the i-th anchor point, represents the true label bounding box corresponding to the i-th anchor point, N cls Represents the number of mini-batch training samples, N reg Indicates the number of anchor positions, The classifier loss uses binary cross entropy loss, namely: Among them, y i is the true label of sample i, is the predicted probability of the sample, The regression loss is calculated using the smooth L1 loss function, expressed as: The smooth L1 function is defined as follows: The RCNN loss includes classifier loss and regression loss, where the classifier loss uses the soffmax multi-classification cross entropy loss, and the regression loss is still the smooth L1 loss, that is: L(p,u,t u ,v)=L cls (p,u)+λ[u≥1]L reg (t u ,v) Where p is the softmax probability distribution predicted by the classifier, u is the encoding of the corresponding target true category label, u = (u0, ..., u k ), [u≥1] is the Iverson bracket, and the multi-classification cross entropy loss is expressed as: The step (2.1) of performing alternating training optimization on the target detection model includes: The mask image obtained in step (1) is used as the supervised label. The feature extraction network inputs the obtained feature image into the generation network module to obtain the fake image, and the MSE loss is calculated by the two to optimize the generation network module and the feature extractor module. The loss function is: L g =MSELoss(fake,label) Among them, fake is the image generated by the network module, label is the label image obtained in step (1), is the generated image, Y is the real image, n is the number of pixels, is the predicted value of the i-th pixel, y i is the true value of the ith pixel, The classifier of step (2.2) based on cosine similarity is expressed as: Among them, p i，j represents the predicted probability that the i-th RoI feature belongs to class j, s i，j represents the cosine similarity of the i-th candidate box corresponding to the j-th category, w j represents the weight vector of the jth class, α is the scale factor, and F(x) is the input feature.

2. The SAR aircraft target detection and recognition algorithm based on significant features according to claim 1 is characterized in that: The step (3) comprises: (3.1) Perform basic training on all images in the dataset that do not contain any new class objects, and randomly select a very small training set containing the base class and the new class for fine-tuning; (3.2) The evaluation metric used is used to evaluate the few-shot object detection algorithm and calculate the mAP for new classes with different values.

3. The SAR aircraft target detection and recognition algorithm based on significant features according to claim 2 is characterized in that: The evaluation indicators used in the step (3.2) include precision and recall, where precision refers to the probability of correctly detecting all detected targets, and recall refers to the probability of correctly identifying all positive samples; Among them, TP represents the number of samples correctly classified as positive examples; FP represents the number of samples incorrectly classified as positive examples; FN represents the number of samples incorrectly classified as negative examples; and TN represents the number of samples correctly classified as negative examples.

Images