A small sample SAR image target recognition method based on multi-task representation learning

Abstract

The present invention discloses a small-sample SAR image target recognition method based on multi-task representation learning, which is applied to the field of SAR image interpretation to solve the problem of SAR image target feature extraction under small sample conditions. First, the present invention constructs a feature extraction model with residual learning as the basic architecture, adopts deformation convolution operation to realize target morphological feature extraction, and completes feature channel screening based on the attention mechanism; then, a multi-task learning method is adopted to improve the representation ability of the feature extraction model; finally, a feature extraction model with multi-task representation learning ability is adopted to extract small-sample SAR image target features to fit a logistic regression classifier, thereby realizing target category inference. The present invention can improve the small-sample SAR image target recognition performance under different degrees of sample scarcity conditions and different pitch angle conditions, and has good generalization ability and accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of Synthetic Aperture Radar (SAR) image interpretation, and in particular relates to a small-sample SAR image target recognition method based on multi-task representation learning. Background Art

[0002] Synthetic aperture radar (SAR) can acquire high-resolution images of target scenes around the clock and in all weather conditions, and has been widely used in both military and civilian fields. Extracting effective intelligence from image data is crucial for SAR image interpretation. Automatic target recognition (ATR), a key tool for SAR image interpretation, has been a hot research topic in the SAR field. In recent years, with the advancement of deep learning technology, deep learning-based recognition methods have achieved remarkable results in addressing target recognition in SAR images. However, data-driven deep learning methods require a large number of training samples to provide prior knowledge for target identification, and are prone to overfitting when limited sample sizes are available. In practical SAR applications, particularly military scenarios, target samples are a common occurrence. Most existing SAR image target recognition methods perform poorly under these conditions. Therefore, research on small-sample SAR automatic target recognition methods is of vital practical value.

[0003] Few-shot learning aims to learn a model that can achieve robust target recognition under limited sample conditions. Existing methods for solving target recognition in few-shot SAR images can be roughly divided into two categories: meta-learning and transfer learning. Among them, meta-learning trains the model to quickly learn knowledge by simulating few-shot tasks during the training phase, while transfer learning uses the knowledge learned from the source task to assist in completing the target task. The common point of existing few-shot SAR image target recognition methods based on meta-learning and transfer learning is that they learn a powerful feature extraction model to achieve few-shot target recognition. Therefore, training an effective feature extraction model is very important for improving the performance of few-shot recognition. However, most existing few-shot SAR image target recognition methods based on deep learning only learn feature representation models from a single task, which limits their representation capabilities and greatly affects the performance of few-shot SAR image target recognition.

[0004] In order to achieve robust SAR image target recognition with limited training samples, this paper proposes a small-sample SAR image target recognition method based on multi-task representation learning. Summary of the Invention

[0005] To address the challenges of SAR image target recognition under sample-scarce conditions, this paper proposes a small-sample SAR image target recognition method based on multi-task representation learning. This method integrates three concepts: target equivariant feature learning, invariant feature learning, and target class boundary learning to establish a target recognition model. A feature extractor with robust representational capabilities is trained using multi-task learning. Next, based on the feature extractor obtained during the training phase, small-sample target features are extracted. A logistic regression classifier is then fitted to these small-sample target features to achieve target recognition.

[0006] The technical approach adopted by the present invention to achieve small-sample SAR image target recognition is as follows: in the learning phase, an effective feature representation model is learned through auxiliary class samples with a sufficient number of samples; in the inference phase, small-sample target features are extracted based on the feature representation model obtained in the training phase, and small-sample target recognition is achieved by fitting a logistic regression classifier. Specifically, to achieve small-sample SAR image target recognition, a feature representation model based on a residual structure is first established. The target morphological features are effectively perceived through deformation convolution operations, and an attention mechanism is used to adaptively refine target features at the feature channel level to enhance useful information and suppress redundant information; then, multi-feature learning ideas such as equivariant feature learning, invariant feature learning, and class boundary learning are adopted to encourage the representation model to learn rich target identification information; finally, small-sample target features are extracted based on the feature representation model, and small-sample SAR image target recognition is achieved by fitting a logistic regression classifier.

[0007] The technical solution of the present invention is a small sample SAR image target recognition method based on multi-task representation learning, the method steps are as follows:

[0008] Step 1: Collect sufficient number of auxiliary SAR image samples D b ={(x,y)}, where It is a SAR image with a channel number of 1 and a pixel size of h×w. Indicates the category label of the SAR image, there are N b categories; at the same time, a small sample SAR target sample set D is collected f ={(x,y)}, where represents the category label of the small sample SAR image, and N b auxiliary classes and N f The target classes of the small samples are completely disjoint;

[0009] Step 1.1: Crop all SAR images into target areas of interest with uniform pixel size;

[0010] Step 1.2: Perform multiple geometric transformations on the auxiliary SAR image samples and perform transformation annotation on the transformed SAR image samples;

[0011] Step 2: Design a multi-task representation learning network and use the auxiliary SAR image samples in step 1 for network training;

[0012] The multi-task representation learning network includes: feature extractor f B , equivariant feature learning branch f E , invariant feature learning branch f I , class boundary learning branch f C , the feature extractor f B The input is SAR image, and the equivariant feature learning branch f E , invariant feature learning branch f I , class boundary learning branch f C The input of the feature extractor f B Output;

[0013] Step 2.1: Build feature extractor f B ;

[0014] Use deformable convolution operation to give feature extractor f B The ability to perceive the morphological features of the target and adaptively enhance and suppress features at the feature channel level through the attention mechanism to obtain feature maps with high information content;

[0015] Step 2.2: Construct equivariant feature learning branch f E ;

[0016] According to the transformation annotation, a supervised approach is adopted to enable the feature extractor to have the ability to perceive changes in the target sample. The loss function of the equivariant feature learning branch adopts the cross entropy loss function, which is expressed as follows:

[0017]

[0018] Among them, N represents the number of SAR image samples input to the network, N t Indicates the number of conversion categories of SAR images, y ij is the manually assigned conversion category label, P ij represents the predicted probability that sample i belongs to category j;

[0019] Step 2.3: Construct the invariant feature learning branch f I ;

[0020] In order to enable the feature extractor to have the ability to extract target invariant features, a contrast loss function is used to make the similarity of the same SAR image samples with different conversion categories higher. Its expression is as follows:

[0021]

[0022]

[0023] Where s(·) represents the similarity function, D n represents the negative sample feature set, τ represents the adjustable temperature coefficient, M represents the total number of conversion categories, v 0 Represents the original image features without any transformation, v m represents the SAR image features after the mth transformation, v 0 Indicates the stored v 0 Image features;

[0024] Step 2.4: Construct the class boundary learning branch f C ;

[0025] According to the category label y, a supervised approach is used to aggregate SAR image samples of the same category in the feature embedding space. The loss function of the class boundary learning branch also uses the cross entropy loss function, which is expressed as follows:

[0026]

[0027] Among them, N b Indicates the number of categories of SAR images, represents the category label; p ij represents the predicted probability that sample i belongs to category j;

[0028] Step 2.5: Use the triple loss function to jointly train the multi-task representation learning network. The total loss function is defined as follows:

[0029] L=L eq +L in +L ba

[0030] Step 3: Use the trained feature extractor f B Extract target features from small sample SAR images;

[0031] Step 4: Use the features extracted in step 3 to fit a logistic regression classifier to complete the small sample target recognition task.

[0032] In summary, the beneficial effects achieved by the present invention are:

[0033] The small-sample SAR image target recognition method proposed in this paper based on multi-task representation learning perceives the morphological features of the target by embedding a deformable convolutional layer in the feature extractor, and uses an attention mechanism to adaptively screen feature channels with rich target information. At the same time, a multi-task learning method of target equivariant feature learning, invariant feature learning, and target class boundary learning is adopted to improve the representation ability of the feature extractor, so as to achieve robust SAR image target recognition under small-sample conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a network structure diagram of the present invention;

[0035] Figure 2 Experimental results for effectiveness analysis of different learning tasks. DETAILED DESCRIPTION

[0036] In order to better demonstrate the technical points of the present invention, the invention content will be described in detail below. The present invention is a small sample SAR image target recognition method based on multi-task representation learning. The network structure of the method is as follows Figure 1 The specific implementation process of the present invention is as follows:

[0037] Step 1: Collect a sufficient number of auxiliary SAR image samples D b ={(x,y)}, where It is a SAR image with a height of h, a width of w, and a channel number of 1. Indicates that it belongs to N b SAR image category labels of categories; at the same time, a small sample SAR target sample set D is collected f ={(x,y)}, where represents the category label of the small sample SAR image, in particular, N b auxiliary classes and N f The target classes of the small samples are completely disjoint;

[0038] Step 1.1: To reduce the redundant background of the target and facilitate network training, all SAR image samples are uniformly cropped into a target region of interest of 64×64 pixels;

[0039] Step 1.2: To enable the feature extractor to perceive equivariant properties, the auxiliary SAR image samples are subjected to multiple geometric transformations and the transformed SAR image samples are labeled. This embodiment performs a total of 16 geometric transformations, including a combination of multiple rotations and scale transformations.

[0040] Step 2: Design a multi-task representation learning network based on the residual architecture and use the auxiliary SAR image samples in step 1 for network training;

[0041] Step 2.1: Build feature extractor f B . Adopting deformable convolution operation to improve the feature extractor f B The ability to perceive the morphological features of the target and perform adaptive feature channel selection at the channel level through the attention mechanism, thereby enhancing useful information and reducing redundant information, thereby obtaining feature maps with rich target information. To prevent network overfitting, the DropBlock method is used to improve the generalization ability of the convolutional layer;

[0042] Step 2.2: Construct an equivariant feature learning branch f based on a three-layer perceptron model E A supervised approach is used to enable the feature extractor to acquire the ability to perceive changes in target samples. The loss function of the equivariant feature learning branch adopts the cross entropy loss function, which is expressed as:

[0043]

[0044] Among them, N represents the number of SAR image samples input to the network, N t Indicates the number of conversion categories of SAR images, y ij is the manually assigned conversion category label, P ij represents the predicted probability that sample i belongs to category j;

[0045] Step 2.3: Construct an invariant feature learning branch f based on a three-layer structured perceptron model I In order to enable the feature extractor to have the ability to extract target invariant features, a contrast loss function is used to make the similarity of the same SAR image samples with different conversion categories higher, and its expression is as follows:

[0046]

[0047]

[0048] Where s(·) represents the similarity function, D n represents the negative sample feature set, τ represents the temperature coefficient, which is generally set to 1, M represents the total number of conversion categories, and v 0 Represents the original image features without any transformation, v m represents the SAR image features after the mth transformation, v 0 Indicates the stored v 0 Image features;

[0049] Step 2.4: Construct a class boundary learning branch f based on a one-layer perceptron model CA supervised approach is used to aggregate SAR image samples of the same category in the feature embedding space. The loss function of the class boundary learning branch also uses the cross entropy loss function, which is expressed as:

[0050]

[0051] Among them, N b Indicates the number of categories of SAR images, represents the category label; p ij Represents the predicted probability that sample i belongs to category j; Step 2.5: Use the triple loss function to jointly train the multi-task representation learning network. The total loss function is defined as follows:

[0052] L=L eq +L in +L ba

[0053] Step 3: Extend the target perception capability acquired by the feature extractor in the training phase to the inference phase, and use the feature extractor learned in the above steps to extract target features from small-sample SAR images.

[0054] Step 4: Fit a logistic regression classifier based on the small sample target features learned by the feature extractor to complete the small sample SAR image target recognition task.

[0055] 1. Simulation conditions:

[0056] This simulation experiment uses the public MSTAR dataset, which contains SAR images of 10 types of ground military targets collected at different pitch angles and azimuth angles. This experiment uses SAR image samples collected at 15°, 17°, and 30° pitch angles. The specific target types and sample numbers are shown in Table 1. At the same time, a batch of data is randomly sampled from the test set and divided into two datasets: labeled dataset D and labeled dataset D. su And the unlabeled dataset D qu To simulate a small sample size target recognition task, the data set is shown in Table 2. To obtain statistically significant experimental results, all simulation experiments in this paper were repeated 1000 times. This simulation experiment was conducted on a Windows 10 64-bit computer system with 16GB of memory and an NVIDIA GeForce RTX 2060 GPU. PyCharm simulation software was used and implemented in Python based on the PyTorch framework.

[0057] 2. Simulation content and results:

[0058] Simulation 1: Evaluate recognition performance under different degrees of sample scarcity.

[0059] This simulation experiment evaluated the small-shot recognition performance under varying degrees of sample scarcity using the MSTAR dataset. A classic prototype network, ProtoNet, was used as a comparison method. Seven SAR image samples at a 17° elevation angle were selected as the training set, and three SAR image samples at a 17° elevation angle, 2S1, BRDM2, and ZSU23 / 4, were used as the test set. Four small-shot recognition tasks, namely 3-way 1-shot, 3-way 3-shot, 3-way 5-shot, and 3-way 10-shot, were simulated based on the test dataset. The recognition results under varying degrees of sample scarcity are shown in Table 3. As can be seen, under varying degrees of sample scarcity, the proposed method achieved recognition performance improvements of up to 11.69% over ProtoNet. The proposed method achieved over 80% accuracy for the 3-way 1-shot small-shot recognition task and over 90% for the 3-way 5-shot small-shot recognition task, outperforming ProtoNet across all recognition tasks. It can be seen that the method of the present invention has certain advantages in solving the problem of small sample SAR image target recognition under conditions of varying degrees of sample scarcity.

[0060] Simulation 2: Evaluate the recognition performance under different pitch angle conditions.

[0061] This simulation experiment evaluated the small-shot recognition performance under different elevation angles using the MSTAR dataset. Three deep learning-based small-shot target recognition methods, ProtoNet, RelationNet, and MSAR, were selected for comparison. All seven SAR image samples at an elevation angle of 17° were used for model training, while three SAR image samples at elevation angles of 15°, 17°, and 30° were used for model testing to simulate 3-way 1-shot and 3-way 5-shot small-shot recognition tasks. The recognition performance of each method under different elevation angles is shown in Table 4. As can be seen, in the 3-way 1-shot recognition task at a 15° elevation angle, the proposed method achieved the highest recognition rate, improving by 19.52% over the MSAR method. As the elevation angle increases, the difficulty of small-shot SAR image target recognition increases, and recognition performance decreases. Overall, however, the proposed method demonstrates superior small-shot SAR image target recognition performance across all elevation angles.

[0062] Simulation 3: Exploring the effectiveness of different learning tasks.

[0063] In order to explore the effectiveness of different learning tasks in solving the small sample SAR image target recognition problem, this simulation experiment carried out relevant ablation experiments on the MSTAR standard dataset. The network containing class boundary learning is marked as the baseline model, the network containing class boundary learning and invariant feature learning is marked as model 1, the network containing class boundary learning and equivariant feature learning is marked as model 2, and the network containing three learning tasks is the method of the present invention. This experiment will demonstrate the performance of these four models on the 3-way 1-shot small sample target recognition task. The recognition performance of each model is as follows Figure 2 As shown in the figure, compared to the baseline model, adding the invariant feature learning task can achieve a 0.32% improvement in recognition performance, adding the equivariant feature learning task can achieve a 0.86% improvement in recognition performance, and adding both the equivariant and invariant feature learning tasks can achieve a 1.76% improvement in recognition performance. This shows that all three learning tasks have a positive effect on small-sample SAR image target recognition performance.

[0064] Table 1 Details of the MSTAR dataset division

[0065]

[0066]

[0067] Table 2 Data settings in small sample recognition tasks

[0068]

[0069] Table 3 Recognition performance evaluation under different degrees of sample scarcity conditions (%)

[0070]

[0071] Table 4 Recognition performance evaluation under different pitch angle conditions (%)

[0072]

Claims

1. A small-sample SAR image target recognition method based on multi-task representation learning, the method steps are as follows: Step 1: Collect sufficient number of auxiliary SAR image samples D b ={(x,y)}, where It is a SAR image with a channel number of 1 and a pixel size of h×w. Indicates the category label of the SAR image, there are N b categories; at the same time, a small sample SAR target sample set D is collected f ={(x,y)}, where represents the category label of the small sample SAR image, and N b auxiliary classes and N f The target classes of the small samples are completely disjoint; Step 1.1: Crop all SAR images into target areas of interest with uniform pixel size; Step 1.2: Perform multiple geometric transformations on the auxiliary SAR image samples and perform transformation annotation on the transformed SAR image samples; Step 2: Design a multi-task representation learning network and use the auxiliary SAR image samples in step 1 for network training; The multi-task representation learning network includes: feature extractor f B , equivariant feature learning branch f E , invariant feature learning branch f I , class boundary learning branch f C , the feature extractor f B The input is SAR image, and the equivariant feature learning branch f E , invariant feature learning branch f I , class boundary learning branch f C The input of the feature extractor f B Output; Step 2.1: Build feature extractor f B ; Use deformable convolution operation to give feature extractor f B The ability to perceive the morphological features of the target and adaptively enhance and suppress features at the feature channel level through the attention mechanism to obtain feature maps with high information content; Step 2.2: Construct equivariant feature learning branch f E ; According to the transformation annotation, a supervised approach is adopted to enable the feature extractor to have the ability to perceive changes in the target sample. The loss function of the equivariant feature learning branch adopts the cross entropy loss function, which is expressed as follows: Among them, N represents the number of SAR image samples input to the network, N t Indicates the number of conversion categories of SAR images, y ij is the manually assigned conversion category label, P ij represents the predicted probability that sample i belongs to category j; Step 2.3: Construct the invariant feature learning branch f I ; In order to enable the feature extractor to have the ability to extract target invariant features, a contrast loss function is used to make the similarity of the same SAR image samples with different conversion categories higher. Its expression is as follows: Where s(·) represents the similarity function, D n represents the negative sample feature set, τ represents the adjustable temperature coefficient, M represents the total number of conversion categories, v 0 Represents the original image features without any transformation, v m represents the SAR image features after the mth transformation, v 0 Represents the stored v 0 Image features; Step 2.4: Construct the class boundary learning branch f C ; According to the category label y, a supervised approach is used to aggregate SAR image samples of the same category in the feature embedding space. The loss function of the class boundary learning branch also uses the cross entropy loss function, which is expressed as follows: Among them, N b Indicates the number of categories of SAR images, represents the category label; p ij represents the predicted probability that sample i belongs to category j; Step 2.5: Use the triple loss function to jointly train the multi-task representation learning network. The total loss function is defined as follows: L=L eq +L in +L ba Step 3: Use the trained feature extractor f B Extract target features from small sample SAR images; Step 4: Use the features extracted in step 3 to fit a logistic regression classifier to complete the small sample target recognition task.

Images