Small-sample SAR target recognition method based on two-stage contrastive learning framework

Abstract

A small sample SAR target recognition method based on a two-stage contrastive learning framework involves image recognition. Aiming at the practical problems of difficulty in obtaining SAR data sets and time-consuming and labor-intensive data annotation, a contrastive learning framework based on a two-stage training strategy is proposed to solve the SAR target recognition under small sample data conditions. The training process is decoupled into a representation learning stage and a classification learning stage, including: 1) In the representation learning stage, supervised contrastive learning is used to train the representation module of the network on the original SAR data set; 2) In the classification learning stage, the encoding module is fixed and the classification module of the network is trained using a rebalancing strategy. Based on contrastive learning in machine learning and combined with the idea of ​​decoupled learning, the method has strong practicality and good stability, and can meet the requirements of high standards for SAR target recognition accuracy in cases of imbalanced categories and small sample data.

Description

Technical Field

[0001] The invention belongs to the field of image recognition, and relates to a SAR target recognition technology under small sample data conditions, and in particular introduces a decoupling learning and contrast learning method. Background Art

[0002] Synthetic aperture radar (SAR) is a high-resolution radar device based on the aperture principle. Compared with optical remote sensing, it can overcome the influence of harsh climate factors and day and night factors, and can be carried on satellites, drones and other flying platforms. The uniqueness of SAR enables it to play a high application value in military, mapping, agriculture and other fields (A. Moreira, P. Prats-Iraola, M. Younis, G. Krieger, I. Ha-jnsek, and KP Papathanassiou, "A tutorial on synthetic aperture radar," IEEE Geoscience and remote sensing magazine, vol. 1, no. 1, pp. 6–43, 2013). In recent years, with the development of convolutional neural networks, many deep learning algorithms for SAR target recognition have emerged, but most of these algorithms are based on balanced data sets. In real scenes, SAR data is usually unbalanced because some targets rarely appear or are difficult to collect, resulting in small sample data of the tail class. Under the condition of small sample data, the model is easy to overfit to the head class, which brings severe challenges to traditional deep learning algorithms. In addition, collecting a balanced data set requires a lot of manpower and material resources, which is impractical; therefore, making full use of small sample SAR data to train a deep learning model with good generalization performance has important practical significance.

[0003] Rebalancing strategies (e.g., reweighting and resampling) are prominent and effective methods to solve the problem of unbalanced training data, but traditional solutions are mostly classification network learning, hoping to increase the weight of the tail class, thereby reducing the model's bias towards the head class. Although the rebalancing strategy can significantly promote the classification learning of deep networks, it also accidentally damages the representation ability of the learned deep features to a certain extent (Boyan Zhou, Quan Cui, Xiu-Shen Wei, and Zhao-Min Chen, "Bbn: Bilateral-branch network with cumulative learning for long-tailed visual recognition," in Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 9719–9728). Summary of the invention

[0004] The purpose of the present invention is to provide a small sample SAR target recognition method based on a two-stage contrastive learning framework, which can improve the generalization performance of small sample SAR target recognition using a deep learning model, improve the target recognition accuracy of small samples, meet practical application needs, and improve the overall performance of the model, in order to address the problems existing in the prior art, such as the difficulty in acquiring SAR data sets, time-consuming and labor-intensive data annotation, and data imbalance leading to poor generalization ability of the model for small sample SAR targets.

[0005] The present invention decouples the training process into a characterization learning phase and a classification learning phase, which specifically includes the following steps:

[0006] 1) In the representation learning stage, supervised contrastive learning is used to train the representation module of the network on the original SAR dataset;

[0007] 2) In the classification learning stage, the encoding module is fixed and a rebalancing strategy is used to train the classification module of the network.

[0008] In step 1), the representation module that uses supervised contrastive learning to train the network on the original SAR data set first uses the contrastive learning algorithm Supervised Contrastive Learning as the training strategy of the representation module, and designs a data enhancement strategy based on the characteristics of the SAR target to further optimize the training effect of contrastive learning, so that the representation module can extract more generalized features for subsequent classification tasks; the specific steps of the representation learning stage are as follows:

[0009] (1) SAR images are obtained by synthetic aperture radar on a flying platform from a bird's-eye view. The angle of the target in the image is uncertain. In order to identify the SAR target even when part of the image information is missing, the following data enhancement strategy is designed: random cropping and scaling, random flipping, and random rotation. The data enhancement strategy is applied twice to each SAR image to obtain 2N SAR images.

[0010] (2) Send the 2N SAR images to the encoding module in the network respectively, and obtain the feature vector f corresponding to each SAR image i , then the eigenvector f i It is sent to the projection module and mapped to another representation space to obtain a nonlinear representation z i =Proj(x i )=W (2) σ(W (1) f i ), where σ is the ReLu activation function and W is a single-layer linear layer; the projection module used is implemented by a two-layer perceptron with one hidden layer;

[0011] (3) Use Norm normalization z i , and then use the standardized z i Calculate the loss of supervised contrastive learning and train the network until convergence. The loss of supervised contrastive learning is calculated as follows:

[0012]

[0013] Where P(i) represents a sum z i The subscript set of all samples of the same category, |P(i)| is the size of this set, and τ is a temperature hyperparameter.

[0014] In step 2), the rebalancing strategy is used to improve the recognition accuracy of small sample SAR targets, and the rebalancing strategy includes a class-balanced sampling strategy and a cost-sensitive cross entropy loss strategy, and a classification module of the training network is adopted;

[0015] The specific steps of the class-balanced sampling strategy are as follows:

[0016] (1) Perform class-balanced sampling on the training set and randomly select N SAR images to be sent to the encoding module to generate the feature vector f i , and the gradient interruption strategy is adopted for the encoding module to avoid the adverse effects of subsequent training of the classification module. For class-balanced sampling, the probability of the i-th sample being selected is as follows:

[0017]

[0018] Among them, y i is the category of the i-th sample, for y i The number of samples in the class;

[0019] (2) Based on the generated feature vector f i The present invention uses a linear layer as a classification module, and obtains the feature vector f through the classification module i The predicted probability of each class is then used as the loss to train the network until convergence.

[0020] The specific steps of the cost-sensitive cross entropy loss strategy are as follows:

[0021] (1) is similar to step (1) of the class-balanced sampling strategy, except that random sampling is used to replace class-balanced sampling, that is, the original training data is not modified;

[0022] (2) is similar to step (2) of the class-balanced sampling strategy, except that cost-sensitive cross entropy is used as the training loss, that is, the ordinary cross entropy loss is weighted. The cost-sensitive cross entropy loss is defined as follows:

[0023]

[0024] Among them, n min is the minimum number of all categories, L CE is the cross entropy loss function.

[0025] Compared with the prior art, the advantages and technical effects of the present invention are:

[0026] (1) Aiming at the background knowledge of SAR images, the present invention proposes a data augmentation strategy for SAR images, including random cropping and scaling, random flipping and random rotation. Based on this simple and effective strategy, contrastive learning of SAR images is performed to improve the potential of contrastive learning and obtain better SAR image representation capabilities.

[0027] (2) Different from the traditional small sample training strategy, the present invention is based on contrastive learning in machine learning, and combines the idea of ​​decoupled learning, adopts a two-stage training strategy, combines the advantages of contrastive learning and rebalancing strategy, and performs well on SAR image data sets. These methods improve the generalization performance of small sample SAR target recognition and improve the accuracy of the present invention in target recognition with small data volume.

[0028] (3) The present invention addresses the difficulty that data imbalance leads to poor generalization ability of the model for small sample SAR targets in practical problems. The network can learn a more generalized representation ability from the imbalanced data. While maintaining the representation ability of the network, the classification ability of the network is trained through the traditional rebalancing strategy, and the decision boundary of the network is fine-tuned, thereby further improving the network's recognition performance for small sample SAR targets. The present invention has strong practicality and good stability, and can improve the generalization performance of small sample SAR target recognition using a deep learning model, improve the target recognition accuracy of small samples, and meet the requirements of high standard of SAR target recognition accuracy in cases of category imbalance and small sample data. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Schematic diagram of the two-stage contrastive learning framework.

[0030] Figure 2 Schematic diagram of small sample SAR data distribution.

[0031] Figure 3 Schematic diagram of TSNE.

[0032] Figure 4 This is a comparison chart of the ablation experiment results of the classification module. DETAILED DESCRIPTION

[0033] To make the objectives, technical solutions and advantages of the present invention more clear, the present invention will be further described in the following embodiments in conjunction with the accompanying drawings.

[0034] The present invention proposes a small sample SAR target recognition method based on a two-stage contrastive learning framework. This method decouples the training process into representation learning and classification learning, so that the model takes into account both representation learning and classification learning. Through the two-stage training strategy, the model can maintain the accuracy of the head class while improving the recognition accuracy of the small sample of the tail class, thereby improving the overall performance of the model.

[0035] The embodiment of the present invention includes the following steps:

[0036] A. Representation learning stage;

[0037] B. Classification learning stage;

[0038] In order to deal with the difficulty of poor generalization ability of the model for small sample SAR targets caused by data imbalance in practical problems, the present invention hopes that the network can learn a more generalized representation ability from the imbalanced data. While maintaining the representation ability of the network, the classification ability of the network is trained through the traditional rebalancing strategy, and the decision boundary of the network is fine-tuned, thereby further improving the recognition performance of the network for small sample SAR targets. The training process is as follows Figure 1In the representation learning stage, the present invention uses supervised contrastive learning to train the representation module of the network on the original SAR data set; in the classification learning stage, the present invention fixes the encoding module and uses a class-balanced sampling strategy or a cost-sensitive cross entropy loss strategy to train the classification module of the network. In order to describe the subsequent content more clearly, the present invention defines the total number of categories as C, and the number of SAR images in each round of training is N.

[0039] The specific steps of step A are as follows:

[0040] Although the actual SAR data set is unbalanced, it includes real correlation information in the data, which is conducive to the network learning a better representation ability. The present invention hopes that the representation module is trained and learned in the original SAR data, and a more generalized representation ability is obtained through real and accurate data distribution information, so as to avoid the representation module from learning wrong distribution information. In this step, the present invention first adopts the most popular contrastive learning algorithm Supervised Contrastive Learning (Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, ​​Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan, "Supervised contrastive learning," arXiv preprint arXiv:2004.11362, 2020) as the training strategy of the representation module, and designs a data enhancement strategy based on the characteristics of the SAR target, further optimizes the training effect of contrastive learning, and enables the representation module to extract more generalized features for subsequent classification tasks. The specific steps of this method are as follows:

[0041] Step 1: SAR images are obtained by synthetic aperture radar imaging on a flying platform from a bird's eye view, and the angle of the target in the image is uncertain. In addition, the SAR target should also be identified when part of the image information is missing. Therefore, the present invention designs the following data enhancement strategy for the above situation: random cropping and scaling, random flipping and random rotation. The data enhancement strategy is applied twice to each SAR image to obtain 2N SAR images.

[0042] Step 2: Send the 2N SAR images to the encoding module in the network (such as AlexNet, VGG, ResNet, EfficientNet) to obtain the feature vector f corresponding to each SAR image. i Then the eigenvector f i It is sent to the projection module and mapped to another representation space to obtain a nonlinear representation zi =Proj(x i )=W (2) σ(W (1) f i ), where σ is the ReLu activation function and W is a single-layer linear layer. Therefore, the projection module used in the present invention is implemented by a two-layer perceptron with one hidden layer.

[0043] Step 3: In order to be able to express the similarity between two vectors through dot product later, the present invention uses norm to normalize z i Then use the normalized z i Calculate the loss of supervised contrastive learning and train the network until convergence. The loss calculation of supervised contrastive learning is as follows:

[0044]

[0045] Where P(i) represents a sum z i The subscript set of all samples of the same category, |P(i)| is the size of this set, and τ is a temperature hyperparameter.

[0046] The specific steps of step B are as follows:

[0047] The rebalancing strategy can effectively improve the recognition accuracy of small sample SAR targets, but it may distort the original distribution information of the data and destroy the learning of the model's characterization ability. In order to better utilize the advantages of the rebalancing strategy and avoid its impact on the characterization ability, the present invention fixes the encoding module and trains the classification module through one of the rebalancing strategies. The rebalancing strategies adopted by the present invention include: class balanced sampling strategy and cost code sensitive cross entropy loss strategy.

[0048] The specific steps of the class-balanced sampling strategy are as follows:

[0049] Step 1: Perform class-balanced sampling on the training set, and then randomly select N SAR images and send them to the encoding module to generate the feature vector f i , and the gradient interruption strategy is adopted for the encoding module to avoid the adverse effects of subsequent training of the classification module. For class-balanced sampling, the probability of the i-th sample being selected is as follows:

[0050]

[0051] Among them, y i is the category of the i-th sample, for y i The number of samples in the class.

[0052] Step 2: Based on the generated feature vector f iThe present invention uses a linear layer as a classification module, and obtains the feature vector f through the classification module i The predicted probability of each class is then used as the loss to train the network until convergence.

[0053] The specific steps of the cost-sensitive cross entropy loss strategy are as follows:

[0054] The first step is similar to the first step of class-balanced sampling, except that random sampling is used to replace class-balanced sampling, that is, the original training data is not modified.

[0055] Step 2: Similar to the second step of class-balanced sampling, the difference is that cost-sensitive cross entropy is used as the training loss, that is, the ordinary cross entropy loss is weighted. The definition of cost-sensitive cross entropy loss is as follows:

[0056]

[0057] Among them, n min is the minimum number of all categories, L CE is the cross entropy loss function.

[0058] Specific examples are given below.

[0059] 1. In the representation learning stage, the specific experimental settings are as follows: SGD is used as the optimizer, with momentum of 0.9 and weight decay of 1×10 -4 , the batch size is 64 to train the feature network for 1000 rounds. In addition, the invention initializes the learning rate to 0.05, and uses the cosine decay strategy to adjust the learning rate. In addition, the temperature coefficient τ is set to 0.1 for the optimization of the supervised contrastive learning loss function.

[0060] 2. In the classifier learning stage, the specific experimental details are as follows: the weight parameters of the encoding module are fixed, the batch size is 128, and the learning rate is 0.01 to train the classifier for 100 rounds. The learning rate during training is adjusted to the cosine decay strategy, and the optimizer also uses SGD, with the parameters momentum 0.9 and weight decay 5×10 -5 .

[0061] In order to verify that the present invention has good generalization performance for small sample SAR data, the Exp distribution and Step distribution are used to simulate the distribution of small sample data. The specific distribution is as follows: Figure 2As shown in the figure, other traditional methods for solving the class imbalance problem are compared. Table 1 gives an experimental comparison of some existing works under small sample SAR data. It can be seen from the table that compared with other methods, the present invention performs best overall. Specifically, the two-stage contrastive learning framework can maintain the accuracy of the head class, while effectively improving the recognition accuracy and overall performance of the small sample of the tail class.

[0062] Table 1

[0063]

[0064] In order to explore the impact of the rebalancing strategy on the encoding module in the present invention, different strategies are designed for training the encoding module, and a fixed class balancing strategy is used for training the classification module. Table 2 shows the model recognition accuracy under different strategy training under Exp distribution. As shown in the table, the baseline method (directly training the encoding module without any strategy) is much better than the method adding the rebalancing strategy in the small sample of the tail class and the overall performance. This shows that good features produce good classification, and the specific features are represented as follows Figure 3 Specifically, the encoding module directly trained by the original data can obtain good feature representation, thereby improving the recognition performance of the model.

[0065] Table 2

[0066]

[0067] In order to explore the impact of the rebalancing strategy on the classification module in the present invention, multiple strategies are designed to experiment on the classification module. Figure 4 The classification results of the classification module under different strategies under Exp distribution are given. As shown in the figure, when the classification module adopts the reweighting or resampling method, the model can achieve better performance. This shows that the rebalancing strategy helps to change the decision boundary of the classification module, thereby improving the recognition performance of the model.

[0068] The present invention improves the potential of contrastive learning and obtains better SAR image representation capabilities. It adopts a two-stage training strategy, combines the advantages of contrastive learning and rebalancing strategies, and performs well on SAR image data sets. It improves the generalization performance of small sample SAR target recognition and improves the accuracy of target recognition for small data volumes.

[0069] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A small sample SAR target recognition method based on a two-stage contrastive learning framework, characterized by The following steps are involved: 1) In the representation learning stage, supervised contrastive learning is used to train the representation module of the network on the original SAR dataset; The specific steps of the representation learning phase are as follows: (1) SAR images are obtained by synthetic aperture radar from a bird's-eye view on a flying platform. The angle of the target in the image is uncertain. In order to identify the SAR target even when some image information is missing, the following data enhancement strategy is designed: random cropping and scaling, random flipping, and random rotation. The data enhancement strategy is applied twice to each SAR image to obtain 2N SAR images. (2) Send the 2N SAR images to the encoding module in the network respectively, and obtain the feature vector f corresponding to each SAR image i , then the eigenvector f i It is sent to the projection module and mapped to another representation space to obtain a nonlinear representation z i =Proj(x i )=W (2) σ(W (1) f i ), where σ is the ReLu activation function and W is a single-layer linear layer; the projection module used is implemented by a two-layer perceptron with one hidden layer; (3) Use l2 norm to normalize z i , and then use the standardized z i Calculate the loss of supervised contrastive learning and train the network until convergence. The loss of supervised contrastive learning is calculated as follows: Where P(i) represents a sum z i The subscript set of all samples of the same category, |P(i)| is the size of this set, and τ is a temperature hyperparameter; 2) In the classification learning stage, the encoding module is fixed and the classification module of the network is trained using a rebalancing strategy; The rebalancing strategy is used to improve the recognition accuracy of small sample SAR targets. The rebalancing strategy includes a class-balanced sampling strategy and a cost-sensitive cross entropy loss strategy, and a classification module of the training network is adopted in one of them; The specific steps of the class-balanced sampling strategy are as follows: (1) Perform class-balanced sampling on the training set and randomly select N SAR images to be sent to the encoding module to generate the feature vector f i , and the gradient interruption strategy is adopted for the encoding module to avoid the adverse effects of subsequent training of the classification module; for class-balanced sampling, the probability of the i-th sample being selected is as follows: Among them, y i is the category of the i-th sample, for y i The number of samples in the class; (2) Based on the generated feature vector f i , a linear layer is used as the classification module, and the feature vector f is obtained through the classification module i The predicted probability of each class is then used as the loss to train the network until convergence; The specific steps of the cost-sensitive cross entropy loss strategy are as follows: (1) is similar to step (1) of the class-balanced sampling strategy, except that random sampling is used to replace class-balanced sampling, that is, the original training data is not modified; (2) is similar to step (2) of the class-balanced sampling strategy, except that cost-sensitive cross entropy is used as the training loss, that is, the ordinary cross entropy loss is weighted. The cost-sensitive cross entropy loss is defined as follows: Among them, n min is the minimum number of all categories, L CE is the cross entropy loss function.

2. The small sample SAR target recognition method based on the two-stage contrastive learning framework as claimed in claim 1 is characterized in that In step 1), the representation module that uses supervised contrastive learning to train the network on the original SAR data set first uses the contrastive learning algorithm Supervised Contrastive Learning as the training strategy of the representation module, and designs a data enhancement strategy based on the characteristics of the SAR target, further optimizes the training effect of contrastive learning, and enables the representation module to extract more generalized features for subsequent classification tasks.

Images