Few-shot SAR Target Classification Method Based on Self-supervised Learning and Nearest Neighbor Network

Abstract

The present invention relates to a few-shot SAR target classification method based on self-supervised learning and nearest neighbor network, including: obtaining a training task set and a test task set from a number of synthetic aperture radar images; constructing a network model in the pre-training stage; iteratively training the network model in the pre-training stage by using the training task set; constructing a network model in the self-supervised training stage; loading the parameters of the trained network model in the pre-training stage into the network model in the self-supervised training stage, and iteratively training the loaded network model in the self-supervised training stage by using the training task set; inputting the test task set into the trained third feature extraction module for feature extraction to obtain a set of test feature vector groups; calculating the similarity between the test feature vector corresponding to the test query sample and the test feature vector corresponding to each test support sample in the set of test feature vector groups to obtain the target classification result. This classification method effectively improves the classification accuracy of few-shot targets.

Description

Technical Field

[0001] The invention belongs to the technical field of radar image processing, and in particular relates to a small sample SAR target classification method based on self-supervised learning and a nearest neighbor network. Background Art

[0002] Synthetic Aperture Radar (SAR) is an active earth observation system that can be installed on aircraft, satellites, spacecraft and other flying platforms. It has the advantage of all-day and all-weather observation of the earth and has a certain penetration ability to the earth's surface. Therefore, the SAR system has unique advantages in the application of disaster monitoring, environmental monitoring, ocean monitoring, resource exploration, mapping and military, and has attracted more and more attention.

[0003] SAR target classification is an image processing method that distinguishes different categories of targets based on the different features reflected in the image information. The current mainstream SAR target classification methods can be divided into traditional classification methods and deep learning-based classification methods. The traditional SAR target classification method classifies image targets based on image features such as color, texture, shape, and spatial relationship. It usually uses manual feature selection and classifier design. However, this often requires a lot of experience and strong professional knowledge to set specific algorithms for specific targets, which is time-consuming and difficult to promote, and it is impossible to obtain good classification accuracy. In recent years, with the development of deep learning, a series of powerful convolutional neural networks such as VGG, ResNet, and DenseNet have emerged, which can automatically extract more robust features for SAR targets, thereby achieving better SAR target classification results, and have been widely used by researchers.

[0004] SAR target classification methods based on deep learning usually require a large number of training samples to train the model in order to obtain high classification accuracy on the test samples. However, the number of SAR images that can be obtained in reality is relatively small. For small sample SAR targets, SAR target classification methods based on deep learning will overfit due to insufficient training samples, resulting in low SAR target classification accuracy.

[0005] To solve this problem, the existing technology improves the model structure and designs a special model with lower requirements for the number of samples to improve the classification accuracy of small-sample SAR targets. For example, the existing technology discloses a small-sample SAR automatic target recognition method based on a hybrid inference network. The small-sample learning method based on the hybrid inference network proposed by this method overcomes the deficiencies of using only inductive inference or transductive inference in traditional small-sample learning and improves the recognition performance; an enhanced hybrid loss is used to constrain the embedding network for learning, mapping the samples into this embedding space, in which both inductive inference and transductive inference can be well performed; a hybrid inference network is adopted, and only a small number of training samples are required in the target classification problem to obtain a recognition accuracy rate close to that of the traditional SAR target recognition method. In the case of limited SAR image samples, the recognition accuracy is effectively improved. However, due to the limited labeled samples, this method cannot highly rely on the embedding network to obtain better feature representations; the model converges slowly and the model generalization ability is poor; the Euclidean distance metric method adopted has a certain deviation and is not a very efficient metric method when the number of samples is small. Therefore, the accuracy of this method for SAR target classification is still low. Summary of the Invention

[0006] To solve the above problems existing in the prior art, the present invention provides a small-sample SAR target classification method based on self-supervised learning and the nearest neighbor network. The technical problems to be solved by the present invention are realized through the following technical solutions:

[0007] An embodiment of the present invention provides a small-sample SAR target classification method based on self-supervised learning and the nearest neighbor network, including the steps of:

[0008] S1. Obtain a training task set and a test task set from a number of synthetic aperture radar images. Among them, each training task in the training task set includes training support samples and training query samples, and each test task in the test task set includes a test support sample set and a test query sample;

[0009] S2. Construct a pre-training stage network model, where the pre-training stage network model includes a first feature extraction module, a multi-layer perceptron module, and a similarity calculation module cascaded in sequence;

[0010] S3. Iteratively train the pre-training stage network model using the training task set, and update the pre-training stage network model using the cross-entropy loss function to obtain a trained pre-training stage network model;

[0011] S4. Construct a network model for the self-supervised training stage. The network model for the self-supervised training stage includes a second feature extraction module and a third feature extraction module connected in parallel. The first feature extraction module, the second feature extraction module, and the third feature extraction module have the same structure.

[0012] S5. Load the parameters of the first feature extraction module in the trained pre-training stage network model into the network model for the self-supervised training stage, and iteratively train the loaded network model for the self-supervised training stage using the training task set. During the training process, update the third feature extraction module using the negative cosine similarity loss function and the self-supervised loss function to obtain a trained third feature extraction module.

[0013] S6. Input the test task set into the trained third feature extraction module for feature extraction to obtain a set of test feature vector groups.

[0014] S7. Calculate the similarity between the test feature vector corresponding to the test query sample and the test feature vectors corresponding to each test support sample in the set of test feature vector groups to obtain a target classification result.

[0015] In an embodiment of the present invention, the first feature extraction module includes a first feature extraction sub-module and a second feature extraction sub-module cascaded in sequence, where

[0016] The first feature extraction sub-module includes a first convolutional layer, a first batch normalization layer, a first activation function layer, and a max pooling layer cascaded in sequence.

[0017] The second feature extraction sub-module includes a second convolutional layer, a second batch normalization layer, and a second activation function layer cascaded in sequence.

[0018] The multi-layer perceptron module includes a fully connected layer.

[0019] In an embodiment of the present invention, the number of convolution kernels in both the first convolutional layer and the second convolutional layer is 64, the size of the convolution kernels is 3×3, and the stride and padding are both 1. The first activation function layer and the second activation function layer both use Leaky Relu, and their parameters are both 0.2. The size of the pooling kernel of the max pooling layer is 2×2, and the stride is 2.

[0020] In an embodiment of the present invention, step S3 includes:

[0021] S31. Initialize and set the number of iterations in the pre-training stage, the maximum number of iterations in the pre-training stage, and the network model for the pre-training stage at the nth iteration.

[0022] S32. Use the first feature extraction module to extract features from each of the training tasks, obtaining a set of first feature vector groups;

[0023] S33. Use the multi-layer perceptron module to map each feature in the set of feature vector groups to the sample label space, obtaining a set of second feature vector groups;

[0024] S34. Use the similarity calculation module to calculate the similarity between the feature vector corresponding to the training query sample in the set of second feature vector groups and the feature vector corresponding to each of the training support samples, obtaining a set of similarity scores;

[0025] S35. Use the cross-entropy loss function to calculate the first loss value of the network model in the pre-training stage during each iteration, and use the gradient descent method to backpropagate the partial derivative of the first loss value with respect to the first weight parameter in the network model in the pre-training stage to update the first weight parameter, obtaining the updated first weight parameter;

[0026] S36. Determine whether the iteration is completed. If so, obtain the trained network model in the pre-training stage; if not, continue the iterative training.

[0027] In an embodiment of the present invention, the formula for calculating the similarity using the similarity calculation module is:

[0028]

[0029]

[0030] where represents the similarity score between and denotes the feature vector obtained by the support sample successively passing through the first feature extraction module D and the multi-layer perceptron module E, denotes the feature vector obtained by the query sample passing through the feature extraction module D and the multi-layer perceptron module E, m represents the number of local descriptors into which the support feature vector is divided, z represents the number of local descriptors into which the query feature vector is divided, x i represents the i-th local descriptor in the support feature vector, represents the j-th local descriptor in the query feature vector.

[0031] In an embodiment of the present invention, the first loss value is:

[0032]

[0033] Among them, M represents the number of images corresponding to each category, and m1 represents and y m1 the m1-th true label and predicted label in represents the true label corresponding to the predicted label, y m1 represents the predicted label, and ln represents the natural logarithm with base e;

[0034] The updated first weight parameter is:

[0035]

[0036] Among them, represents the update result of the first weight parameter ω s1 ω s1 represents the first weight parameter, L s1 represents the first loss value, and η represents the learning rate of L s1 ; represents the derivative operation.

[0037] In an embodiment of the present invention, the second feature extraction module includes a third convolutional layer, a third batch normalization layer, a third activation function layer, and a second max pooling layer cascaded in sequence;

[0038] The third feature extraction module includes a fourth convolutional layer, a fourth batch normalization layer, a fourth activation function layer, and a third max pooling layer cascaded in sequence.

[0039] In an embodiment of the present invention, step S5 includes:

[0040] S51. Initialize and set the number of iterations in the self-supervised training stage, the maximum number of iterations in the self-supervised training stage, and the network model in the self-supervised training stage at the n-th iteration;

[0041] S52. Load the parameters of the first feature extraction module in the trained pre-training stage network model into the second feature extraction module, and use the loaded second feature extraction module to extract features for each training task to obtain a set of third feature vector groups; load the parameters of the first feature extraction module in the trained pre-training stage network model into the third feature extraction module, and use the loaded second feature extraction module to extract features for each training task to obtain a set of fourth feature vector groups;

[0042] S53. Use the sum of the negative cosine similarity loss function and the self-supervised loss function as the training loss function to calculate the second loss value of the network model in the self-supervised training stage during each iteration. Then, use the partial derivative of the second loss value with respect to the second weight parameter to perform backpropagation in the network model in the self-supervised training stage to update the second weight parameter, and obtain the updated second weight parameter;

[0043] S54. Determine whether the iteration is completed. If so, obtain the trained network model in the self-supervised training stage; if not, continue the iterative training.

[0044] In an embodiment of the present invention, the training loss function is:

[0045] L = L cos + L self-supervised

[0046] where L cos is the negative cosine similarity loss function, and L self-supervised is the self-supervised loss function;

[0047] The negative cosine similarity loss function is:

[0048]

[0049] where D represents the second feature extraction module, G represents the third feature extraction module, and ||·||2 represents l2 regularization;

[0050] The self-supervised loss function is:

[0051] L self-supervised (x) = U(P1, Z2) + U(P2, Z1)

[0052]

[0053]

[0054] where P1 represents the feature vector group obtained by passing X1 through an h module, P2 represents the feature vector group obtained by passing X2 through an h module, X1 and X2 respectively represent two effect diagrams obtained by data augmentation of the training task set, Z1 represents the feature vector group obtained by passing X1 through an f module, and Z2 represents the feature vector group X2 obtained by passing X2 through an f module.

[0055] Compared with the prior art, the beneficial effects of the present invention:

[0056] The classification method of the present invention introduces a network model in the pre-training stage. By loading the parameters of the trained network model in the pre-training stage into the network model in the self-supervised training stage, the convergence of the network model in the self-supervised training stage can be accelerated and the generalization ability of the model can be improved. When training the network model in the self-supervised training stage, a self-supervised loss function and a negative cosine similarity loss function are introduced to obtain better feature representations, and an efficient nearest neighbor metric method is introduced to calculate the similarity between the support samples and the query samples. Therefore, this method avoids the influence on the SAR target classification accuracy caused by the slow model convergence speed, weak model generalization ability, inability to obtain better feature representations, and inefficient metric methods in the prior art, and effectively improves the classification accuracy of small-sample targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a schematic flow chart of a small-sample SAR target classification method based on self-supervised learning and nearest neighbor network provided by an embodiment of the present invention;

[0058] Figure 2 It is a schematic implementation flowchart of iterative training for the network model in the pre-training stage provided by an embodiment of the present invention;

[0059] Figure 3 It is a schematic implementation flowchart of iterative training for the network model in the self-supervised training stage provided by an embodiment of the present invention;

[0060] Figure 4 It is a schematic implementation flowchart of obtaining the target classification result of small-sample SAR images provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0061] The following further describes the present invention in detail with specific embodiments, but the implementation manners of the present invention are not limited thereto.

[0062] Embodiment 1

[0063] Please refer to Figure 1 , Figure 1 It is a schematic flow chart of a small-sample SAR target classification method based on self-supervised learning and nearest neighbor network provided by an embodiment of the present invention. The small-sample SAR target classification method includes the steps:

[0064] S1. Obtain a training task set and a test task set from a number of synthetic aperture radar images; wherein, each training task in the training task set includes a training support sample and a training query sample, and each test task in the test task set includes a test support sample set and a test query sample. Specifically, it includes the steps:

[0065] S11. Obtain U synthetic aperture radar (SAR) images. The size of each SAR image is h×h. The U SAR images contain C different target categories, and each target category corresponds to M SAR images, where C≥10, M≥200, 64≤h≤2560, and U≥2000.

[0066] S12. Mark the target categories in each SAR image, and randomly select C train target categories corresponding to C train ×M SAR images and their labels to form a training data set Use the remaining C test target categories corresponding to C test ×M SAR images and their labels to form a test data set where C train +C test =C, ∩ represents the intersection, C train >C / 2, C train ×M + C test ×M = U.

[0067] S13. Randomly select z = C SAR images containing C test target categories from the training data set. Perform one-hot encoding on the labels of each selected SAR image to obtain test the corresponding set of label vectors, and select z1 = C ×K label vectors and their corresponding SAR images to form a training support sample set test Use the remaining z2 = C (M - K) label vectors and their corresponding SAR images to form a training query sample set test Then combine with each training query sample to obtain a training task set where, represents the a-th training support sample composed of a SAR image and its corresponding label vector, represents the b-th training query sample composed of a SAR image and its corresponding label vector, 1≤K≤10, represents the b-th training task,

[0068] S14. Randomly select z = C SAR images containing C test target categories from the test data set. test×M SAR images, perform one-hot encoding on the labels of each selected SAR image to obtain the corresponding label vector set, and select z1 = C test ×K label vectors and their corresponding SAR images in the label vector set to form a test support sample set The remaining z2 = C test (M - K) label vectors and their corresponding SAR images form a test query sample set Then Combine with each test query sample to obtain a test task set Among them, represents the e-th test support sample composed of a SAR image and its corresponding label vector, represents the g-th test query sample composed of a SAR image and its corresponding label vector, represents the g-th test task,

[0069] S2. Construct a network model in the pre-training stage.

[0070] Specifically, the constructed network model in the pre-training stage includes a first feature extraction module Da, a multi-layer perceptron module E, and a similarity calculation module R cascaded in sequence.

[0071] In a specific embodiment, the first feature extraction module Da includes a first feature extraction sub-module D1 and a second feature extraction sub-module D2 cascaded in sequence. Among them, the first feature extraction sub-module D1 includes a first convolutional layer, a first batch normalization layer, a first activation function layer, and a max pooling layer cascaded in sequence. The second feature extraction sub-module D2 includes a second convolutional layer, a second batch normalization layer, and a second activation function layer cascaded in sequence. The multi-layer perceptron module E includes a fully connected layer.

[0072] In a specific embodiment, in the first feature extraction sub-module D1 and the second feature extraction sub-module D2, the number of convolution kernels in the first convolutional layer and the second convolutional layer is 64, the size of the convolution kernels is 3×3, the stride and padding are both 1; the first activation function layer and the second activation function layer both use Leaky Relu, and their parameters are both 0.2; the pooling kernel size of the max pooling layer is 2×2, and the stride is 2.

[0073] S3. Use the training task set to perform iterative training on the network model in the pre-training stage, and use the cross-entropy loss function to update the network model in the pre-training stage to obtain a trained network model in the pre-training stage.

[0074] Please refer to Figure 2, Figure 2 It is a flowchart for implementing iterative training of a network model in the pre-training stage provided by an embodiment of the present invention. Step S3 specifically includes the following steps:

[0075] S31. Initialize and set the number of iterations in the pre-training stage, the maximum number of iterations in the pre-training stage, and the network model in the pre-training stage for the nth iteration.

[0076] Specifically, initialize the number of iterations in the pre-training stage as n, the maximum number of iterations in the pre-training stage as N, where N≥1000, and the network model of the SAR target pre-training stage for the nth iteration is and set n = 0.

[0077] S32. Use the first feature extraction module to extract features from each training task to obtain a set of first feature vector groups.

[0078] Specifically, use the training task set as the input of the network model H1 in the pre-training stage. The first feature extraction module Da extracts features from each training task to obtain a set of first feature vector groups

[0079] S33. Use the multi-layer perceptron module to map each feature in the set of feature vector groups to the sample label space to obtain a set of second feature vector groups.

[0080] Specifically, use the multi-layer perceptron module E to map each feature in the set of first feature vector groups to the sample label space to obtain a set of second feature vector groups

[0081] S34. Use the similarity calculation module to calculate the similarity between the feature vector corresponding to the training query sample in the set of second feature vector groups and each feature vector corresponding to the training support sample to obtain a set of similarity scores.

[0082] Specifically, use the similarity calculation module R to calculate the similarity scores between the feature vector corresponding to the training query sample in the set of second feature vector groups and each feature vector to obtain a set of similarity scores in

[0083] Divide each training support vector and training query vector in into m local descriptors. For example: and For and The formula for calculating the similarity using the similarity calculation module R is as follows:

[0084]

[0085]

[0086] Wherein, denotes the similarity score between and The feature vector obtained by the support sample successively passing through the first feature extraction module D and the multi-layer perceptron module E, denotes the query sample The feature vector obtained by passing through the feature extraction module D and the multi-layer perceptron module E, m represents the number of local descriptors into which the support feature vector is divided, z represents the number of local descriptors into which the query feature vector is divided, and x i denotes the i-th local descriptor in the support feature vector, denotes the j-th local descriptor in the query feature vector.

[0087] Finally, the similarity score set of all training support vectors and training query vectors is obtained Wherein, the training query set belongs to the training support set with the largest score in the similarity score set P.

[0088] S35. Calculate the first loss value of the network model in the pre-training stage during each iteration using the cross-entropy loss function, and use the gradient descent method to backpropagate the partial derivative of the first loss value with respect to the first weight parameter in the network model in the pre-training stage to update the first weight parameter to obtain the updated first weight parameter.

[0089] Specifically, use the cross-entropy loss function, and calculate the first loss value L m1 of the network model in the pre-training stage during each iteration through each predicted label y and its corresponding true label , then obtain the partial derivative s1 of the first loss value L s1 with respect to the first weight parameter ω s1 , and then use the gradient descent method to update the first weight parameter ω by backpropagating in the network model in the pre-training stage to obtain the updated first weight parameter. s1

[0090] The calculation formula of the first loss value L s1 is as follows: ​

[0091]

[0092] Among them, M represents the number of images corresponding to each category, and m1 represents and y m1 the m1-th true label and predicted label in represents the true label corresponding to the predicted label, y m1 represents the predicted label, and ln represents the natural logarithm with base e.

[0093] The updated first weight parameter is:

[0094]

[0095] Among them, represents the update result of the first weight parameter ω s1 ω s1 represents the first weight parameter, and L s1 represents the first loss value, and η represents the learning rate of L s1 represents the derivative operation.

[0096] S36. Determine whether the iteration is completed. If so, obtain the trained pre-training stage network model; if not, continue the iterative training.

[0097] Specifically, determine whether n ≥ N holds. If so, obtain the trained pre-training stage network model H1', otherwise, let n = n + 1 and execute step S32.

[0098] S4. Construct a self-supervised training stage network model.

[0099] ​Specifically, the constructed self-supervised training phase network model includes a second feature extraction module Db and a third feature extraction module G connected in parallel. The structures of the first feature extraction module Da, the second feature extraction module Db, and the third feature extraction module G are the same. That is, both the second feature extraction module Db and the third feature extraction module G include a first feature extraction sub-module D1 and a second feature extraction sub-module D2 connected in series. Among them, the first feature extraction sub-module D1 includes a first convolutional layer, a first batch normalization layer, a first activation function layer, and a max pooling layer connected in series. The second feature extraction sub-module D2 includes a second convolutional layer, a second batch normalization layer, and a second activation function layer connected in series. The multi-layer perceptron module E includes a fully connected layer. In the first feature extraction sub-module D1 and the second feature extraction sub-module D2, the number of convolution kernels in the first convolutional layer and the second convolutional layer is 64, the size of the convolution kernels is 3×3, and the stride and padding are both 1; the first activation function layer and the second activation function layer both use Leaky Relu, and their parameters are both 0.2; the size of the pooling kernel of the max pooling layer is 2×2, and the stride is 2.

[0100] S5. Load the parameters of the first feature extraction module in the trained pre-training phase network model into the self-supervised training phase network model, and use the training task set to iteratively train the loaded self-supervised training phase network model. During the training process, use the negative cosine similarity loss function and the self-supervised loss function to update the third feature extraction module to obtain a trained third feature extraction module.

[0101] Please refer to Figure 3 , Figure 3 FIG. is a flowchart for implementing iterative training of a self-supervised training phase network model provided by an embodiment of the present invention. Step S5 specifically includes the following steps:

[0102] S51. Initialize the iteration number of the self-supervised training phase, the maximum iteration number of the self-supervised training phase, and the self-supervised training phase network model at the nth iteration.

[0103] Specifically, initialize the iteration number of the self-supervised training phase as n, the maximum iteration number of the self-supervised training phase as N, N≥1000, and the self-supervised training phase network model at the nth iteration as and let n = 0,

[0104] S52. Load the parameters of the first feature extraction module in the trained pre-training stage network model into the second feature extraction module, and use the loaded second feature extraction module to extract features for each training task to obtain a set of third feature vector groups; load the parameters of the first feature extraction module in the trained pre-training stage network model into the third feature extraction module, and use the loaded second feature extraction module to extract features for each training task to obtain a set of fourth feature vector groups.

[0105] Specifically, take the training task set as the input of the self-supervised training stage network model . The second feature extraction module Db loads the parameters of the first feature extraction module Da in the trained pre-training stage network model H1', and then extracts features for each training task to obtain a set of feature vector groups . Among them, the second feature extraction module Db does not update its parameters during the training process; the third feature extraction module G loads the parameters of the first feature extraction module Da in the trained pre-training stage network model H1', and then extracts features for each training task to obtain a set of feature vector groups . The third feature extraction module G needs to update its parameters during the training process.

[0106] S53. Use the sum of the negative cosine similarity loss function and the self-supervised loss function as the training loss function to calculate the second loss value of the self-supervised training stage network model in each iteration process, and then use the partial derivative of the second loss value with respect to the second weight parameter to perform backpropagation in the self-supervised training stage network model to update the second weight parameter, obtaining the updated second weight parameter.

[0107] Specifically, use the negative cosine similarity loss function L cos and the self-supervised loss function L self-supervised The sum of them is used as the training loss function L to calculate the second loss value L of the self-supervised training stage network model s2 . Then, find the partial derivative of the second loss value L s2 with respect to the second weight parameter ω s2 . Then, use the gradient descent method to perform backpropagation in the self-supervised training stage network model by to update the second weight parameter ω . s2 Specifically, the training loss function is:

[0108] L = L

[0109] + L cos + Lself-supervised

[0110] Among them, L cos is the negative cosine similarity loss function, and L self-supervised is the self-supervised loss function.

[0111] The negative cosine similarity loss function is as follows:

[0112]

[0113] Among them, D represents the second feature extraction module, G represents the third feature extraction module, and ||·||2 represents l2 regularization.

[0114] The calculation process of the self-supervised loss function is as follows: The training task set is replicated to obtain two task sets. After performing three data augmentation methods of random cropping, random horizontal flipping, and Gaussian filtering on these two task sets respectively, two different augmentation effects and After X1 and X2 pass through an f module, they respectively obtain and After X1 and X2 pass through an h module, they respectively obtain and Among them, the f module includes a feature extraction module G and a feature mapping module O1 in sequential cascade; the h module only includes a feature mapping module O2; the feature mapping module O1 includes a fully connected layer, a normalization layer, an activation function layer, a fully connected layer, a normalization layer, an activation function layer, a fully connected layer, and a normalization layer in sequential cascade; the feature mapping module O2 includes a fully connected layer, a normalization layer, an activation function layer, and a fully connected layer in sequential cascade. Thus, the self-supervised loss function L self-supervised has the following calculation formula:

[0115] L self-supervised (x) = U(P1, Z2) + U(P2, Z1)

[0116]

[0117]

[0118] Among them, P1 represents the feature vector group obtained by X1 passing through an h module, P2 represents the feature vector group obtained by X2 passing through an h module, X1 and X2 respectively represent the two effect diagrams obtained after the training task set undergoes data augmentation, Z1 represents the feature vector group obtained by X1 passing through an f module, and Z2 represents the feature vector group obtained by X2 passing through an f module X2.

[0119] S54. Determine whether the iteration is completed. If so, obtain the trained network model in the self-supervised training stage; if not, continue the iterative training.

[0120] Specifically, determine whether n≥N holds. If so, obtain the trained network model H'2 in the self-supervised training stage. Otherwise, set n=n + 1 and execute step S52.

[0121] S6. Input the test task set into the trained third feature extraction module for feature extraction to obtain a set of test feature vector groups.

[0122] Please refer to Figure 4 , Figure 4 which is a flowchart for implementing the acquisition of the target classification result of small-sample SAR images provided by an embodiment of the present invention.

[0123] Specifically, use the test task set as the input of the trained network model H'2 in the self-supervised training stage for forward propagation to obtain the predicted labels of all test task sets It can be understood that the trained third feature extraction module G is used to perform feature extraction on each SAR image included in each test task in the test task set to obtain a set of test feature vector groups

[0124]

[0125]

[0126] where z2 = C test (M - K), z3 = C test K + 1, represents the test feature vector group corresponding to the test task set The one that satisfies e≠z3 represents the test feature vector corresponding to the test support sample and represents the test feature vector corresponding to the test query sample

[0127] S7. Calculate the similarity between the test feature vector corresponding to the test query sample in the set of test feature vector groups and the test feature vector corresponding to each test support sample to obtain the target classification result.

[0128] Specifically, the test feature vector corresponding to the test query sample and ​Each element in is calculated for the similarity score by the similarity calculation module R. By this method, the set of test prediction result vectors is finally obtained. where z2 = C test (M - K), each test prediction result vector The dimension number corresponding to the maximum value in is the corresponding test query sample including the predicted category of the target in the SAR image, so as to obtain the target classification result.

[0129] Furthermore, in this embodiment, combined with the simulation experiment, the technical effect of the small-sample SAR target classification method based on self-supervised learning and the nearest neighbor network is further described:

[0130] 1. Simulation experiment conditions and content:

[0131] The hardware platform for the simulation experiment is: the GPU is NVIDIA GeForce RTX 1650, and the software platform is: the operating system is Ubuntu18.04. The dataset for the simulation experiment is the publicly available AID dataset. Among them, C = 30, and the categories are BareLand, Beach, Desert, Meadow, Mountain, Parking, Port, RailwayStation, School, StorageTanks, Airport, BaseballField, Bridge, Center, Church, Commercial, DenseResidential, Farmland, Forest, Industrial, MediumResidential, Park, Playground, Pond, Resort, River, SparseResidential, Square, Stadium, Viaduct. The SAR image of each type of target is 290, that is, M = 290.

[0132] In order to compare the small-sample SAR target classification accuracy with the existing small-sample SAR automatic target recognition method based on the hybrid inference network, a total of 5800 SAR images of 20 target categories and the label of each SAR image are selected from the AID dataset as the training sample set, that is, C train= 20, with categories being Airport, BaseballField, Bridge, Center, Church, Commercial, DenseResidential, Farmland, Forest, Industrial, MediumResidential, Park, Playground, Pond, Resort, River, SparseResidential, Square, Stadium, Viaduct; 2900 SAR images of the remaining 10 target categories and the labels of each SAR image are selected as the test sample set, C test = 10, with categories being BareLand, Beach, Desert, Meadow, Mountain, Parking, Port, RailwayStation, School, StorageTanks. At the same time, the number of training / test support samples K sampled for each target category in each training / test task is 10, and the number of training / test query samples M - K is 280.

[0133] The average accuracies of the method of this embodiment and the existing small-sample SAR automatic target recognition method based on a hybrid inference network (i.e., the comparison algorithm) are simulated and compared in two modes of 5-way 1-shot and 5-way 5-shot, and the results are shown in Table 1:

[0134] Table 1

[0135] N-way K-shot 5-way 1-shot 5-way 5-shot Comparison algorithm 60.62％±0.35 68.85％±0.28 The present invention 67.94％±0.29 75.36％±0.21

[0136] It can be seen from Table 1 that, compared with the comparison algorithm, the method of this embodiment has an average accuracy improvement of 7.32% and 6.51% respectively in the two modes of 5-way 1-shot and 5-way 5-shot.

[0137] In summary, the classification method of this embodiment introduces a network model in the pre-training stage. By loading the parameters of the trained network model in the pre-training stage into the network model in the self-supervised training stage, the convergence of the network model in the self-supervised training stage can be accelerated and the generalization ability of the model can be improved; when training the network model in the self-supervised training stage, a self-supervised loss function and a negative cosine similarity loss function are introduced to obtain better feature representations, reduce feature representation biases, and an efficient nearest neighbor metric method is introduced to calculate the similarity between support samples and query samples. Thus, this method avoids the influence on the SAR target classification accuracy caused by the slow model convergence speed, weak model generalization ability, inability to obtain better feature representations, and inefficient metric methods in the prior art, and effectively improves the classification accuracy of small-sample targets.

[0138] The above content is a further detailed description of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those of ordinary skill in the technical field to which the present invention pertains, without departing from the concept of the present invention, several simple deductions or substitutions can still be made, and all should be regarded as falling within the protection scope of the present invention.

Claims

1. A few-shot SAR target classification method based on self-supervised learning and nearest neighbor network, characterized in that, Including the steps: S1. Obtain a training task set and a test task set from a number of synthetic aperture radar images. Each training task in the training task set includes a training support sample and a training query sample, and each test task in the test task set includes a test support sample set and a test query sample; S2. Construct a pre-training phase network model, where the pre-training phase network model includes a first feature extraction module, a multi-layer perceptron module, and a similarity calculation module cascaded in sequence; S3. Iteratively train the pre-training phase network model using the training task set, and update the pre-training phase network model using the cross-entropy loss function to obtain a trained pre-training phase network model; S4. Construct a self-supervised training phase network model, where the self-supervised training phase network model includes a second feature extraction module and a third feature extraction module connected in parallel, and the structures of the first feature extraction module, the second feature extraction module, and the third feature extraction module are the same; S5. Load the parameters of the first feature extraction module in the trained pre-training phase network model into the self-supervised training phase network model, and iteratively train the loaded self-supervised training phase network model using the training task set. During the training process, update the third feature extraction module using the negative cosine similarity loss function and the self-supervised loss function to obtain a trained third feature extraction module; S6. Input the test task set into the trained third feature extraction module for feature extraction to obtain a set of test feature vector groups; S7. Calculate the similarity between the test feature vector corresponding to the test query sample and the test feature vectors corresponding to each of the test support samples in the set of test feature vector groups to obtain a target classification result.

2. The few-shot SAR target classification method based on self-supervised learning and nearest neighbor network according to claim 1, characterized in that, The first feature extraction module includes a first feature extraction sub-module and a second feature extraction sub-module cascaded in sequence, where the first feature extraction sub-module includes a first convolutional layer, a first batch normalization layer, a first activation function layer, and a max pooling layer cascaded in sequence; the second feature extraction sub-module includes a second convolutional layer, a second batch normalization layer, and a second activation function layer cascaded in sequence; the multi-layer perceptron module includes a fully connected layer.

3. The few-shot SAR target classification method based on self-supervised learning and nearest neighbor network according to claim 2, characterized in that, The number of convolution kernels in both the first convolutional layer and the second convolutional layer is 64, the size of the convolution kernels is 3×3, the stride and padding are both 1; both the first activation function layer and the second activation function layer use Leaky Relu, and their parameters are both 0.2; the pooling kernel size of the max pooling layer is 2×2, and the stride is 2.

4. The few-shot SAR target classification method based on self-supervised learning and nearest neighbor network according to claim 2, wherein Step S3 includes: S31. Initialize the number of iterations in the pre-training phase, the maximum number of iterations in the pre-training phase, and the pre-training phase network model at the nth iteration; S32. Use the first feature extraction module to perform feature extraction on each training task to obtain a set of first feature vector groups; S33. Use the multi-layer perceptron module to map each feature in the set of feature vector groups to the sample label space to obtain a set of second feature vector groups; S34. Calculate the similarity between the feature vector corresponding to the training query sample in the second feature vector group set and the feature vector corresponding to each training support sample by using the similarity calculation module, and obtain a similarity score set; S35. Calculate the first loss value of the network model in the pre-training stage during each iteration by using the cross-entropy loss function, and perform backpropagation of the partial derivative of the first loss value with respect to the first weight parameter in the network model in the pre-training stage by using the gradient descent method to update the first weight parameter and obtain the updated first weight parameter; S36. Determine whether the iteration is completed. If so, obtain the trained network model in the pre-training stage; if not, continue the iterative training.

5. The few-shot SAR target classification method based on self-supervised learning and nearest neighbor network according to claim 4, wherein The formula for calculating the similarity by using the similarity calculation module is: Among them, represents the similarity score between the support sample and the feature vector obtained by successively passing through the first feature extraction module D and the multi-layer perceptron module E, represents the feature vector obtained by passing the query sample through the feature extraction module D and the multi-layer perceptron module E, m represents the number of local descriptors into which the support feature vector is divided, z represents the number of local descriptors into which the query feature vector is divided, x represents the i-th local descriptor in the support feature vector, represents the j-th local descriptor in the query feature vector.

6. The few-shot SAR target classification method based on self-supervised learning and nearest neighbor network according to claim 4, characterized in that, The first loss value is: where M represents the number of images corresponding to each category, and m1 represents and y m1 the m1-th true label and predicted label in represents the true label corresponding to the predicted label, y m1 represents the predicted label, and ln represents the natural logarithm with base e; The updated first weight parameter is: Among them, represents the updated result of the first weight parameter ω s1 , ω s1 represents the first weight parameter, and L s1 represents the first loss value, and η represents the learning rate of L s1 and represents the derivative operation.

7. The small-sample SAR target classification method based on self-supervised learning and nearest neighbor network according to claim 2, characterized in that The second feature extraction module includes a third convolutional layer, a third batch normalization layer, a third activation function layer, and a second max pooling layer cascaded in sequence; The third feature extraction module includes a fourth convolutional layer, a fourth batch normalization layer, a fourth activation function layer, and a third max pooling layer cascaded in sequence.

8. The few-shot SAR target classification method based on self-supervised learning and nearest neighbor network according to claim 7, characterized in that, Step S5 includes: S51. Initialize and set the number of iterations in the self-supervised training stage, the maximum number of iterations in the self-supervised training stage, and the network model in the self-supervised training stage at the nth iteration; S52. Load the parameters of the first feature extraction module in the trained network model in the pre-training stage into the second feature extraction module, and use the loaded second feature extraction module to extract features from each training task to obtain a third feature vector group set; load the parameters of the first feature extraction module in the trained network model in the pre-training stage into the third feature extraction module, and use the loaded second feature extraction module to extract features from each training task to obtain a fourth feature vector group set; S53. Use the sum of the negative cosine similarity loss function and the self-supervised loss function as the training loss function to calculate the second loss value of the network model in the self-supervised training stage during each iteration, and then perform backpropagation of the partial derivative of the second loss value with respect to the second weight parameter in the network model in the self-supervised training stage to update the second weight parameter and obtain the updated second weight parameter; S54. Determine whether the iteration is completed. If so, obtain the trained network model in the self-supervised training stage; if not, continue the iterative training.

9. The few-shot SAR target classification method based on self-supervised learning and nearest neighbor network according to claim 8, wherein The training loss function is: L = L cos + L self-supervised Among them, L cos is the negative cosine similarity loss function, and L self-supervised is the self-supervised loss function; The negative cosine similarity loss function is: where D represents the second feature extraction module, G represents the third feature extraction module, and ||·||2 represents l2 regularization; The self-supervised loss function is: L self-supervised L(x) = U(P1, Z2) + U(P2, Z1) Among them, P1 represents the feature vector group obtained by X1 passing through an h module, P2 represents the feature vector group obtained by X2 passing through an h module, X1 and X2 respectively represent two effect diagrams obtained after data augmentation of the training task set, Z1 represents the feature vector group obtained by X1 passing through an f module, and Z2 represents the feature vector group X2 obtained by X2 passing through an f module.

Images