SAR Target Recognition Method, Device and Computer Equipment Based on Small Samples

Abstract

The present application relates to a small-sample-based SAR target recognition method, device, and computer equipment. By using a trained feature extractor to extract features from the SAR image to be target-recognized and mapping them into the Grassmann manifold space, classification is performed through multiple subspaces corresponding to known target categories in this space, so as to achieve the purpose of recognizing targets in the SAR image. When the target category in the image is an unknown category, it is input into the target recognition network in real time for online small-sample training. During the training process, the parameters of the feature extractor and the Grassmann manifold space are adjusted until convergence by using the semantic margin separation loss function, the unstructured forgetting loss function, and the deep subspace separation loss function respectively, so that it has the ability to recognize this unknown target category. By adopting this method, continuous observation can be carried out in an open non-cooperative scenario, new target types can be learned using small-sample data, and the accuracy of category prediction can be effectively improved.

Description

Technical Field

[0001] The present application relates to the technical field of SAR target recognition, and particularly to a SAR target recognition method, device and computer device based on few samples. Background Art

[0002] Synthetic Aperture Radar (SAR) has excellent and stable imaging capabilities under almost all weather and lighting conditions, and plays an indispensable role in various applications. As a basic and challenging task of SAR image interpretation, SAR Automatic Target Recognition (SAR ATR) has received long-term and extensive attention for decades. Especially driven by Deep Learning (DL) technology, SAR ATR has reached an unprecedented prosperity.

[0003] However, in the prior art, when observing targets of new categories, in most non-cooperative situations, due to the limitation of imaging conditions, the target of interest may contain few shot instances with little information clues. Therefore, the current DL-based SAR ATR methods urgently need to promote the ability of Few-Shot Class Incremental Learning (FSCIL) to meet the actual scenarios. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a SAR target recognition method, device and computer device based on few samples that can perform online learning on unknown category targets of few samples.

[0005] A SAR target recognition method based on few samples, the method includes:

[0006] Obtain a SAR image to be subject to target recognition;

[0007] Input the SAR image into a trained target recognition network, in the target recognition network, a feature extractor extracts features of the SAR image, maps the extracted features into a Grassmann manifold space, and classifies the extracted features through subspaces corresponding to various known category targets included in this space;

[0008] If the target in the SAR image is an unknown category, use the SAR image as few-shot training data to perform online iterative training on the target recognition network. During the online iterative training process, also adjust the parameters in the feature extractor by using a semantic margin separation loss function and an unstructured forgetting loss function, and adjust the parameters of the Grassmann manifold space by using a deep subspace separation loss function until each loss function converges, then obtain a target recognition network with the ability to recognize the unknown category target;

[0009] If the target in the SAR image is of a known category, the predicted result of the target is output to realize the recognition of the target in the SAR image.

[0010] In one embodiment, after the target recognition network is online trained using a SAR image with an unknown target category, the Grassmann manifold space in the target recognition network contains a subspace corresponding to the unknown category.

[0011] In one embodiment, the semantic margin separation loss function is constructed based on the features extracted from the SAR images of known-category targets and unknown-category targets by the feature extractor respectively, and is specifically expressed as:

[0012]

[0013] Wherein,

[0014] In the above formula, E t-1 represents the old-category exemplars stored in the t-th learning stage, represents the set of new-category data used for training in the current stage, x a represents the anchor sample selected from , x i and x j are respectively the positive sample selected from and the negative sample selected from E t-1 , d(x a , x i ) represents the cosine distance between the anchor and the positive sample, d(x a , x j ) represents the cosine distance between the anchor and the negative sample, f φ represents the trainable feature extractor.

[0015] In one embodiment, the unstructured forgetting loss function is constructed based on the features extracted by the feature extractor in the target recognition network before the previous online iterative training and the features extracted by the feature extractor in the current target recognition network, and is specifically expressed as:

[0016]

[0017] Wherein,

[0018] In the above formula, and are respectively defined as the spatial and global loss constraints, s t-1 (x i ) and s t (x irespectively represent the spatial features output by the feature extractors in the t-th and (t - 1)-th stages for the sample x i The spatial features output by the feature extractors in the t-th and (t - 1)-th stages for the sample x and represents the output global feature value, where, in the t-th stage, the sample x i is selected from the current set of all training samples

[0019] In one embodiment, the deep subspace separation loss function is constructed according to the subspace corresponding to the newly established unknown class target and the subspaces corresponding to all known class targets in the Grassmann manifold space, and is specifically expressed as:

[0020]

[0021] where

[0022] In the above formula represents the new class set in the t-th stage, represents the historical class set, d P (Y j , Y k ) calculates the manifold space distance between two subspaces Y j and Y k , where Y k represents the subspace corresponding to the known class targets that already exist in the Grassmann manifold space, and Y j represents the subspace corresponding to the newly established unknown class targets in the Grassmann manifold space.

[0023] In one embodiment, the feature extractor uses a CNN neural network.

[0024] In one embodiment, the SAR target recognition method based on small samples further includes training the target recognition network before obtaining the SAR image to be recognized for target recognition, including:

[0025] Obtain an SAR training data set, where the SAR training data set includes a set of SAR training images of multiple known class targets, and each set of training images corresponding to a known class target includes multiple SAR training images;

[0026] Construct a base class training data set for the base class training stage and multiple small sample training data sets for the small sample incremental learning stage according to the SAR training data set;

[0027] In the base class training stage, the base class training dataset is input into the target recognition network for iterative training, and the cosine cross-entropy loss function is used to adjust the parameters of the feature extractor in the target recognition network in each iterative training until convergence, then a target recognition network with the ability to recognize base class targets is obtained, and at this time, the Grassmann manifold space in the target recognition network contains subspaces corresponding to each base class target;

[0028] Samples are selected from the base class training dataset according to the azimuth angle perception example selection strategy, and the selected samples are used to construct a historical sample dataset;

[0029] In the few-shot incremental learning stage, multiple few-shot training datasets are sequentially input into the target recognition network with the ability to recognize base class targets for training;

[0030] Before using the first few-shot training dataset to train the target recognition network with the ability to recognize base class targets, the few-shot training dataset is also merged with the historical sample dataset, and the merged training dataset is input into the target recognition network with the ability to recognize base class targets until the target recognition network converges, and then the next few-shot training dataset is used to train the target recognition network until all few-shot training datasets have been used to train the target recognition network, and a trained target recognition network is obtained.

[0031] In one embodiment, in the few-shot incremental learning stage, the semantic margin separation loss function, the unstructured forgetting loss function, and the deep subspace separation loss function are also used to train the target recognition network.

[0032] A few-shot based SAR target recognition device, the device includes:

[0033] An SAR image acquisition module, configured to acquire an SAR image to be subjected to target recognition;

[0034] A target recognition network prediction module, configured to input the SAR image into the trained target recognition network, extract the features of the SAR image by a feature extractor in the target recognition network, map the extracted features into the Grassmann manifold space, and classify the extracted features through the subspaces corresponding to various known category targets included in this space;

[0035] An online learning module for unknown-class targets, which is used to, if the target in the SAR image is of an unknown class, use the SAR image as small-sample training data to perform online iterative training on the target recognition network. During the online iterative training process, the parameters in the feature extractor are also adjusted by using a semantic margin separation loss function and an unstructured forgetting loss function, and the parameters in the Grassmann manifold space are adjusted by using a deep subspace separation loss function. Until each loss function converges, a target recognition network with the ability to recognize the unknown-class target is obtained;

[0036] A known-class target recognition module, which is used to, if the target in the SAR image is of a known class, output the prediction result of the target to achieve the recognition of the target in the SAR image.

[0037] A computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the following steps are implemented:

[0038] Obtain a SAR image to be subjected to target recognition;

[0039] Input the SAR image into the trained target recognition network. In the target recognition network, the feature extractor extracts the features of the SAR image, maps the extracted features into the Grassmann manifold space, and classifies the extracted features through the subspaces corresponding to various known-class targets in this space;

[0040] If the target in the SAR image is of an unknown class, use the SAR image as small-sample training data to perform online iterative training on the target recognition network. During the online iterative training process, the parameters in the feature extractor are also adjusted by using a semantic margin separation loss function and an unstructured forgetting loss function, and the parameters in the Grassmann manifold space are adjusted by using a deep subspace separation loss function. Until each loss function converges, a target recognition network with the ability to recognize the unknown-class target is obtained;

[0041] If the target in the SAR image is of a known class, output the prediction result of the target to achieve the recognition of the target in the SAR image.

[0042] A computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0043] Obtain a SAR image to be subjected to target recognition;

[0044] Input the SAR image into the trained target recognition network. In the target recognition network, a feature extractor extracts the features of the SAR image, maps the extracted features into the Grassmann manifold space, and classifies the extracted features through the subspaces corresponding to various known category targets contained in this space;

[0045] If the target in the SAR image is of an unknown category, use the SAR image as small-sample training data to perform online iterative training on the target recognition network. During the online iterative training process, also adjust the parameters in the feature extractor by using the semantic margin separation loss function and the unstructured forgetting loss function, and adjust the parameters of the Grassmann manifold space by using the deep subspace separation loss function until each loss function converges, then obtain a target recognition network capable of recognizing this unknown category target;

[0046] If the target in the SAR image is of a known category, output the prediction result of the target to achieve the recognition of the target in the SAR image.

[0047] The above-mentioned small-sample-based SAR target recognition method, device and computer equipment extract the features of the SAR image to be target-recognized by using the trained feature extractor, map the extracted features into the Grassmann manifold space, and classify the features through multiple pre-existing subspaces corresponding to known target categories in this space, so as to achieve the purpose of recognizing the target in the SAR image. When the target category in the image is an unknown category, also input it into the target recognition network in real time for online small-sample training. During the training process, adjust the parameters of the feature extractor and the Grassmann manifold space respectively by using the semantic margin separation loss function, the unstructured forgetting loss function and the deep subspace separation loss function until convergence, so that it has the ability to recognize this unknown target category. By using this method, continuous observation can be carried out in an open non-cooperative scenario, new target types can be learned by using small-sample data, and the accuracy of category prediction can be effectively improved. Description of the Drawings

[0048] Figure 1 It is a schematic flowchart of the small-sample-based SAR target recognition method in an embodiment;

[0049] Figure 2 It is a schematic flowchart of the target recognition network training method in an embodiment;

[0050] Figure 3 It is a schematic overall framework diagram of the target recognition network during training in an embodiment;

[0051] Figure 4Schematic diagram of semantic margin separation loss in an embodiment;

[0052] Figure 5 Schematic diagram of structural forgetting constraint loss in an embodiment;

[0053] Figure 6 Block diagram of a small - sample - based SAR target recognition device in an embodiment;

[0054] Figure 7 Internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0055] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

[0056] In the prior art, implementing SAR automatic target recognition based on deep learning technology is an important technical means. However, in the actual application scenario of SAR target recognition, in the non - cooperative case, due to the limitations of SAR imaging conditions, the samples for deep learning training are very few, and the effective information contained in the targets of interest in the samples is also less. Therefore, it is particularly important to apply small - sample class incremental learning in this scenario.

[0057] To address the above problems, as Figure 1 shown, the small - sample - based SAR target recognition method provided by the present application includes the following steps:

[0058] Step S100, obtaining an SAR image to be recognized for the target;

[0059] Step S110, inputting the SAR image into a trained target recognition network. In the target recognition network, a feature extractor extracts the features of the SAR image, maps the extracted features into the Grassmann manifold space, and classifies the extracted features through the sub - spaces corresponding to various known - class targets contained in this space;

[0060] Step S120, if the target in the SAR image is an unknown class, using the SAR image as small - sample training data to perform online iterative training on the target recognition network. During the online iterative training process, the parameters in the feature extractor are also adjusted by using a semantic margin separation loss function and a structure - less forgetting loss function, and the parameters of the Grassmann manifold space are adjusted by using a deep subspace separation loss function until each loss function converges, then a target recognition network capable of recognizing the unknown - class target is obtained;

[0061] Step S130: If the target in the SAR image is a known category, output the prediction result of the target to achieve the recognition of the target in the SAR image.

[0062] In this embodiment, a SAR target recognition method for online knowledge learning of small-sample training data is provided. First, the trained target recognition network is used to process the SAR image to be recognized. Among them, the feature extractor is used to extract the features of the SAR image, and then the extracted features are projected into the Grassmann manifold space. By calculating the distances between the features and the subspaces of each known target category in the space, the classification is performed. For the case where the target category in the SAR image to be recognized is an unknown category, in this method, the SAR image of the unknown category target is used to perform real-time online training on the target recognition network. During the training process, the semantic margin separation loss function (SMS), the deep subspace separation loss function (DSS), and the structural forgetting constraint loss function (SLF) are used to constrain the stability of the target recognition network inside the network, the orthogonality of the subspaces, and the semantic drift of the target space and the global structure, so that the target recognition network has the ability to learn unknown target categories using small-sample data.

[0063] In this embodiment, the Grassmann manifold space in the trained target recognition network already contains the subspaces of multiple known category targets. When the target in the input SAR image is a known category, its category can be accurately predicted. When the target in the input SAR image is an unknown category, that is, there is no corresponding subspace for the category of this target in the Grassmann manifold space, through real-time online learning, a subspace corresponding to the unknown category is constructed in the Grassmann manifold space, so that the target recognition network has the ability to recognize the unknown category. That is, after using the SAR image with an unknown target category to perform online training on the target recognition network, the Grassmann manifold space in the target recognition network contains the subspace corresponding to the unknown category.

[0064] In this embodiment, the category prediction of the extracted features through the Grassmann manifold space is actually to find the subspace with the closest projection distance as the target category prediction result of the SAR image by calculating the projection distances between the extracted features and the subspaces in the space, such as Euclidean and cosine or linear classifiers.

[0065] In this embodiment, the method steps shown in the above steps S100 to S130 are actually the process of using the trained target recognition network to recognize targets in SAR images. The training method that enables the network to have target recognition capabilities and few-shot learning capabilities is also a particularly important part of this method. Since the loss function applied during the real-time online training of the target recognition network is also used during its training, in order to avoid repeated explanations, the loss function will be described when the training method is elaborated later.

[0066] In this embodiment, the SAR target recognition method based on few-shot also includes training the target recognition network before obtaining the SAR image to be recognized for target, as Figure 2 described, including:

[0067] Step S200, obtain an SAR training data set, which includes multiple SAR training image sets of known-class targets, and each SAR training image set corresponding to a known-class target includes multiple SAR training images;

[0068] Step S210, construct a base-class training data set for the base-class training stage and multiple few-shot training data sets for the few-shot incremental learning stage according to the SAR training data set;

[0069] Step S220, in the base-class training stage, input the base-class training data set into the target recognition network for iterative training, and use the cosine cross-entropy loss function to adjust the parameters of the feature extractor in the target recognition network in each iterative training until convergence, then obtain a target recognition network with the ability to recognize base-class targets, and at this time, the Grassmann manifold space in the target recognition network contains subspaces corresponding to each base-class target;

[0070] Step S230, select samples in the base-class training data set according to the azimuth awareness example selection strategy, and construct a historical sample data set with the selected samples;

[0071] Step S240, in the few-shot incremental learning stage, input multiple few-shot training data sets into the target recognition network with the ability to recognize base-class targets for training;

[0072] Step S250. Before training the target recognition network capable of recognizing base-class targets using the first small-sample training dataset, the small-sample training dataset is also merged with the historical sample dataset, and the merged training dataset is input into the target recognition network capable of recognizing base-class targets until the target recognition network converges. Then, the next small-sample training dataset is used to train the target recognition network until all small-sample training datasets have been used to train the target recognition network, and the trained target recognition network is obtained.

[0073] In this embodiment, in fact, few-shot class-incremental learning (FSCIL) is used to train the target recognition network so that it can continuously learn new knowledge from a small number of samples and will not forget the knowledge learned before. To mimic the actual learning scenario, in few-shot incremental learning, it includes two training phases. One is the base-class training phase to enable the network to have the ability to recognize known target classes, and the other is the few-shot incremental learning phase to enable the network to have the ability to learn new knowledge using small samples.

[0074] Specifically, in the base-class training phase, large-scale training data is used to mimic offline learning in a cooperative situation. The feature extractor is trained through the cosine cross-entropy (CE) loss to establish a semantic hypersphere to capture the target structure details. Then, the subspace of the base class is saved in the Grassmann manifold space for subsequent recognition of known targets.

[0075] Specifically, in the few-shot incremental training phase, only a small number of samples are provided to simulate the online learning process. The loss function covers the projection and semantic levels. Among them, the semantic margin separation loss function (SMS), the deep subspace separation loss function (DSS), and the structureless forgetting loss function (SLF) are used to alleviate the inherent dilemmas and domain-specific challenges of the network model. Among them, for the plasticity of the network model, the SMS loss aims to expand the difference in target semantic features around the decision boundary to achieve a clear separation between new and old classes. The DSS loss considers the projection metric derived from the principal angles of the Grassmann manifold and introduces making the subspaces as orthogonal as possible for knowledge separation. For the stability of the model, the knowledge of the old classes is retained by constraining the semantic drift of the space and the global target structure, and these semantic drifts are mapped by the previous fixed model and the current trainable model in the SLF loss.

[0076] In this embodiment, considering the significant periodicity and volatility of the target backscattering characteristics with the change of azimuth angle, an azimuth angle-aware sample selection (AES) strategy is proposed to select complementary and representative samples to effectively preserve the knowledge of the old classes. This strategy is applied after the target recognition network completes the base class training phase. In the large-scale training data, representative sample images are selected using this strategy. In the few-shot incremental training phase, the selected sample images are merged with a small number of samples and then used to train the target network, so that it will not forget the knowledge learned before while learning new knowledge.

[0077] Before training the target recognition network, it is also necessary to construct the training data.

[0078] In step S200, first, a large number of labeled datasets D 1 , D 2 ,..., D t are prepared, where contains M pairs of samples x i and the corresponding labels y i . Each dataset D 1 , D 2 ,..., D t are the sample data of different target classes respectively.

[0079] Then, in step S210, it is set that when always holds, the first dataset D 1 represents the base session data, which provides a large number of base class training samples, including multiple target classes, and each class includes multiple sample images. And the dataset is the incremental session data, which contains a small number of samples of some new classes, that is, N classes, with K data for each class, and the values of N and K are very small.

[0080] In each training i, the network is parameterized by a CNN-based backbone network η and a decoupled classifier φ, and is optimized on the dataset D i and some old class samples (i.e., E1, E2,..., E i-1 ) that may be retained in the previous training. In the interference phase of training i, the model needs to be able to distinguish all visible classes including the current session in the previous sessions. Few-shot class incremental learning is a practical and challenging problem, which requires to balance the model stability and plasticity by retaining the knowledge of the old classes and integrating the knowledge of the new classes.

[0081] As Figure 3 shown, it is a schematic diagram of the overall framework for training the target recognition network.

[0082] In step S220, during the base class training phase (t = 1), first train a CNN-based feature extractor f φ Classify the base classes and extract the generalization features of the new classes during subsequent few-shot incremental training. Considering the accuracy of the network in representing the target orientation perception structure in the cosine feature space, use the cosine cross-entropy loss to optimize the feature extractor in the target recognition network during this phase until it converges to complete the training of the base class training phase.

[0083] Then, in step S230, in the example set E of the base classes 1 Select through the proposed azimuth angle perception example selection (AES) strategy for knowledge review during subsequent incremental learning. At the same time, use the 1 class samples in E to construct subspaces representing the known class targets in the Grassmann manifold, that is

[0084] Next, in step S240, during the few-shot incremental learning phase (t > 1), taking the second (t = 2) as an example, first combine the current new class data D 2 with the previously saved old class example set E 1 to form the training data set required for this learning phase. To balance the plasticity of the model's adaptation to new knowledge and the stability of preserving old knowledge, semantic boundary separation (SMS), structured forgetting constraint (SLF), and deep subspace separation (DSS) losses covering the original semantic space and the Grassmann manifold are designed. First, to constrain the plasticity of the model, use the SMS loss to expand the distance between the current class and other classes based on the cosine distance.

[0085] In particular, considering the limited knowledge of the saved samples and the new class samples, only consider the semantic boundaries of the old and new class targets with similar azimuth perception structures to avoid inappropriate knowledge transfer. The DSS loss is based on the projection metric theory in Grassmann geometry to constrain the orthogonality of the constructed subspaces. To alleviate the model's forgetting of old knowledge, use the knowledge distillation technique in the SLF loss to constrain the drift of the spatial and global target features generated by the model obtained in the previous learning phase and the current trainable model, thereby maintaining the consistency of the model's output signals for the detailed and global structures of the old class targets in different phases.

[0086] In this embodiment, not only after the completion of the base class training phase, the AES strategy is adopted to select sample data from the training dataset for the next phase of training, but also in multiple few-shot incremental training phases, after each training is completed, this strategy is used to select sample data from the training set of this time, merge it with the next few-shot incremental training set and then perform training. Each time a few-shot sample uses the AES strategy to select complementary and representative examples of new classes after balancing the feature-level correlation and energy for effective knowledge review and subspace construction.

[0087] In this embodiment, in the few-shot incremental training phase, each few-shot incremental training set is used to iteratively train the target network model until convergence, and then the next few-shot incremental training set is used to train the network until all the training sets are trained, and then the training of the target recognition network is completed.

[0088] In Figure 3 the overall training framework shown, it also includes a part for testing the trained target recognition network.

[0089] In this embodiment, the algorithm process for training the target recognition network is as shown in Algorithm 1 as follows:

[0090]

[0091] Next, three loss functions used in the few-shot incremental training phase and when online training is performed using the trained target recognition network are introduced.

[0092] In this embodiment, in order to release the plasticity of the model, the obvious separation between new and old classes in the semantic feature space should be particularly considered. In particular, due to the scarcity of recognition target information provided by few-shot samples, the target features of new classes of the feature extraction network trained by the cross-entropy (CE) loss are prone to deviation, with weak generalization ability, and will also cause serious distortion in the feature space of learning classes. Considering the close relationship between the phase perception of semantic features and the target orientation perception structure under the cosine criterion, semantic margin separation (SMS) loss is proposed in this method to adaptively learn new knowledge based on specific sample pairs.

[0093] Specifically, the semantic margin separation loss function is constructed based on the features obtained by the feature extractor extracting the SAR images of known-class targets and unknown-class targets respectively, and is specifically expressed as:

[0094]

[0095] Among them,

[0096] In formula (1), E t-1 represents the old-class exemplars that have been stored in the t-th learning stage, represents the set of new-class data used for training in the current stage, x a represents the anchor sample selected from , x i and x j are respectively the positive sample selected from and the negative sample selected from E t-1 . d(x a , x i ) represents the cosine distance between the anchor and the positive sample, and d(x a , x j ) represents the cosine distance between the anchor and the negative sample. f φ represents a trainable feature extractor.

[0097] Among them, the cosine distance between x a and x i can be calculated by formula (2), and the distance between x a and x j is the closest among all heterogeneous samples. m is a coefficient that controls the separation between positive and negative pairs. The larger the value of m, the more obvious the difference between the new and old classes. The SMS loss can be expressed as Figure 4 . The model trained by the SMS loss can learn new concepts while avoiding unnecessary parameter updates.

[0098] In this embodiment, according to the definition of the principal angle in the Grassmann manifold space, the cosine-based principal angle cosθ k is calculated based on the nearest normalized similarity of two basis vectors from different subspaces, which implicitly expresses the correlation relationship of the key features of heterogeneous subspaces. The smaller this value, the more obvious the difference between classes. Therefore, by reducing the cosine similarity between the bases corresponding to heterogeneous subspaces, the separability of heterogeneous classes can be improved, thereby enhancing the discriminative ability of the model for classes. For this purpose, based on the projection metric theory in Grassmann geometry, we have improved the Deep Subspace Separation Loss (DSS). Generally speaking, the distance d j between two subspaces Y k and Y P on the Grassmann manifold j , Y k) can be defined as formula (3), which is the L2 norm of the protagonist's sine. Further, it can be formalized as formula (4), representing the matrix norm of the class subspace in the Grassmann manifold space. Therefore, expanding the distance of the heterogeneous subspace is equivalent to minimizing the second term in formula (4). Based on the above theory, the DSS loss designed by us is shown in formula (5), which fully considers the orthogonality constraint between the new class and all historical classes to enhance the inter-class difference.

[0099] Specifically, the deep subspace separation loss function is constructed according to the subspace corresponding to the newly established unknown class target and the subspaces corresponding to all known class targets in the Grassmann manifold space, and is specifically expressed as:

[0100]

[0101]

[0102]

[0103] In formulas (3) to (5), represents the set of new classes at the t-th stage, represents the set of historical classes, and d P (Y j , Y k ) calculates the manifold space distance between two subspaces Y j and Y k , where Y k represents the subspace corresponding to the existing known class target in the Grassmann manifold space, and Y j represents the subspace corresponding to the newly established unknown class target in the Grassmann manifold space.

[0104] In this embodiment, due to the non-uniqueness of matrix decomposition and the finiteness of the subspace's representation of target detailed information, the retention of old knowledge by the manifold space constraint model is inaccurate and insufficient. In particular, considering the diverse characteristics of the target's local scattering features and the generalization recognition of global scattering features, the SLF loss shown in Figure 5 is designed in this method to constrain the local and global target semantic information and achieve the retention of old knowledge.

[0105] Specifically, the unstructured forgetting loss function is constructed according to the features extracted by the feature extractor in the target recognition network before the previous online iterative training and the features extracted by the feature extractor in the current target recognition network, and is specifically expressed as:

[0106]

[0107] In formula (6), and Are defined as the spatial and global loss constraints, s t-1 (x i ) and s t (x i ) represent the spatial features output by the feature extractors in the t-th and (t - 1)-th stages for the sample x i respectively, and represent the output global feature values. Among them, in the t-th stage, the sample x i is selected from the set of all current training samples Feature extractor For the sample x i The feature extraction process can be decoupled into where s t (·) refers to the stack convolution operation for extracting spatial features, and g is the global average pooling operation for obtaining aggregated features. s t (x i ) and g(s t (x i )) have rich local and global target features respectively. In the SLF loss, by constraining the feature consistency of the sample in the local and global aspects by the previously solidified model and the current trainable model , the retention of old knowledge is achieved. The proposed SLF loss function can be formalized as formula (6), which is the superposition of the distillation terms of the spatial and global features generated by the historical and current models.

[0108] It should be noted here that in formulas (2) and (6), f φ and both refer to the feature extractor. Among them, f φ is used to refer to the feature extractor. However, because there are many sequential learning processes in the algorithm, in the t-th learning process, its specific writing of the feature extractor At the same time, since this loss function involves using f φ at two moments t and t - 1 to participate in the operation, at these two different moments, it can be specifically written as

[0109] In this embodiment, considering the azimuth dependence and partial sparsity of the targets in the SAR image, an azimuth-aware sample selection (AES) strategy is also provided to select representative and complementary old-class samples.

[0110] The samples selected based on the Azimuth Angle Sensing (AES) strategy are based on the criteria of representativeness and complementarity. Among them, the sample images that meet the representativeness usually have high energy values, and contain more distinguishable target features in terms of quality and quantity, and have a generally high correlation with the samples under various azimuth angle imaging within the class. However, the samples selected according to this criterion may be concentrated in a small azimuth angle area, which may not represent the general features of the target in most poses.

[0111] The samples that meet the complementarity can cover more uncommon target features without involving too much redundancy, providing complete information for knowledge preservation. However, selecting samples only under this criterion may contain a lot of specific information, thus hindering the model's expression of the general features of the target.

[0112] Therefore, it is crucial to combine representativeness and complementarity for the selection of exemplars. Especially considering the association between representativeness and the scattering energy of the target and the complementarity with the within-class correlation, the Azimuth Angle Sensing (AES) strategy is introduced in this method to effectively preserve the knowledge of the old classes.

[0113] In this embodiment, Algorithm 2 is as follows, which details the specific process of the strategy for selecting exemplar samples of class c in the t-th incremental learning stage.

[0114]

[0115] In the above SAR target recognition method based on small samples, for the actual SAR ATR application scenario where new category targets continuously appear and the sample size is limited during the rapid acquisition of high-resolution SAR images, the few-shot class incremental learning (FSCIL) is introduced into the SAR ATR task for the first time in this method. Considering the compactness of the Grassmann manifold space in representing target information, good inter-class separability, and robustness to background interference, an azimuth-aware subspace classifier (AASC), that is, a target recognition network, is proposed. Aiming at the dilemmas of the plasticity and stability of FSCIL and the unique challenges in the SAR ATR field, in this method, by combining the azimuth dependence and component sparsity characteristics of the target's own scattering points, loss functions covering the semantic space and the subspace manifold (SMS, DSS, SLF) and an azimuth-aware sample selection strategy (AES) are designed. For the plasticity of the model (target recognition network), SMS expands the semantic boundary at the sample level of new classes with respect to old classes, and DSS maintains the orthogonality of class subspaces in the manifold. For the stability of the model, SLF constrains the consistency of the model with respect to the sample feature space and the global structure. Considering the periodicity and volatility of the target azimuth-aware features, the AES strategy aims to select representative and complementary samples. The proposed method can be better applied to more practical SAR recognition scenarios and, at the same time, can recognize targets more accurately.

[0116] It should be understood that although Figure 1 the steps in the flowchart of Figure 1 are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover,

[0117] In one embodiment, as Figure 6 shown, a SAR target recognition device based on small samples is provided, including: a SAR image acquisition module 300, a target recognition network prediction module 310, an unknown category target online learning module 320, and a known category target recognition module 330, where:

[0118] The SAR image acquisition module 300 is configured to acquire a SAR image to be subject to target recognition;

[0119] The target recognition network prediction module 310 is configured to input the SAR image into the trained target recognition network. In the target recognition network, a feature extractor extracts the features of the SAR image, maps the extracted features into the Grassmann manifold space, and classifies the extracted features through the subspaces corresponding to various known-class targets included in this space.

[0120] The unknown-class target online learning module 320 is configured to, if the target in the SAR image is of an unknown class, use the SAR image as small-sample training data to perform online iterative training on the target recognition network. During the online iterative training process, it also adjusts the parameters in the feature extractor by using the semantic margin separation loss function and the unstructured forgetting loss function, and adjusts the parameters of the Grassmann manifold space by using the deep subspace separation loss function. Until each loss function converges, a target recognition network capable of recognizing this unknown-class target is obtained.

[0121] The known-class target recognition module 330 is configured to, if the target in the SAR image is of a known class, output the prediction result of the target to achieve the recognition of the target in the SAR image.

[0122] For the specific limitations of the SAR target recognition device based on small samples, reference can be made to the limitations of the SAR target recognition method based on small samples in the above text, which will not be elaborated here. Each module in the above SAR target recognition device based on small samples can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0123] In one embodiment, a computer device is provided. This computer device can be a terminal, and its internal structure diagram can be as Figure 7As shown in the figure. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it realizes a SAR target recognition method based on small samples. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0124] Those skilled in the art can understand that Figure 7 the structure shown in the figure is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0125] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented:

[0126] Obtain a SAR image to be subjected to target recognition;

[0127] Input the SAR image into a trained target recognition network. In the target recognition network, a feature extractor extracts the features of the SAR image, maps the extracted features into a Grassmann manifold space, and classifies the extracted features through subspaces corresponding to various known category targets included in this space;

[0128] If the target in the SAR image is of an unknown category, use the SAR image as small sample training data to perform online iterative training on the target recognition network. During the online iterative training process, also adjust the parameters in the feature extractor by using a semantic margin separation loss function and an unstructured forgetting loss function, and adjust the parameters of the Grassmann manifold space by using a deep subspace separation loss function until each loss function converges, then obtain a target recognition network with the ability to recognize this unknown category target;

[0129] If the target in the SAR image is of a known category, output the prediction result of the target to realize the recognition of the target in the SAR image.

[0130] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0131] Obtain a SAR image to be subjected to target recognition;

[0132] Input the SAR image into a trained target recognition network. In the target recognition network, a feature extractor extracts features of the SAR image, maps the extracted features into a Grassmann manifold space, and classifies the extracted features through subspaces corresponding to various known-class targets included in this space;

[0133] If the target in the SAR image is of an unknown class, use the SAR image as small-sample training data to perform online iterative training on the target recognition network. During the online iterative training process, also adjust the parameters in the feature extractor by using a semantic margin separation loss function and an unstructured forgetting loss function, and adjust the parameters of the Grassmann manifold space by using a deep subspace separation loss function until each loss function converges, then obtain a target recognition network capable of recognizing the target of this unknown class;

[0134] If the target in the SAR image is of a known class, output a prediction result of the target to implement recognition of the target in the SAR image.

[0135] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0136] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0137] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. A SAR target recognition method based on small samples, characterized in that The method includes: Obtaining a SAR image to be subjected to target recognition; Inputting the SAR image into a trained target recognition network. In the target recognition network, a feature extractor extracts the features of the SAR image, maps the extracted features into a Grassmann manifold space, and classifies the extracted features through subspaces corresponding to various known category targets included in this space; If the target in the SAR image is of an unknown category, then use the SAR image as small-sample training data to perform online iterative training on the target recognition network. During the online iterative training process, also adjust the parameters in the feature extractor by using a semantic margin separation loss function and an unstructured forgetting loss function, and adjust the parameters of the Grassmann manifold space by using a deep subspace separation loss function until each loss function converges, then obtain a target recognition network capable of recognizing the target of this unknown category. Among them, the semantic margin separation loss function is constructed based on the features respectively extracted by the feature extractor from the SAR images of known category targets and the SAR images of unknown category targets, and is specifically represented as: ； Among them, ; In the above formula, represents the old category exemplars stored in the th learning stage, represents the new category data set used for training in the current stage, represents the anchor sample selected from , and are the positive sample and the negative sample selected from and respectively, represents the cosine distance between the anchor and the positive sample, represents the cosine distance between the anchor and the negative sample, represents the trainable feature extractor; The unstructured forgetting loss function is constructed based on the features extracted by the feature extractor in the target recognition network before the previous online iterative training and the features extracted by the feature extractor in the current target recognition network, and is specifically represented as: ； Among them, ; In the above formula, and are respectively defined as the spatial and global loss constraints, and respectively represent the spatial features output by the feature extractors in the -th and -th stages for the sample , and represent the output global feature values, where, in the -th stage, the sample is selected from the current set of all training samples ; The deep subspace separation loss function is constructed based on the subspace corresponding to the newly established unknown category target in the Grassmann manifold space and the subspaces corresponding to all known category targets, and is specifically represented as: ； Among them, ; In the above formula, represents the new category set of the th stage, represents the historical category set, the manifold space distances of two subspaces and are calculated, where represents the subspace corresponding to the known category target that already exists in the Grassmann manifold space, represents the subspace corresponding to the newly established unknown category target in the Grassmann manifold space; If the target in the SAR image is of a known category, then output the prediction result of the target to realize the recognition of the target in the SAR image.

2. The SAR target recognition method according to claim 1, wherein After using the SAR image with an unknown target category to perform online training on the target recognition network, the Grassmann manifold space in the target recognition network includes a subspace corresponding to this unknown category.

3. The SAR target recognition method according to any one of claims 1-2, characterized in that, The feature extractor adopts a CNN neural network.

4. The SAR target recognition method according to claim 3, wherein, The SAR target recognition method based on small samples further includes training the target recognition network before obtaining the SAR image to be subjected to target recognition, including: Obtaining a SAR training data set, where the SAR training data set includes multiple SAR training image sets of known category targets, and each SAR training image set corresponding to a known category target includes multiple SAR training images; Constructing a base class training data set for the base class training stage and multiple small-sample training data sets for the small-sample incremental learning stage according to the SAR training data set; In the base class training stage, input the base class training data set into the target recognition network to perform iterative training on it, and use a cosine cross-entropy loss function to adjust the parameters of the feature extractor in the target recognition network in each iterative training until convergence, then obtain a target recognition network capable of recognizing base class targets, and at this time, the Grassmann manifold space in the target recognition network includes subspaces corresponding to each base class target; Select samples from the base class training dataset according to the azimuth angle perception example selection strategy, and construct a historical sample dataset with the selected samples; In the few-shot incremental learning stage, input multiple few-shot training datasets into the target recognition network capable of recognizing base class targets in sequence for training; Before training the target recognition network capable of recognizing base class targets with the first few-shot training dataset, also merge this few-shot training dataset with the historical sample dataset, and input the merged training dataset into the target recognition network capable of recognizing base class targets until the target recognition network converges, and then use the next few-shot training dataset to train the target recognition network until all few-shot training datasets have been used to train the target recognition network, and then obtain the trained target recognition network.

5. The SAR target recognition method according to claim 4, wherein In the few-shot incremental learning stage, also use the semantic margin separation loss function, the unstructured forgetting loss function, and the deep subspace separation loss function to train the target recognition network.

6. SAR target recognition device based on small samples, characterized in that, Implement the few-shot based SAR target recognition method according to any one of claims 1-5 in the device, including: An SAR image acquisition module, configured to acquire an SAR image to be target-recognized; A target recognition network prediction module, configured to input the SAR image into the trained target recognition network, extract the features of the SAR image by a feature extractor in the target recognition network, map the extracted features to the Grassmann manifold space, and classify the extracted features through the subspaces corresponding to various known category targets included in this space; An unknown category target online learning module, configured to, if the target in the SAR image is an unknown category, use the SAR image as a few-shot training data to perform online iterative training on the target recognition network. During the online iterative training process, also adjust the parameters in the feature extractor by using the semantic margin separation loss function and the unstructured forgetting loss function, and adjust the parameters of the Grassmann manifold space by using the deep subspace separation loss function until each loss function converges, and then obtain a target recognition network capable of recognizing this unknown category target; A known category target recognition module, configured to, if the target in the SAR image is a known category, output a prediction result of the target to realize the recognition of the target in the SAR image.

7. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 4 to 5.

Images