Cosine prototype learning based small sample incremental sar image target recognition method

Abstract

The application relates to a small sample incremental SAR image target recognition method based on cosine prototype learning. The method proposes a cosine prototype learning framework, enables an existing deep learning algorithm to continuously update iteration based on limited samples, and does not lose the discriminability of old knowledge; class-specific knowledge is separately expressed as a cosine prototype, the model generalization ability is not lost, and target features are accurately represented; a pair structure separation loss is used to learn new knowledge and the smooth influence of diversified target azimuth angles, a prototype consistency loss is used to prevent the learned new knowledge from degrading in migration; a prototype sample distillation loss and a prototype recalibration strategy are used to overcome the catastrophic forgetting of old knowledge by the algorithm, to continuously punish semantic drift and reduce the misplacement of learned prototypes; during reasoning, the cosine similarity between a test sample and a class-specific prototype is compared, and accurate prediction of an unknown sample class is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of SAR image processing, in particular to a small sample incremental SAR image target recognition method based on cosine prototype learning. BACKGROUND

[0002] As a basic problem in the field of synthetic aperture radar (SAR) image interpretation, SAR automatic target recognition (SAR ATR) has attracted extensive attention and in-depth research due to its important role in military and civilian fields. In recent years, with the rapid development of SAR imaging technology, the number of high-resolution SAR images has increased exponentially, providing more and more research opportunities for the challenging SAR target recognition task.

[0003] In recent years, with the introduction of deep learning (DL) in the field of SAR ATR, SAR ATR technology has made great progress. However, the ability of these algorithms to continuously learn and update from effective samples is still limited, which limits their application ability in real-world scenarios. In particular, due to the limited number of new class samples, the model is prone to catastrophic forgetting of historical knowledge when learning new knowledge, thereby reducing the algorithm's recognition ability for previously seen classes. Therefore, it is necessary to study the problem of few-shot class incremental learning (FSCIL) to improve the model's discriminative ability for different classes in a dynamic environment.

[0004] In addition, due to the obvious differences in target features between SAR and natural images, FSCIL for SAR ATR is more unique and complex, and faces more challenges.

[0005] (1) Limited target information. Due to the special imaging characteristics of SAR, targets in SAR images are roughly represented as backscattering points with specific topological patterns. Despite this, they still lack rich color and appearance cues, providing limited guidance for optimizing feature extractors. Therefore, obtaining representative features of targets in SAR images requires in-depth mining of the structural information of target scattering points.

[0006] (2) Inter-class similarity and intra-class difference. Due to the diversity of imaging conditions and the similarity of target configurations, such as target azimuth angle, components, for example, T72 and T62 equipped with turret, etc., the features of different classes are easily overlapped, and the differences within the same class can be very serious. Therefore, maintaining the model's considerable ability to distinguish different targets during the incremental learning process requires more attention to targets located at the decision boundary and enhancing the consistency of intra-class sample features.

[0007] (3) Severe background interference. Due to the diversity and non-uniformity of the surrounding environment and the particularity of the SAR imaging mechanism, there may be extensive clutter around the target, which causes a large amount of interference to the accurate representation of the target features based on the DL model, especially under the support of limited samples. SUMMARY

[0008] Therefore, it is necessary to provide a small sample incremental SAR image target recognition method based on cosine prototype learning in view of the above technical problems.

[0009] A small sample incremental SAR image target recognition method based on cosine prototype learning, the method comprises:

[0010] Obtain a plurality of labeled SAR image data sets; the first SAR image data set is used as the basic session data set, and the other SAR image data sets are used as the incremental session data sets.

[0011] In the basic training stage, the feature extractor is trained on the basic session data set through the cosine cross-entropy loss, and the basic trained feature extractor and a plurality of class-specific prototypes are obtained; and the saved samples selected from the class set are stored.

[0012] In the incremental training stage, the saved samples and the first incremental session data set are combined as input, and the basic trained feature extractor is trained through the incremental training loss. After the learning is completed, the prototypes of the new class and the old class are recalibrated through the PR strategy, and the feature extractor after the first incremental session and the class-specific prototype are obtained. The saved samples selected from the first incremental session data set are stored, and the above steps are repeated until the last incremental session data set, and the trained feature extractor and the final class-specific prototype are obtained; the incremental training loss comprises: a pair-wise structural disentanglement loss, a prototype consistency loss, and a prototype sample distillation loss.

[0013] In the inference, the cosine similarity between the test sample and the final class-specific prototype is compared, a non-parametric classifier is used for evaluation, and the class of the test sample is determined.

[0014] In one of the embodiments, the incremental training loss is:

[0015] ;

[0016] wherein, is the incremental training loss, is the pair-wise structural disentanglement loss, is the prototype consistency loss, is the prototype sample distillation loss, is the parameter.

[0017] In one embodiment, the pairwise structure separation loss is:

[0018] ;

[0019] in, For paired structure separation loss, These are the selected anchor point and the positive sample of the same category as the anchor point. It is a negative sample for comparison. It is a comparison of the correct quantity. For the first t A dataset of SAR images, For the first t -1 saved sample, Number of SAR image datasets; It comes from the first Sample sets of each incremental stage Samples in High-dimensional semantic features To control the boundaries between samples, For the sample Tags; The distance between the selected anchor point and the positive sample of the same class as the anchor point.

[0020] In one embodiment, the prototype consistency loss is:

[0021] ;

[0022] in, For prototype consistency loss, In the first t With the t -1 incremental learning process category c The cosine similarity measurement results of the prototypes, For the first t Incremental process categories c The prototype, For the sample The tag, For the first t A dataset of SAR images, For category c The number of training samples included. For category labels, This represents the total set of category labels for the current incremental phase.

[0023] In one embodiment, the prototype sample distillation loss is:

[0024] ;

[0025] in, Distillation loss of prototype sample For from the t Sample sets for each incremental stage This represents the total set of category labels for the current incremental phase. In the first t-1 With the t The cosine similarity measure of the deviation between each category sample instance and its category prototype in each incremental learning stage. It is an old category The prototype, It comes from the first t Sample sets of each incremental stage Samples in High-dimensional semantic features For the total number of samples, The samples extracted by the feature extractor in the t-th stage semantic features For category labels, For the sample The tag.

[0026] In one embodiment, during inference, a nonparametric classifier is used to evaluate and determine the category of the test sample by comparing the cosine similarity between the test sample and the final class-specific prototype, including:

[0027] During inference, the category of the test sample is determined by comparing the cosine similarity between the test sample and the final class-specific prototype using a non-parametric classifier based on the cosine classification criterion; the expression for the metric of the cosine classification criterion is as follows:

[0028] ;

[0029] in, Because of the high-dimensional semantic features output by the feature extractor, For category Prototype features, k This is a metric distance between sample features and class prototype features.

[0030] The output of the NCM classifier is:

[0031] ;

[0032] in, The model is on the test sample The output, Indicates the first t Stage Categories c The calibrated prototype, The result of the prediction.

[0033] In one embodiment, the saved samples are selected from the class set using a swarm strategy.

[0034] The above-mentioned small sample incremental SAR image target recognition method based on cosine prototype learning proposes a cosine prototype learning framework to teach a deep learning-based method to gradually learn new knowledge using few samples without losing discrimination for old knowledge. By individually compressing class-specific knowledge into cosine prototypes, a stable profile of the target can be described without losing generality. For the plasticity of the model, a pair-wise structural disentanglement loss is introduced to learn new knowledge and smooth the influence of the diverse target azimuth angle, and a prototype consistency loss is introduced to prevent the learned new knowledge from degrading in transferability; prototype sample distillation loss and prototype recalibration strategy are adopted to overcome the catastrophic forgetting of old knowledge, to continuously punish semantic drift and mitigate misalignment of learned prototypes. During inference, the cosine similarity between the test sample and the class-specific prototype is compared to achieve accurate prediction of the unknown sample class. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 The flowchart of the small sample incremental SAR image target recognition method based on cosine prototype learning in one embodiment;

[0036] Figure 2 The overall framework diagram of the small sample incremental SAR image target recognition method based on cosine prototype learning in another embodiment;

[0037] Figure 3 The overlap and difference between inter-class and intra-class targets in SSIM and semantic similarity in another embodiment;

[0038] Figure 4 The PSS loss function in another embodiment;

[0039] Figure 5 The evolution of the model feature space in the incremental learning process in another embodiment;

[0040] Figure 6 The PSS loss function in another embodiment; The old class sample set of the tth incremental stage;

[0041] Figure 7 The PR strategy in another embodiment. DETAILED DESCRIPTION

[0042] ​In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0043] To address the dilemma of the above FSCIL and the specific domain challenges in the SARAIR task, the present application proposes a cosine prototype learning (CPL) framework, in which the topological features of the target are represented as diverse cosine relationships in a deep embedding space. In the cosine prototype learning framework, various cosine structures of deep embedding features are carefully considered, benefiting from their inherent connection with the topological information of the target and robustness to background clutter. Notably, the advantages of this representation can be specifically described from two aspects. One is the inner product operation, by which the structural semantic links related to the backscattering clues of the target components can be completely preserved. The other is the normalization operation, by which the background interference caused by multiplicative speckle noise and feature deviation between different classes can be smoothed to a certain extent. By condensing the structural features of the few-shot samples into prototypes of specific classes, a general and stable profile of the intra-class target with diverse configurations and azimuth angles can be formulated. Specifically, in the initial learning phase where large-scale training data can be collected in advance, a CNN-based feature extractor is first learned through cosine-based cross-entropy loss for extracting the general structural features of the target in the later incremental phase. Pairwise structure separation (PSS) loss and prototype consistency (PC) loss are introduced to make the present application method plastic by continuously learning the structural clues of new targets without losing transferability. Prototype sample distillation (PED) loss and prototype recalibration (PR) strategy are introduced to maintain the stability of our old knowledge method by continuously punishing semantic drift and rectifying the discrimination of the learned prototype. At inference time, a non-parametric classifier is used to assign the test sample to the class with the maximum cosine similarity among all the saved prototypes of all classes.

[0044] In one embodiment, as shown in Figure 1 a cosine prototype learning based small sample incremental SAR image target recognition method is provided, which comprises the following steps:

[0045] Step 100: Obtain a plurality of labeled SAR image data sets; take the first SAR image data set as the base session data set, and take the other SAR image data sets as the incremental session data sets.

[0046] Specifically, the plurality of labeled SAR image data sets are , wherein any one SAR image data set contains samples and corresponding labels Setting for This has always been true. The first SAR image dataset. This dataset provides a large number of basic class training samples for the basic session dataset. This is an incremental session dataset containing small samples of some new categories, i.e. N Each category K One data point, N and K The values ​​are all very small.

[0047] Step 102: In the basic training phase, the feature extractor is trained on the basic session dataset using cosine cross-entropy loss to obtain the basic trained feature extractor and several class-specific prototypes; and the saved samples selected from the class set are stored.

[0048] Specifically, during the basic training phase, the parameters of the CNN-based feature extractor... First, cosine cross-entropy (CE) loss is applied to the base session data. The feature extractor is trained to identify base conversation classes and extract general features of new classes in subsequent incremental conversations. Then, the trained feature extractor... class prototype Calculated using average class features The perceptual training samples, and the class set selected by the clustering strategy. Saved samples It will be saved for future study.

[0049] The overall framework diagram of the small-sample incremental SAR image target recognition method based on cosine prototype learning is as follows: Figure 2 As shown.

[0050] Step 104: In the incremental training phase, the saved samples and the first incremental session dataset are merged and used as input. The feature extractor trained on the basis is trained using the incremental training loss. After learning, the prototypes of the new class and the old class are recalibrated using the PR strategy to obtain the feature extractor and class-specific prototype after the first incremental session. The saved samples selected from the first incremental session dataset are stored. The above steps are repeated until the last incremental session dataset to obtain the trained feature extractor and the final class-specific prototype. The incremental training loss includes: pairwise structure separation loss, prototype consistency loss, and prototype sample distillation loss.

[0051] Specifically, during the incremental training phase, such as the first incremental session (t=2), the small sample novel dataset... and preserved samples The current session was merged into the training data, which includes numbers from 1 to... classes. In the incremental learning process, PSS and PC losses are used to expand the differences between the novel and other classes and mitigate the overfitting of the model to the novel class-specific features. The PED loss is introduced to preserve the knowledge of the old classes, especially the structural relationships, by regularizing the old class feature space mapped by the previous fixed model and the current trainable model. After learning, the prototypes of the new class and the old classes are recalibrated by the PR strategy. Some representative examples of the new class are selected by the herding strategy for data replay in later sessions. By repeating the above steps until the last session t , the trained CNN-based feature extractor and the prototypes of the classes can be finally obtained.

[0052] In each session , the network is parameterized by a CNN-based backbone and a decoupled classifier and optimized on the dataset and some old class samples that can be preserved from previous sessions (i.e. ). In the interference phase of the session , the model needs to be able to distinguish all visible classes including the current session in previous sessions. Small sample class incremental learning is a practical and challenging problem that requires a balance between model stability and plasticity by preserving old class knowledge and integrating new class knowledge.

[0053] The pair-wise structural separation loss is used to learn new knowledge and smooth the influence of diverse target azimuth angles.

[0054] The prototype consistency loss is used to prevent the degradation of the learned new knowledge transferability.

[0055] The prototype sample distillation (PED) loss and the prototype recalibration (PR) strategy are used to continuously penalize semantic drift and mitigate the misalignment of learned prototypes.

[0056] The pair-wise structural separation (PSS) loss and the prototype consistency (PC) loss are introduced to make the method of the present application plastic by constantly learning the structural cues of new targets without losing transferability. The prototype sample distillation (PED) loss and the prototype recalibration (PR) strategy are introduced to maintain the stability of the method of the old knowledge by continuously penalizing semantic drift and rectifying the discrimination of learned prototypes.

[0057] Step 106: At inference, the class of the test sample is determined by comparing the cosine similarity between the test sample and the final class-specific prototype using a non-parametric classifier for evaluation.

[0058] Specifically, in the reasoning, for example, the last conversation t , through the trained feature extractor model , feature extraction is performed, by comparing the cosine distance between the features of each sample and all class prototypes from to , the test sample can be assigned to the specific class closest to all the compared prototypes.

[0059] In the above small sample incremental SAR image target recognition method based on cosine prototype learning, the method proposes a cosine prototype learning framework to teach a deep learning-based method, which gradually learns new knowledge using few samples without losing discrimination for old knowledge. By individually compressing class-specific knowledge into cosine prototypes, the stable profile of the target can be described without losing generality. For the plasticity of the model, a pair-wise structural disentanglement loss is introduced to learn new knowledge and smooth the influence of the multi-azimuth angle of the target, a prototype consistency loss is introduced to prevent the learned new knowledge from degrading in transferability; prototype sample distillation loss and prototype recalibration strategy are adopted to overcome the catastrophic forgetting of old knowledge, to continuously punish semantic drift and mitigate the misalignment of learned prototypes. In the reasoning, by comparing the cosine similarity between the test sample and the class-specific prototype, accurate prediction of the class of unknown samples is realized.

[0060] In one embodiment, the incremental training loss in step 104 is:

[0061] (1)

[0062] wherein, is the incremental training loss, is the pair-wise structural disentanglement loss, is the prototype consistency loss, is the prototype sample distillation loss, is the parameter.

[0063] The of the total loss is used to balance the plasticity of the model learning new concepts and the stability of knowledge preservation.

[0064] In one embodiment, the pair-wise structural disentanglement loss is:

[0065] (2)

[0066] wherein, is the pair-wise structural disentanglement loss, are the selected anchor and positive sample of the same class as the anchor, respectively, is the negative sample of the comparison, which is selected from other samples outside the anchor sample category. The positive sample and the negative sample have different labels. is the number of comparisons, is the t th SAR image dataset, is the t th saved sample, is the number of SAR image datasets; is the sample set from the th incremental stage is the high-dimensional semantic feature of the sample in the sample set, is the control sample boundary, is the label of the sample ; is the distance between the selected anchor and the positive sample of the same category as the anchor. For a specific anchor, the first samples with the farthest cosine distance from the anchor are selected to form a positive sample pair. At the same time, the first samples with the closest cosine distance from the anchor are selected to form a negative sample.

[0067] Specifically, the pair structure separation loss, Pairwise Structure Separation loss, is abbreviated as PSS loss.

[0068] The overlap and difference between the inter-class and intra-class targets on SSIM and semantic similarity are shown in Figure 3 , where (a) is the SSIM score between different targets, and (b) is the feature similarity score between different targets.

[0069] As can be seen from Figure 3 , in a small area of the target azimuth (rectangular area), both scores of the intra-class samples are higher than those of the inter-class samples, but are lower in other positions. In addition, with the change of the target azimuth, the two scores of the intra-class pair have serious fluctuations, for example, the structural or feature similarity changes from 1.0 to 0.90 or to 0.30. Therefore, it is necessary to expand the feature difference of the inter-class targets in the near azimuth (high similarity), while making the features of the intra-class targets at different azimuths (low similarity) more compact. The PSS loss is shown in Figure 4 , which can be expressed by equation (2).

[0070] In one embodiment, the prototype consistency loss is:

[0071] (3)

[0072] (4)

[0073] in, For prototype consistency loss, In the first t With the t -1 incremental learning process category c The cosine similarity measurement results of the prototypes, For the first t Incremental process categories c The prototype, For the sample The tag, For the first t A dataset of SAR images, For category c The number of training samples included. For category labels, This represents the total set of category labels for the current incremental phase.

[0074] Specifically, Figure 5 This demonstrates the evolution of the model's feature space during incremental learning. Figure 5 During the underfitting stage, the initial model can maintain a compact low-dimensional space on the old class (dark dashed ellipse), but it remains underfitted on the new class (light dashed ellipse). This can easily lead to misclassification of new boundary samples (light-filled hexagons). Conversely, in the right-hand subplot, models trained on small samples of the new class are prone to overfitting to the new class. The new class has biased and distorted feature spaces on both the new and old classes, resulting in a weak ability to distinguish the old class. Considering the generalization representativeness of the class prototype, a prototype consistency (PC) loss is proposed to achieve a balance. The prototype consistency loss is shown in equation (3), which is achieved by minimizing... and The method proposed in this application can learn new knowledge without losing generalization by using the cosine distance between them. Equation (4) gives the final class prototype calculation result. Accordingly, the feature space of the model is as follows: Figure 5 As shown in the intermediate subgraph, it constructs a compact, generalized feature space for the new class.

[0075] In one embodiment, the prototype sample distillation loss is:

[0076] (5)

[0077] in, Distillation loss of prototype sample For from the t Sample sets for each incremental stage This represents the total set of category labels for the current incremental phase. In the first t-1 With thet cosine similarity measure between the exemplar samples of each incremental learning stage and their class prototypes, is the prototype of the old class, is the prototype of the old class, is the sample in the sample set from the t th incremental stage is the high-dimensional semantic feature of the sample is the total number of samples, is the semantic feature of the sample extracted by the feature extractor of the t th stage, is the class label, is the label of the sample .

[0078] Specifically, the Prototype-Exemplar Distillation (PED) loss and the Prototype Re-calibration (PR) strategy are adopted to overcome the catastrophic forgetting problem in CPL and improve the model stability.

[0079] (1) PED loss

[0080] Knowledge distillation has been widely used in most incremental learning tasks to address the catastrophic forgetting problem. However, it requires large-scale training data, which is not available in the FSCIL of SAR ATR. In contrast to extracting a single point in the feature space, the proposed Prototype-Exemplar Distillation (PED) loss aims to provide rich supervision signals by regularizing the stability of limited samples and the corresponding cosine semantic structure, which can be defined as equation (5). By minimizing equation (5), the internal semantic structure spanned by class-aware exemplars and prototypes can be aligned in the learning session. Figure 6 The PED loss diagram is plotted in FIG. 2.

[0081] (2) Prototype Re-calibration (PR) strategy

[0082] Although the PED loss can preserve learned knowledge, the semantic structure of old classes inevitably drifts due to insufficient supervision provided by limited exemplars, which can lead to a significant performance degradation of old classes based on previous prototypes. To this end, we propose a simple but effective post-processing strategy, Prototype Re-calibration (PR), to mitigate the drift problem. As shown in equation (6).

[0083] (6)

[0084] (7)​

[0085] In the first t learning phase, is the prototype of the class before calibration, is the prototype after calibration, if is the old class, the prototype is compensated for the class perception bias . Since the overall sample of the old class is not available, we estimate the bias by where the average semantic bias of the class-specific examples is obtained by the difference between and . The number of examples of the class is denoted by . Further, the compensation strategy can be expressed as shown in Figure 7 . The final position of the prototype of the old class (solid gray pentagon) is obtained by moving the original (dashed gray pentagon) by the bias indicated by arrow 1. The start and end of the arrow refer to the average features of the projected old class samples and . The class prototype of the new class sample (yellow pentagon) is the average of the target features of the current space. It is worth noting that the network is fixed during calibration, and the prototype after calibration is used for prediction in the current session.

[0086] In one embodiment, step 106 includes: at the time of inference, determining the class of the test sample by comparing the cosine similarity between the test sample and the final class-specific prototype using a non-parametric classifier based on a cosine classification criterion; the expression of the measurement method of the cosine classification criterion is:

[0087] (8)

[0088] where, is the high-dimensional semantic feature due to the feature extractor output, is the prototype feature of the class , is the measurement distance between the sample feature and the class prototype feature.

[0089] The output of the NCM classifier is:

[0090] (9)

[0091] where, is the output of the model on the test sample , denotes the prototype of the class t after calibration in the c th phase, The predicted result is the result of the prediction. The predicted result can be obtained by assigning the class c whose cosine similarity with is the highest among all prototypes.

[0092] Specifically, in general, the posterior probability of a sample x can be expressed as formula 1.

[0093] (10)

[0094] wherein, is the high-dimensional semantic feature output by the feature extractor, is the prototype feature of the class c . is the metric distance between the sample feature and the class prototype feature. For the Euclidean distance or the fully connected classifier, it can be expressed as and . However, for the cosine classification criterion, the metric way can be expressed as is (8), which has the following three advantages.

[0095] 1) The target orientation perception topology can be accurately captured and represented by the cosine semantic feature.

[0096] 2) The target salient component can be accurately represented by the cosine semantic feature.

[0097] 3) The target feature represented under the cosine criterion is robust to background interference.

[0098] For the FSCIL of SAR ATR, the ability of the model to learn new concepts from a small number of samples without forgetting previous concepts needs to be balanced between the plasticity and stability of the model. In particular, considering the scarcity of recognition target information provided by limited instances and severe background interference, detailed information of the target orientation perception backscattering structure should be fully captured to solve the forgetting problem and learn adaptive new concepts. According to the above advantages of the cosine criterion, it is reasonable to construct a cosine-based framework for the FSCIL of SAR ATR to dynamically retain class perception recognition structure and distinguish different classes of targets, especially those imaged under similar conditions.

[0099] Due to the scarcity of important information provided by a small number of new samples, it is difficult to learn the discriminative features of new classes through a parametric classifier. In addition, the unbalanced training samples provided by the base and incremental sessions inevitably lead to a biased representation of different classes by the trained CNN model. Considering the form of the cosine criterion and its advantage of representing target structure details without losing generalization, the cosine-based NCM classifier is adopted in this application for model prediction, as shown in formula (9).

[0100] In one embodiment, the saved samples are a clustering strategy. The selected sample.

[0101] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0102] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0103] In a verification embodiment, MSTAR (Moving and Stationary Target Acquisition and Recognition) is used as the dataset for evaluating the method of this application. MSTAR includes a large number of target images captured under various configurations (such as type, azimuth, target function, radar concave angle, background, etc.). Its completeness, diversity, and standardization are considered for method performance evaluation. To simulate a real learning environment, we constructed a dataset based on MSTAR with the following configuration, where the configurations of the base class and incremental class are shown in Tables 1 and 2.

[0104] Table 1. Large-scale sample data in the foundational phase

[0105]

[0106] Table 2 Small sample data during the incremental phase

[0107]

[0108] (1) Training settings

[0109] In this embodiment, a light-weight deep residual network ResNet-18 is used as the feature extractor for both the proposed method and all the comparison methods. In the base session, the model is trained for 50 epochs with a batch size of 32 using the stochastic gradient descent optimizer (SGD). The momentum decay rate and the weight decay rate are set to 0.9 and 5e-4, respectively. The initial learning rate is set to 1e-2, and is decayed by 0.1 at the 30th and 40th epoch. In the incremental session, the model is trained using the same optimizer and configuration. Different from before, the initial learning rate is set to 1e-4, and is decayed by 0.1 at the 45th and 70th epoch. All input images are standardized and cropped to the size of 64x64 pixels. In terms of data augmentation, all training images are randomly rotated with a small angle to expand their diversity. All experiments are conducted on a server equipped with an Nvidia GTX 3090 card and CUDA 11.1.

[0110] (2) Evaluation scheme

[0111] Three schemes are adopted in this embodiment to comprehensively evaluate the performance of the comparison methods. The incremental classification accuracy is a general indicator calculated in each incremental session, which is used to judge the performance of the current model in classifying all encountered classes. The average accuracy (Avg.Acc) reflects the overall performance of the trained model by averaging the incremental accuracy of all sessions. The performance drop (PD) measures the absolute performance drop of the model in the last session, which is calculated by subtracting the accuracy of the last session from the accuracy of the first session. The larger the value of PD, the worse the incremental performance of the model.

[0112] (3) Ablation experiment comparison

[0113] 1) Cosine feature representation advantage. In this embodiment, the influence of the cosine criterion on representing important features of the target is studied. In order to purely verify the contribution, we consider a stubborn model that does not adapt to new tasks. Most specifically, the model is trained on base session data according to the three criteria of FC, Euclid and cosine, and directly serves as a feature extractor for the NCM classifier to reason in subsequent sessions. In particular, the model is first trained and tested by the same criterion, and then all trained and tested by the cosine criterion. Table 3 shows the performance improvement under the cosine metric strategy.

[0114] Table 3 Superiority of FSCIL using cosine prototype for SAR ATR in 1-way 5-hot setting

[0115]

[0116] Table 3 shows the performance of the models trained and tested according to different criteria. The criteria used in the training and testing phases are placed on the left and right sides of the arrows, respectively. From the first two rows of Table 3, we can conclude that considering only the target amplitude features within the Euclid criterion is prone to inaccurate predictions due to severe background clutter. However, by exploring the relationships between target features through the FC criterion, target discriminative features can be captured, and the performance of the model can be improved to some extent. From the third and fourth rows of Table 3, the structural cues of the target in the Euclid and FC feature spaces can be further exposed and exploited in the cosine-based feature space, significantly improving the performance of the original model in classifying both the base and incremental classes. Notably, the accuracy of the model can be further improved when using the cosine criterion for judgment, possibly due to the inherent information of the target captured by the FC-learned feature space. Additionally, as shown in the last column of Table 3, the model trained and tested using the cosine criterion performs the best among all these comparisons, benefiting from the advantage of capturing the target's normalized structural features and robustness to background interference.

[0117] 1) PSS loss effectiveness. Numerically, as shown in the first three and fifth rows of Table 4, our method using the PSS loss consistently outperforms the method trained by the cosine cross-entropy (CE) loss (first row of Table 4) and even those learned by the CE and other proposed strategies (see the second and third rows of Table 4), indicating the effectiveness of the PSS loss for learning new concepts. Specifically, the Avg.Acc of the method with the PSS loss is 1.36% and 0.75% higher than those of the method with only the CE and the method with three losses (CE, PED, and PR), respectively. Moreover, the PD of the method with the PSS loss is 0.88% and 0.14% lower than those achieved in the same model without the PSS loss.

[0118] 2) PC loss effectiveness. Numerically, the performance of our method with both PSS and PC losses, as shown in the sixth row of Table IV, can be further improved by 0.17% in Avg.Acc and 0.3% lower in PD compared to the method with the PSS loss (fifth row of Table 4).

[0119] 3) PED loss effectiveness. The PED loss through the CPL mitigates the catastrophic forgetting of the model on old knowledge. As shown in the seventh row of Table 4, the method using the PED loss outperforms the method without the loss in the last and middle sessions. Specifically, the Avg.Acc and PD of our method with the loss are 75.52% and 42.05%, respectively, which are 0.39% higher and 0.38% lower than those of the method without the loss.

[0120] 4) PR strategy effectiveness. The misalignment between the prototype and the old class feature space is mitigated by the PR strategy in CPL. As shown in the second to last row of Table 4, the method using PR can achieve a significant improvement in classification accuracy in all incremental sessions compared to the method without the strategy. Specifically, the Avg.Acc and PD of our method using the PR strategy are 76.83% and 39.53%, respectively, which are 1.31% higher and 2.53% lower than the method without the strategy, respectively. In addition, the progress of our method in classifying all visible classes becomes more prominent as the session increases, verifying the contribution of the PR strategy in correcting the cumulative drift of the feature space in consecutive sessions.

[0121] Table 4 Effectiveness of the components within the proposed framework tested on the MSTAR dataset under the 1-way 5-shot setting

[0122]

[0123] (4) Baseline algorithm comparison

[0124] a) Numerical results and performance curves. Our method is compared with various baselines under the three FSCIL settings of the SAR ATR task, and the comparison results of our method with other methods under the 1-way 5-shot setting on the MSTAR dataset are shown in Table 5, the comparison results of our method with other methods under the 2-way 5-shot setting on the MSTAR dataset are shown in Table 6, and the comparison results of our method with other methods under the one-step 5-shot setting on the MSTAR dataset are shown in Table 7.

[0125] Overall, the proposed method achieves the best performance among these methods in three FSCIL settings, except for Joint-CNN and cRT. No doubt, benefiting from the access to training samples of all old and new classes and the sophisticated long-tail training strategy, Joint-CNN and cRT achieve competitive performance. Nevertheless, the proposed method still achieves results close to or even higher than the scores of these two methods. In addition to this, the proposed method significantly outperforms most of the compared methods designed for IL, FSL, and FSCIL tasks in terms of Avg.Acc and PD. Specifically, the Avg.Acc of the proposed method is 76.83%, 76.23%, and 79.16% in the three settings, respectively, which is 5.04%, 4.17%, and 4.89% higher than the second-ranked method (CEC). Although sample-based comparison methods such as iCaRL, LUCIR, and IDLVQC preserve the knowledge of old classes, the ignorance of the relationship between samples and prototypes and the original classification strategy (e.g., Euclid in iCaRL and IDLVQC) can easily hinder their preservation of old knowledge and accurate extraction of new class features.

[0126] Table 5. Comparison results of the proposed method and other methods on the MSTAR dataset in the 1-way 5-shot setting

[0127]

[0128] Table 6. Comparison results of the proposed method and other methods on the MSTAR dataset in the 2-way 5-shot setting

[0129]

[0130] Table 7. Comparison results of the proposed method and other methods on the MSTAR dataset in the one-step 5-shot setting

[0131]

[0132] In view of the mismatch between the streaming data in real-world SAR ATR scenarios and the decision-making process of data-driven deep learning-based models, the CPL framework is proposed to solve the inherent plasticity and stability dilemma and the domain-related challenges in the SAR ATR field of few-shot class-incremental learning (FSCIL). In CPL, the target knowledge is represented as class-aware prototypes, and various cosine structure losses are designed according to the relationship between the azimuth-sensitive structure of the target and its semantic features under the cosine criterion. The learning ability of the model for new knowledge is guaranteed by PSS and PC losses, achieving good inter-class separation and intra-class aggregation, without losing the generalization representation property. The PED loss and PR strategy aim to preserve old knowledge, which alleviates the feature mismatch problem by constraining the structural relationship between class examples and prototype samples. In the test, the cosine-based NCM classifier is used for evaluation. A large number of experiments on the MSTAR dataset under three settings of FSCIL and detailed ablation studies prove that the method has significant advantages compared with various benchmarks, verifying its effectiveness in solving the FSCIL of SAR ATR.

[0133] The above-described embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the protection scope of the present application. It should be noted that, for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A small sample incremental SAR image target recognition method based on cosine prototype learning, characterized in that, The method comprises: Obtaining a plurality of labeled SAR image data sets; taking the first SAR image data set as a basic session data set and taking the other SAR image data sets as incremental session data sets; In the basic training stage, the feature extractor is trained on the basic session data set through the cosine cross-entropy loss, and a basic trained feature extractor and a plurality of class-specific prototypes are obtained; and the saved samples selected from the class set are stored; In the incremental training stage, the saved samples and the first incremental session data set are combined as input, the basic trained feature extractor is trained through the incremental training loss, the prototypes of new classes and old classes are recalibrated through the PR strategy after learning is completed, a first incremental session feature extractor and class-specific prototypes are obtained, the saved samples selected from the first incremental session data set are stored, and the above steps are repeated until the last incremental session data set, so that a trained feature extractor and final class-specific prototypes are obtained; the incremental training loss comprises a pair-wise structural disentanglement loss, a prototype consistency loss, and a prototype sample distillation loss; the incremental training loss is: in, For incremental training loss, For paired structure separation loss, For prototype consistency loss, Distillation loss of prototype sample For parameters; These are the selected anchor point and the positive sample of the same category as the anchor point. It is a negative sample for comparison. It is a comparison of the correct quantity. For the first t A dataset of SAR images, For the first t -1 saved sample, Number of SAR image datasets; It comes from the first Sample sets of each incremental stage Samples in High-dimensional semantic features To control the boundaries between samples, For the sample Tags; The distance between the selected anchor point and the positive samples of the same category as the anchor point. In the first t With the t -1 incremental learning process category c The cosine similarity measurement results of the prototypes, For the first t Incremental process categories c The prototype, For the sample The tag, For the first t A dataset of SAR images, For category c The number of training samples included. For category labels, This represents the total set of category labels for the current incremental phase. In inference, the cosine similarity between the test sample and the final class-specific prototype is compared, a non-parametric classifier is used for evaluation, and the class of the test sample is determined.

2. The method of claim 1, wherein, The prototype sample distillation loss is: in, Distillation loss of prototype sample For from the Sample sets for each incremental stage This represents the total set of category labels for the current incremental phase. In the first t-1 With the t The cosine similarity measure of the deviation between each category sample instance and its category prototype in each incremental learning stage. It is an old category The prototype, It comes from the first Sample sets of each incremental stage Samples in High-dimensional semantic features For the total number of samples, For the first The samples extracted by the feature extractor at each stage semantic features For category labels, For the sample The tag.

3. The method of claim 1, wherein, In inference, the cosine similarity between the test sample and the final class-specific prototype is compared, a non-parametric classifier is used for evaluation, and the class of the test sample is determined. In inference, the cosine similarity between the test sample and the final class-specific prototype is compared, a non-parametric classifier based on the cosine classification criterion is used for evaluation, and the class of the test sample is determined; the expression of the measurement method of the cosine classification criterion is: wherein, is a high-dimensional semantic feature output by the feature extractor, is a prototype feature of the class , k is a measured distance between the sample feature and the class prototype feature; The output of the NCM classifier is: wherein, is the output of the model on the test sample , represents the calibrated prototype of the class t at the c th stage, is the predicted result.

4. The method of claim 1, wherein, The saved samples are the selected samples under the ensemble strategy of the class set.

Images