A SAR Target Class Incremental Recognition Method Based on Knowledge Robust-Rebalanced Network

Abstract

The present invention relates to a SAR target class incremental recognition method based on a knowledge-robust and rebalanced network, including: obtaining a SAR target image training set; constructing a class incremental learning model based on the knowledge-robust and rebalanced network, and iteratively training the class incremental learning model with the SAR target image training set to obtain a trained class incremental learning model; using the trained class incremental learning model to achieve target recognition of the SAR image to be measured. Among them, the class incremental learning model includes: an old target teacher sub-network, a new target incremental learning sub-network, and a hybrid knowledge distillation module. The SAR target class incremental recognition method of the present invention can fully consider maintaining the separability information of the old class features and enhancing the feature separability of the new and old classes, and while realizing the incremental learning of the recognition model to recognize new categories, ensure its robustness in recognizing old targets.

Description

Technical Field

[0001] The present invention belongs to the technical field of target recognition, and particularly relates to a SAR target class incremental recognition method based on a knowledge-robust and rebalanced network. Background Art

[0002] Traditional target recognition methods are usually limited to static environment settings, that is, it is considered that the labeled training data of all target classes to be recognized can be obtained at one time. However, most actual recognition environments do not conform to the assumption of static conditions, and the labeled data of new target classes may be gradually obtained as the sensor data is continuously collected. Therefore, the actual recognition environment is dynamic.

[0003] At present, radar automatic target recognition (RATR) has made great progress and been widely applied in military and civilian fields. However, RATR is a typical dynamic environment target recognition task, which faces a dynamic observation environment where target classes present time-varying characteristics, and new classes of targets keep emerging. Therefore, it is difficult to obtain the data of all target classes to be recognized at the same time during the training stage to establish a complete target recognition library. In the above dynamic environment, once a new target class is observed, traditional target recognition methods need to retrain the recognition model by combining all the training data of new and old target classes to ensure its recognition ability for all observed target classes, resulting in significant consumption of time and space resources, which greatly limits the application of RATR technology in real dynamic environments.

[0004] The class incremental learning (CIL) method is a learning algorithm that can continuously update and evolve with the newly acquired data of new classes in a dynamic environment. It can update the existing recognition model based on the data of new target classes, endowing the updated recognition model with the ability to recognize both new and old target classes. The class incremental learning (CIL) method can efficiently update the SAR target recognition model based on the data of new target classes obtained by the SAR sensor, and at the same time realize the recognition of new and old SAR target classes, avoiding the inefficient repeated training of the recognition model in a dynamic environment.

[0005] However, in the process of updating the recognition model by combining the data of new target classes, the mainstream class incremental learning methods face a serious problem of catastrophic forgetting, that is, the recognition performance of the updated recognition model for old target classes drops significantly. To solve the above problems, researchers have made improvements from two aspects: how to make full use of the old target recognition model to transfer old knowledge, and how to fully correct the unbalanced learning of new and old target class data in model updating.

[0006] However, the existing old knowledge transfer and class imbalance correction strategies of class incremental learning methods only focus on the classification space, ignoring the key information of the features of old target classes and the impact of class imbalance on the learnability of the separability of new and old target class features, resulting in a significant decline in the recognition performance of the updated model for old target classes. Summary of the Invention

[0007] To solve the above problems existing in the prior art, the present invention provides a SAR target class incremental recognition method based on a knowledge-robust rebalancing network. The technical problems to be solved by the present invention are realized through the following technical solutions:

[0008] The present invention provides a SAR target class incremental recognition method based on a knowledge-robust rebalancing network, including:

[0009] Obtain a SAR target image training set;

[0010] Construct a class incremental learning model based on a knowledge-robust rebalancing network. The class incremental learning model includes: an old target teacher sub-network, a new target incremental learning sub-network, and a hybrid knowledge distillation module. Among them,

[0011] The old target teacher sub-network is used to provide old target knowledge for the new target incremental learning sub-network, and includes a cascaded first backbone feature extraction module and a classification module;

[0012] The new target incremental learning sub-network is used to incrementally learn new target classes to simultaneously recognize new target classes and old target classes, and includes a cascaded second backbone feature extraction module and a multi-prototype rebalancing module. The multi-prototype rebalancing module includes an unbiased classifier learning branch and a separability feature learning branch arranged in parallel;

[0013] The hybrid knowledge distillation module is used to transfer the knowledge of correctly recognizing old targets from the old target teacher sub-network to the new target incremental learning sub-network;

[0014] The loss function of the class incremental learning model is:

[0015] L total = λ1·[L mp + L frd +(1 - λ1)·[(1 - λ2)·L ce + λ2·L rd ;

[0016] In the formula, L mp represents the loss function of the separability feature learning branch, L ce represents the loss function of the unbiased classifier learning branch, L frd represents the feature topology relationship distillation loss function, L rd represents the classification layer response distillation loss function; λ1 represents the first balance coefficient, and λ2 represents the second balance coefficient;

[0017] Use the SAR target image training set to iteratively train the class incremental learning model to obtain a trained class incremental learning model;

[0018] Use the trained class-incremental learning model to achieve target recognition for the SAR image to be measured.

[0019] In one embodiment of the present invention, obtaining a SAR target image training set includes:

[0020] Obtain multiple SAR target images, crop each SAR target image into an image with a pixel of 64×64, and assign a class label to each SAR target image;

[0021] Divide the multiple SAR target images into multiple groups according to the category, and each group contains SAR target images of 2 categories;

[0022] Use one group of images as the initial training set to train the old target recognition model, and use other groups of images as the new target class training set to perform incremental updates on the class-incremental learning model. Among them, in each incremental update, select no more than 300 SAR target images of the old target class to jointly form an incremental learning training set with the new target class training set, and train and update the class-incremental learning model.

[0023] In one embodiment of the present invention, the structures of the first backbone feature extraction module and the second backbone feature extraction module are the same, and both include a cascaded first convolutional layer, multiple residual convolutional layers, and a pooling layer; among them,

[0024] The convolution kernel size of the first convolutional layer is 3×3, and the number of convolution kernels is set to 64;

[0025] The residual convolutional layer includes a second convolutional layer, a first Batch Norm layer, a first ReLU non-linear activation function layer, a third convolutional layer, a second Batch Norm layer, and a second ReLU non-linear activation function layer; among them,

[0026] The input feature map of the residual convolutional layer sequentially passes through the second convolutional layer, the first Batch Norm layer, and the first ReLU non-linear activation function layer to obtain a first feature map. The first feature map is sequentially mapped through the third convolutional layer and the second Batch Norm layer to obtain a second feature map. The second feature map is added to the downsampled input feature map of the residual convolutional layer and then passes through the second ReLU non-linear activation function layer to obtain the output feature map of the residual convolutional layer;

[0027] The pooling layer uses global average pooling.

[0028] In one embodiment of the present invention, the classification module includes a first fully connected layer, the number of input nodes of the first fully connected layer is 512, and the number of output nodes is N, where N represents the total number of old target classes.

[0029] In one embodiment of the present invention, the unbiased classifier learning branch includes a second fully connected layer, the number of input nodes of the second fully connected layer is 512, and the number of output nodes is N + M, where M represents the total number of newly added target classes;

[0030] The loss function of the unbiased classifier learning branch is:

[0031]

[0032] In the formula, B represents the number of training samples input to the unbiased classifier learning branch during the current batch training process, y b represents the true class label of the b-th training sample x b , represents the predicted classification score obtained by the training sample x b after passing through the second backbone feature extraction module and the unbiased classifier learning branch, and softmax() represents the normalized exponential function.

[0033] In one embodiment of the present invention, the separability feature learning branch includes a cascaded third fully connected layer and a fourth fully connected layer, the number of input nodes of the third fully connected layer is 512, the number of output nodes is 512, and the number of input nodes of the fourth fully connected layer is 512, and the number of output nodes is 128;

[0034] The loss function of the separability feature learning branch is:

[0035]

[0036] In the formula, B represents the number of training samples input to the separability feature learning branch during the current batch training process, represents the embedded feature obtained by the d-th training sample x d after passing through the second backbone feature extraction module and the separability feature learning branch, y d represents the true class label of the training sample x d , represents the s-th prototype parameter in the class prototype set of the j-th class, represents the s-th prototype parameter in the class prototype set corresponding to the class y d , and S represents the total number of prototype parameters in each class prototype set.

[0037] In one embodiment of the present invention, the feature topological relationship distillation loss function is:

[0038]

[0039] Wherein, Γ o represents the feature similarity matrix of the sample features extracted by the old target teacher sub-network, and Γ t represents the feature similarity matrix of the sample features extracted by the new target incremental learning sub-network, and W mask is the mask matrix;

[0040] The classification layer response distillation loss function is:

[0041]

[0042] Wherein, represents the soft output probability of the new target incremental learning sub-network for the sample x i in the j-th class, and q j (x i ) represents the soft output probability of the old target teacher sub-network for the sample x i in the j-th class.

[0043] In an embodiment of the present invention, before using the SAR target image training set to iteratively train the class incremental learning model to obtain a trained class incremental learning model, it further includes:

[0044] Training the old target recognition model using the initial training set, and using the trained old target recognition model as the old target teacher sub-network of the class incremental learning model.

[0045] In an embodiment of the present invention, using the SAR target image training set to iteratively train the class incremental learning model to obtain a trained class incremental learning model includes the following steps:

[0046] Step 3a: Initialize the parameters of the new target incremental learning sub-network;

[0047] Step 3b: Select 2B SAR target images from the incremental learning training set, where B SAR target images are obtained by randomly sampling the incremental learning training set, and the other B SAR target images are obtained by class-balanced sampling of the incremental learning training set;

[0048] Step 3c: Input the 2B SAR target images into the second backbone feature extraction module of the new target incremental learning sub-network to obtain corresponding features, input the features corresponding to the randomly sampled samples into the separability feature learning branch, and calculate the loss function of the corresponding separability feature learning branch, input the features corresponding to the class-balanced sampled samples into the unbiased classifier learning branch, and calculate the loss function of the corresponding unbiased classifier learning branch;

[0049] Step 3d: Input 2B SAR target images into the first backbone feature extraction module of the old target teacher sub-network to obtain corresponding features, calculate the corresponding feature topological relationship distillation loss function based on the features corresponding to the randomly sampled samples, input the features corresponding to the class-balanced sampled samples into the classification module of the old target teacher sub-network, and calculate the corresponding classification layer response distillation loss function;

[0050] Step 3e: According to the calculated loss functions of the discriminative feature learning branch, the unbiased classifier learning branch, the feature topological relationship distillation loss function, and the classification layer response distillation loss function, use the gradient descent method to update the parameters of the new target incremental learning sub-network;

[0051] Step 3f: Repeat steps 3b - 3e for multiple iterative trainings until a preset training stop condition is reached, complete one incremental update, and obtain a trained class incremental learning model.

[0052] In an embodiment of the present invention, the target recognition of the SAR image to be measured is realized by using the trained class incremental learning model, including:

[0053] Input the SAR image to be measured into the new target incremental learning sub-network of the trained class incremental learning model. After the second backbone feature extraction module extracts features from the SAR image to be measured, the extracted features are input into the unbiased classifier learning branch to obtain the predicted classification probability of the SAR image to be measured, and the predicted class of the SAR image to be measured is obtained according to the maximum probability.

[0054] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0055] The SAR target class incremental recognition method based on the knowledge-robust re-balanced network of the present invention uses a class incremental learning model based on the knowledge-robust re-balanced network that has been trained to achieve target recognition of the SAR image to be measured. This class incremental learning model, through a class imbalance correction strategy that combines designed feature separability learning and unbiased classifier learning, enhances the feature separability between new and old sample classes at the feature level based on multi-prototype metric learning, and optimizes a classifier with balanced class weights based on class-balanced sampling at the classification level, fully correcting the problem that the network is prone to misclassifying old target class test samples as new target classes due to new and old class imbalance learning; through a strategy for robustly preserving old target knowledge that combines designed feature topology relationship preservation and classification prediction preservation, at the feature level, the feature separability of the old classes learned by the old target recognition model is maintained during incremental updates based on topology relationship distillation, and at the classification level, the correct prediction ability of the old classes learned by the old target recognition model is maintained based on response knowledge distillation. The SAR target class incremental recognition method of the present invention can fully consider maintaining the feature separability information of old classes and enhancing the feature separability of new and old classes, and while realizing the incremental learning of the recognition model to recognize new classes, ensure its robustness in recognizing old targets.

[0056] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present invention more obvious and understandable, the following specifically gives preferred embodiments and, in conjunction with the drawings, details are described as follows. Brief Description of the Drawings

[0057] Figure 1 is a flowchart of a SAR target class incremental recognition method based on a knowledge-robust re-balanced network provided by an embodiment of the present invention;

[0058] Figure 2 is a schematic framework diagram of a class incremental learning model based on a knowledge-robust re-balanced network provided by an embodiment of the present invention;

[0059] Figure 3 is a schematic structural diagram of a residual convolutional layer provided by an embodiment of the present invention;

[0060] Figure 4 is a result graph of the recognition accuracy of an incremental experiment based on the MSTAR dataset of the method of the present invention and an existing class incremental method provided by an embodiment of the present invention. Detailed Embodiments

[0061] To further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following provides a detailed description of a SAR target class incremental recognition method based on a knowledge-robust re-balanced network proposed according to the present invention in combination with the accompanying drawings and specific embodiments.

[0062] The foregoing and other technical contents, features, and effects of the present invention will be clearly presented in the following detailed description in conjunction with the accompanying drawings. Through the description of the specific embodiments, a more in-depth and specific understanding of the technical means and effects adopted by the present invention to achieve the intended purpose can be obtained. However, the accompanying drawings are only for reference and illustration, and are not used to limit the technical solutions of the present invention.

[0063] Embodiment 1

[0064] Please refer to Figure 1 , Figure 1 which is a flowchart of a SAR target class incremental recognition method based on a knowledge-robust re-balanced network provided by an embodiment of the present invention. As shown in the figure, the SAR target class incremental recognition method based on the knowledge-robust re-balanced network in this embodiment includes:

[0065] Step 1: Obtain a SAR target image training set;

[0066] In an optional embodiment, Step 1 includes:

[0067] Step 1a: Obtain multiple SAR target images, crop each SAR target image into an image with a pixel size of 64×64, and assign a class label to each SAR target image;

[0068] In this embodiment, all 5172 images in the SAR vehicle target dataset MSTAR (moving and stationary target acquisition) are used as the sample set. Each of the 2746 images with a radar operating pitch angle of 17° in the sample set is cropped into an image with a pixel size of 64×64, and a class label from 1 to 10 is assigned to each image, that is, the 2746 images are divided into 10 classes.

[0069] Step 1b: Divide the multiple SAR target images into multiple groups according to the class, and each group contains SAR target images of 2 classes;

[0070] In this embodiment, the 10 classes of SAR target images are randomly divided into 5 groups, and each group contains SAR target images of 2 classes.

[0071] Step 1c: Use one set of images as the initial training set to train the old target recognition model, and use the other sets of images as the newly added target class training set to incrementally update the class incremental learning model. Among them, in each incremental update, select no more than 300 SAR target images of the old target class and the newly added target class training set to jointly form an incremental learning training set, and train and update the class incremental learning model.

[0072] In this embodiment, use the first set of images as the initial training set and the other 4 sets as the newly added target class training set. In each incremental update, select 300 SAR target images of the old target class and the newly added target class training set to jointly form an incremental learning training set. It should be noted that the proportion of each category in the 300 SAR target images of the old target class is equal.

[0073] Step 2: Construct a class incremental learning model based on the knowledge-robust re-balanced network;

[0074] Please refer to Figure 2 the schematic framework diagram of a class incremental learning model based on the knowledge-robust re-balanced network provided by the embodiment of the present invention shown in the figure. The class incremental learning model based on the knowledge-robust re-balanced network in this embodiment includes an old target teacher sub-network M o , a new target incremental learning sub-network M t and a hybrid knowledge distillation module HKD.

[0075] Among them, the old target teacher sub-network M o is used to provide old target knowledge for the new target incremental learning sub-network M t . In this embodiment, the old target teacher sub-network M o is a given old target recognition network.

[0076] Optionally, the old target teacher sub-network M o includes a cascaded first backbone feature extraction module F and a classification module C.

[0077] In an optional embodiment, the first backbone feature extraction module F includes a cascaded first convolutional layer, multiple residual convolutional layers, and a pooling layer; the classification module C includes a first fully connected layer.

[0078] In this embodiment, the first backbone feature extraction module F includes nine residual convolutional layers. The specific structure of the first backbone feature extraction module F is: the first convolutional layer → the first residual convolutional layer → the second residual convolutional layer → the third residual convolutional layer → the fourth residual convolutional layer → the fifth residual convolutional layer → the sixth residual convolutional layer → the seventh residual convolutional layer → the eighth residual convolutional layer → the ninth residual convolutional layer → the pooling layer. Among them, the convolutional kernel size of the first convolutional layer is 3×3, and the number of convolutional kernels is set to 64.

[0079] Please refer to Figure 3 the schematic structural diagram of a residual convolutional layer provided by the embodiment of the present invention shown in the figure. The residual convolutional layer of this embodiment includes a second convolutional layer, a first Batch Norm layer, a first ReLU non-linear activation function layer, a third convolutional layer, a second Batch Norm layer, and a second ReLU non-linear activation function layer. Among them, the input feature map of the residual convolutional layer passes through the second convolutional layer, the first Batch Norm layer, and the first ReLU non-linear activation function layer in sequence to obtain a first feature map. The first feature map is mapped through the third convolutional layer and the second Batch Norm layer in sequence to obtain a second feature map. The second feature map is added to the input feature map of the downsampled residual convolutional layer and then passes through the second ReLU non-linear activation function layer to obtain the output feature map of the residual convolutional layer. In this embodiment, the convolutional kernel sizes of the first to ninth residual convolutional layers are all 3×3, and the numbers of convolutional kernels are set to 64, 64, 128, 128, 256, 256, 256, 512, 512 in sequence;

[0080] In this embodiment, the pooling layer is set to global average pooling, and the 512×H×W feature map output by the ninth residual convolutional layer is mapped into a 512×1 feature vector. The number of input nodes of the first fully connected layer is 512, and the number of output nodes is N, where N represents the total number of old target classes.

[0081] Among them, the new target incremental learning sub-network M t is a network extended on the basis of the old target teacher sub-network M o and is used for incremental learning of new target classes to simultaneously achieve the recognition of new target classes and old target classes.

[0082] In an optional embodiment, the new target incremental learning sub-network M t includes a cascaded second backbone feature extraction module F' and a multi-prototype rebalancing module MPR. In this embodiment, the structures of the second backbone feature extraction module F' and the first backbone feature extraction module F are the same and will not be elaborated here. The multi-prototype rebalancing module MPR includes an unbiased classifier learning branch and a separability feature learning branch arranged in parallel.

[0083] Optionally, the unbiased classifier learning branch includes a second fully connected layer. The number of input nodes of the second fully connected layer is 512, and the number of output nodes is N+M, where M represents the total number of newly added target classes. Among them, the loss function of the unbiased classifier learning branch is:

[0084]

[0085] Wherein, B represents the number of training samples input to the unbiased classifier learning branch during the current batch training process, and y b represents the true class label of the b-th training sample x b . represents the predicted classification score obtained by the training sample x b after passing through the second backbone feature extraction module and the unbiased classifier learning branch, and softmax() represents the normalized exponential function.

[0086] In this embodiment, the loss function L of the unbiased classifier learning branch ce represents the cross-entropy loss between the predicted labels and the true labels obtained by the current batch of B samples through the unbiased classifier learning branch. It should be noted that the above B samples are from the class-balanced sampling of the incremental learning training set to ensure that the classifier learns unbiased class weight parameters.

[0087] Optionally, the separability feature learning branch includes a cascaded third fully connected layer and a fourth fully connected layer. The input node number of the third fully connected layer is 512, and the output node number is 512. The input node number of the fourth fully connected layer is 512, and the output node number is 128. Among them, the loss function of the separability feature learning branch is as follows:

[0088]

[0089] Wherein, B represents the number of training samples input to the separability feature learning branch during the current batch training process, represents the embedded feature obtained by the d-th training sample x d after passing through the second backbone feature extraction module and the separability feature learning branch, and y d represents the true class label of the training sample x d , represents the s-th prototype parameter in the class prototype set of the j-th class, represents the s-th prototype parameter in the class prototype set corresponding to the class y d , and S represents the total number of prototype parameters in each class prototype set.

[0090] In this embodiment, the loss function L of the separability feature learning branch mp represents the multi-prototype contrast loss between the embedded features obtained by the current batch of B samples through the separability feature learning branch and each class prototype set . The optimization of this loss can constrain the embedded features of each class of samples to be close to the prototype set of its own class in the feature space Furthermore, it is ensured that all the new target class features and old target class features extracted by the second backbone feature extraction module F' have good intra-class compactness and inter-class separability. It should be noted that the above B samples are randomly sampled from the incremental learning training set to ensure that the second backbone feature extraction module F' has good separable feature extraction ability under the original data distribution.

[0091] Among them, the hybrid knowledge distillation module HKD aims to make the feature response and classification decision response of the old target teacher sub-network M o and the new target incremental learning sub-network M t consistent, and is used to transfer the knowledge of correctly identifying the old target from the old target teacher sub-network M o to the new target incremental learning sub-network M t . In this embodiment, the hybrid knowledge distillation module HKD includes a feature topological relationship distillation loss function L frd and a classification layer response distillation loss function L rd two parts of distillation loss functions, that is, L HKD = L frd + L rd .

[0092] Among them, the feature topological relationship distillation loss function is:

[0093]

[0094] In the formula, Γ o represents the feature similarity matrix of the sample features extracted by the old target teacher sub-network, and Γ t represents the feature similarity matrix of the sample features extracted by the new target incremental learning sub-network, and W mask is the mask matrix.

[0095] Specifically, Among them, z i and respectively represent the features obtained by the first backbone feature extraction module F of the sample x i input into the old target teacher sub-network M o and the second backbone feature extraction module F' of the new target incremental learning sub-network M t , · represents the cosine similarity calculation, and W mask is used to extract the feature similarity terms of the old target class samples involved in Γ o and Γ t .

[0096] In this embodiment, the feature topological relationship distillation loss function L frd is used to constrain the current batch of B samples input into the old target teacher sub-network M o and the new target incremental learning sub-network M tThe similarity between the features is consistent. This loss transfers the discriminative knowledge learned by the old target teacher sub-network M o with high intra-class similarity and low inter-class similarity of the old target class features to the new target incremental learning sub-network M t , so that the new target incremental learning sub-network M t retains the ability to extract separable features of the old target.

[0097] Among them, the classification layer response distillation loss function is:

[0098]

[0099] In the formula, represents the soft output probability of the new target incremental learning sub-network for the sample x i in the j-th class, and q j (x i ) represents the soft output probability of the old target teacher sub-network for the sample x i in the j-th class.

[0100] Specifically, represents the classification score of the sample x i in the new target incremental learning sub-network M t , τ is the softening temperature coefficient,

[0101] represents the classification score of the sample x i in the old target teacher sub-network M o .

[0102] In this embodiment, the classification layer response distillation loss function L rd constrains the classification responses of the current batch of B samples input to the old target teacher sub-network M o and the new target incremental learning sub-network M t to be consistent, so that the new target incremental learning sub-network M t still retains the correct prediction ability for the old target class.

[0103] Step 3: Iteratively train the class incremental learning model using the SAR target image training set to obtain the trained class incremental learning model;

[0104] In this embodiment, before using the SAR target image training set to iteratively train the class incremental learning model to obtain the trained class incremental learning model, it further includes: training the old target recognition model using the initial training set, and using the trained old target recognition model as the old target teacher sub-network of the class incremental learning model.

[0105] In this embodiment, the specific training process of training the old target recognition model is the same as the traditional target recognition training method under static environment settings, which will not be elaborated here.

[0106] In an alternative embodiment, iterative training of the class incremental learning model using the SAR target image training set includes the following steps:

[0107] Step 3a: Initialize the parameters of the new target incremental learning sub-network;

[0108] Optionally, randomly initialize the parameters of the new target incremental learning sub-network based on the normal distribution. Set the iterative training parameters of the class incremental learning model, including the number of iterations q and the maximum number of iterations Q, where Q≥100 and q = 0. In this embodiment, Q = 200.

[0109] Step 3b: Select 2B SAR target images from the incremental learning training set. Among them, B SAR target images are obtained by randomly sampling the incremental learning training set, and the other B SAR target images are obtained by class-balanced sampling of the incremental learning training set;

[0110] Optionally, the B SAR target images obtained by random sampling are denoted as D rad , and the B SAR target images obtained by class-balanced sampling are denoted as D bal .

[0111] Among them, the sampling probability of selecting one sample from the training data of the j-th class is where q∈[0,1], n j represents the number of SAR target images included in the training data of the j-th class. When q = 0, it corresponds to random sampling, and when q = 1, it corresponds to class-balanced sampling. In this embodiment, B = 128.

[0112] Step 3c: Input the 2B SAR target images into the second backbone feature extraction module of the new target incremental learning sub-network to obtain the corresponding features. Input the features corresponding to the randomly sampled samples into the discriminative feature learning branch and calculate the corresponding loss function of the discriminative feature learning branch. Input the features corresponding to the class-balanced sampled samples into the unbiased classifier learning branch and calculate the corresponding loss function of the unbiased classifier learning branch;

[0113] In this embodiment, input D rad and D bal into the second backbone feature extraction module F' of the new target incremental learning sub-network M t and after global average pooling, obtain the corresponding K-dimensional features and Input the features corresponding to the randomly sampled samples The divisible feature learning branch of the input multi-prototype rebalancing module MPR is used to obtain the mapped embedded features And based on Calculate the loss function L of the divisible feature learning branch mp . The features corresponding to the class-balanced sampling samples Are input into the unbiased classifier learning branch of the multi-prototype rebalancing module MPR to obtain the mapped classification scores And based on Calculate the loss function L of the unbiased classifier learning branch ce .

[0114] Step 3d: Input 2B SAR target images into the first backbone feature extraction module of the old target teacher sub-network to obtain the corresponding features, calculate the corresponding feature topology relation distillation loss function based on the features corresponding to the randomly sampled samples, input the features corresponding to the class-balanced sampling samples into the classification module of the old target teacher sub-network, and calculate the corresponding classification layer response distillation loss function according to the predicted classification scores of the old target teacher sub-network and the predicted classification scores of the new target incremental learning sub-network;

[0115] In this embodiment, D rad And D bal Are input into the first backbone feature extraction module F of the old target teacher sub-network M o And after global average pooling, the corresponding K-dimensional features are obtained And Respectively calculate the feature similarity matrix of the features extracted by the old target teacher sub-network M o And the feature similarity matrix of the features extracted by the new target incremental learning sub-network M t And based on Γ o And Γ t Calculate the feature topology relation distillation loss function L frd . The features corresponding to the class-balanced sampling samples frd Are input into the classification module C to obtain the mapped classification scores And based on the old target teacher sub-network M o The predicted classification score O of the class-balanced sampling samples bal And the new target incremental learning sub-network M t The predicted classification score of the class-balanced sampling samples Calculate the loss classification layer response distillation loss function L rd .

[0116] Step 3e: According to the calculated loss functions of the separability feature learning branch, the unbiased classifier learning branch, the feature topological relationship distillation loss function, and the classification layer response distillation loss function, use the gradient descent method to update the parameters of the new target incremental learning sub-network;

[0117] In this embodiment, the gradient descent method is used to calculate the loss function L based on Step 3d mp and L ce and the loss functions L calculated in Step 3e frd and L rd , and update the parameters of each layer of the new target incremental learning sub-network M t of the class incremental learning model according to the loss function of the class incremental learning model, and obtain the updated class incremental recognition model.

[0118] Among them, the loss function of the class incremental learning model is:

[0119] L total = λ1 · [L mp + L frd + (1 - λ1) · [(1 - λ2) · L ce + λ2 · L rd ;

[0120] In the formula, λ1 represents the first balance coefficient, and λ2 represents the second balance coefficient;

[0121] Optionally, the momentum SGD optimizer is used in the training process, the weight regularization term is 1×10 -4 , the momentum factor is 0.9, the initial learning rate is set to 0.5, and it decays to 1 / 10 of the original learning rate at 120 rounds of iteration and 160 rounds of iteration respectively.

[0122] Step 3f: Repeat Steps bc-3e for multiple iterative trainings until the preset training stop condition is reached, complete one incremental update, and obtain the trained class incremental learning model.

[0123] In this embodiment, the preset training stop condition is that the number of iterations q reaches the maximum number of iterations Q, or the loss function of the class incremental learning model reaches the preset threshold.

[0124] It should be noted that after one incremental update is completed, if a new incremental update is to be performed, follow the training process of the class incremental learning model in Steps 3b-3f as described above, and then use the new incremental learning training set to train the class incremental learning model.

[0125] It should be noted that the class incremental learning model based on the knowledge-robust rebalancing network constructed in this embodiment updates and trains the new target incremental recognition model based on a small number of stored old target class samples and the obtained new target class samples. During the update and training process, the key feature knowledge and key classification knowledge for correctly recognizing old targets in the old target recognition model are transferred to the new target incremental recognition model to ensure the robustness of the learned old target knowledge. Based on the rebalancing module, the new target incremental recognition model realizes balanced learning of new and old target class samples, avoiding the problems of poor separability of old target class features proposed by the new target incremental recognition model and preference in classification prediction, and solving the problem that the prior art cannot effectively maintain the recognition performance of old target classes while incrementally learning new target classes.

[0126] Step 4: Use the trained class incremental learning model to achieve target recognition of the SAR image to be measured.

[0127] In an optional implementation manner, the SAR image to be measured is input into the new target incremental learning subnet of the trained class incremental learning model. After the second backbone feature extraction module extracts features from the SAR image to be measured, the extracted features are input into the unbiased classifier learning branch to obtain the predicted classification probability of the SAR image to be measured, and the predicted class of the SAR image to be measured is obtained according to the maximum probability.

[0128] In this embodiment, each of the 2426 images with a radar working pitch angle of 15° in the sample set composed of SAR vehicle target data can be cropped into an image with a pixel of 64×64. The class label of each image belongs to 1-10. Then, all the cropped images are divided into test sets corresponding to 5 groups of training sets according to the class labels for use as SAR images to be measured to test the trained class incremental learning model.

[0129] The SAR target class incremental recognition method based on the knowledge-robust rebalancing network in this embodiment can fully consider maintaining the separability information of old class features and enhancing the feature separability of new and old classes, and while realizing the incremental learning and recognition of new classes by the recognition model, ensure its robustness in recognizing old targets.

[0130] Please refer to Figure 4 the result graph of the recognition accuracy of an embodiment of the present invention provided by the present invention method and the existing class incremental method based on the MSTAR dataset incremental experiment as shown. It can be seen from the figure that the recognition accuracy of the method of the present invention is better than that of the existing class incremental method.

[0131] It should be noted that the above description is only a specific example of the present invention and does not constitute any limitation to the present invention. Obviously, for professionals in this field, after understanding the content and principle of the present invention, various modifications and changes in form and details may be made without departing from the principle and structure of the present invention. However, these corrections and changes based on the idea of the present invention are still within the scope of protection of the claims of the present invention. For example, a possible alternative is to optimize the extraction of hidden layer features of new and old target class samples based on instance contrast learning at the feature level, and describe the feature topological relationship learned by the old target recognition model based on any metric method, but still aim to ensure the robustness of the old target feature relationship and the separability of new and old target features. The improvement of this basic optimization strategy does not depart from the idea of the present invention and is still within the scope of protection of the claims of the present invention.

[0132] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant are intended to cover non-exclusive inclusion, so that an article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of another identical element in the article or device comprising the said element. "Connection" or "connected" and other similar words are not limited to physical or mechanical connection, but may include electrical connection, whether direct or indirect.

[0133] The above content is a further detailed description of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those of ordinary skill in the technical field to which the present invention pertains, without departing from the concept of the present invention, several simple deductions or substitutions can still be made, and all should be regarded as belonging to the protection scope of the present invention.

Claims

1. A SAR target class incremental recognition method based on a knowledge-robust and rebalanced network, characterized in that Including: Obtain the SAR target image training set; Construct a class-incremental learning model based on a knowledge-robust and rebalanced network. The class-incremental learning model includes: an old-target teacher sub-network, a new-target incremental learning sub-network, and a hybrid knowledge distillation module, where The old-target teacher sub-network is used to provide old-target knowledge for the new-target incremental learning sub-network, and includes a cascaded first backbone feature extraction module and a classification module; The new-target incremental learning sub-network is used to incrementally learn new target classes to simultaneously achieve the recognition of new target classes and old target classes, and includes a cascaded second backbone feature extraction module and a multi-prototype rebalancing module. The multi-prototype rebalancing module includes an unbiased classifier learning branch and a separability feature learning branch arranged in parallel; The hybrid knowledge distillation module is used to transfer the knowledge of correctly recognizing old targets from the old-target teacher sub-network to the new-target incremental learning sub-network; The loss function of the class-incremental learning model is: L total = λ1·[L mp + L frd + (1 - λ1)·[(1 - λ2)·L ce + λ2·L rd ; Where, L mp represents the loss function of the separable feature learning branch, L ce represents the loss function of the unbiased classifier learning branch, L frd represents the feature topological relationship distillation loss function, L rd represents the classification layer response distillation loss function; λ1 represents the first balance coefficient, and λ2 represents the second balance coefficient; Iteratively train the class-incremental learning model using the SAR target image training set to obtain a trained class-incremental learning model; Use the trained class-incremental learning model to achieve target recognition of the SAR image to be measured.

2. The SAR target class incremental recognition method based on the knowledge-robust rebalancing network according to claim 1, wherein, Obtaining the SAR target image training set includes: Obtain multiple SAR target images, crop each SAR target image into an image with a pixel size of 64×64, and assign a class label to each SAR target image; Divide the multiple SAR target images into multiple groups according to classes, and each group contains SAR target images of 2 classes; Use one group of images as the initial training set to train the old-target recognition model, and use the other groups of images as the new target class training set to incrementally update the class-incremental learning model. Among them, in each incremental update, select no more than 300 SAR target images of old target classes to jointly form an incremental learning training set with the new target class training set, and train and update the class-incremental learning model.

3. The SAR target class incremental recognition method based on the knowledge-robust rebalancing network according to claim 1, wherein The structures of the first backbone feature extraction module and the second backbone feature extraction module are the same, and both include a cascaded first convolutional layer, multiple residual convolutional layers, and a pooling layer; where The convolutional kernel size of the first convolutional layer is 3×3, and the number of convolutional kernels is set to 64; The residual convolutional layer includes a second convolutional layer, a first Batch Norm layer, a first ReLU non-linear activation function layer, a third convolutional layer, a second Batch Norm layer, and a second ReLU non-linear activation function layer; where The input feature map of the residual convolutional layer sequentially passes through the second convolutional layer, the first Batch Norm layer, and the first ReLU non-linear activation function layer to obtain a first feature map. The first feature map is sequentially mapped through the third convolutional layer and the second Batch Norm layer to obtain a second feature map. The second feature map is added to the downsampled input feature map of the residual convolutional layer and then passes through the second ReLU non-linear activation function layer to obtain the output feature map of the residual convolutional layer; The pooling layer uses global average pooling.

4. The SAR target class incremental recognition method based on the knowledge-robust rebalancing network according to claim 1, wherein The classification module includes a first fully connected layer. The number of input nodes of the first fully connected layer is 512, and the number of output nodes is N, where N represents the total number of old target classes.

5. The SAR target class incremental recognition method based on the knowledge-robust re-balanced network according to claim 4, wherein The unbiased classifier learning branch includes a second fully connected layer. The number of input nodes of the second fully connected layer is 512, and the number of output nodes is N + M, where M represents the total number of newly added target classes. The loss function of the unbiased classifier learning branch is: Wherein, B represents the number of training samples input to the unbiased classifier learning branch during the current batch training process, and y b represents the true class label of the b-th training sample x b , represents the predicted classification score obtained by the second backbone feature extraction module and the unbiased classifier learning branch for the training sample x b , and softmax() represents the normalized exponential function.

6. The SAR target class incremental recognition method based on the knowledge-robust re-balanced network according to claim 5, wherein The separability feature learning branch includes a cascaded third fully connected layer and a fourth fully connected layer. The number of input nodes of the third fully connected layer is 512, and the number of output nodes is 512. The number of input nodes of the fourth fully connected layer is 512, and the number of output nodes is 128. The loss function of the separability feature learning branch is: Where B represents the number of training samples input to the divisibility feature learning branch during the current batch training, represents the d-th training sample x d The embedding feature obtained through the second backbone feature extraction module and the divisibility feature learning branch, y d represents the training sample x d 's true class label, represents the s-th prototype parameter in the set of class prototypes of the j-th class, represents the s-th prototype parameter in the set of class prototypes corresponding to the class y d S represents the total number of prototype parameters in the set of class prototypes for each class.

7. The SAR target class incremental recognition method based on the knowledge-robust rebalancing network according to claim 6, characterized in that, The feature topological relationship distillation loss function is: In the formula, Γ o represents the feature similarity matrix of the sample features extracted by the old target teacher sub-network, and Γ t represents the feature similarity matrix of the sample features extracted by the new target incremental learning sub-network. W mask is the mask matrix; The classification layer response distillation loss function is: In the formula, represents the soft output probability of the new target incremental learning sub-network for the sample x i in the j-th class, and q j (x i ) represents the soft output probability of the old target teacher sub-network for the sample x i in the j-th class.

8. The SAR target class incremental recognition method based on the knowledge-robust rebalancing network according to claim 2, characterized in that Before using the SAR target image training set to iteratively train the class incremental learning model to obtain a trained class incremental learning model, it further includes: Training the old target recognition model using the initial training set, and using the trained old target recognition model as the old target teacher sub-network of the class incremental learning model.

9. The SAR target class incremental recognition method based on the knowledge-robust re-balanced network according to claim 8, characterized in that, Using the SAR target image training set to iteratively train the class incremental learning model to obtain a trained class incremental learning model, including the following steps: Step 3a: Initialize the parameters of the new target incremental learning sub-network. Step 3b: Select 2B SAR target images from the incremental learning training set. Among them, B SAR target images are obtained by randomly sampling the incremental learning training set, and the other B SAR target images are obtained by class-balanced sampling of the incremental learning training set. Step 3c: Input the 2B SAR target images into the second backbone feature extraction module of the new target incremental learning sub-network to obtain corresponding features. Input the features corresponding to the randomly sampled samples into the separability feature learning branch, and calculate the corresponding loss function of the separability feature learning branch. Input the features corresponding to the class-balanced sampled samples into the unbiased classifier learning branch, and calculate the corresponding loss function of the unbiased classifier learning branch. Step 3d: Input the 2B SAR target images into the first backbone feature extraction module of the old target teacher sub-network to obtain corresponding features, and calculate the corresponding feature topological relationship distillation loss function based on the features corresponding to the randomly sampled samples. Input the features corresponding to the class-balanced sampled samples into the classification module of the old target teacher sub-network, and calculate the corresponding classification layer response distillation loss function. Step 3e: According to the calculated loss functions of the separability feature learning branch, the unbiased classifier learning branch, the feature topological relationship distillation loss function, and the classification layer response distillation loss function, use the gradient descent method to update the parameters of the new target incremental learning sub-network. Step 3f: Repeat steps 3b - 3e for multiple iterative trainings until the preset training stop condition is reached, complete one incremental update, and obtain a trained class incremental learning model.

10. The SAR target class incremental recognition method based on the knowledge-robust rebalancing network according to claim 1, characterized in that Using the trained class-incremental learning model to achieve target recognition of the SAR image to be tested, including: Inputting the SAR image to be tested into the new target incremental learning sub-network of the trained class-incremental learning model. After the second backbone feature extraction module extracts features from the SAR image to be tested, the extracted features are input into the unbiased classifier learning branch to obtain the predicted classification probability of the SAR image to be tested, and the predicted category of the SAR image to be tested is obtained according to the maximum probability.

Images