A Class-Incremental Learning Method and System Based on Analogy Learning

Abstract

The present invention proposes a class-incremental learning method and system based on analogical learning, which relates to the field of artificial intelligence technology. By proposing a novel few-shot class-incremental learning method inspired by the human brain's analogical learning mechanism, a new class weight builder is used to calculate the analogy between the new class and all old classes, and the new class classifier is obtained by mixing all old classifiers using the calculated analogy. At the same time, a meta-analogical training module is designed to apply the scenario training strategy to update the parameters of the CNN module and the new class weight builder, as well as the knowledge of each encountered class. The meta-analogical training module applies the scenario training strategy to optimize the feature representations and classifiers of all encountered classes after each few-shot class-incremental learning stage. By combining the meta-analogical training module and the few-shot class-incremental learning method, the class-incremental learning method proposed by the present invention achieves state-of-the-art performance accuracy in typical settings on benchmark image classification datasets.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular, to a class incremental learning method and system based on analogical learning. Background Art

[0002] Deep convolutional neural networks (CNNs) have been successfully applied to a wide range of computer vision and pattern recognition tasks. To achieve excellent performance, CNNs need to be trained using a large-scale training set with a predefined number of image classes. Once the training process is completed, the image classes that the CNN model can recognize, as well as the network performance on a given image set, are fixed. Since CNN models are often deployed in a constantly changing dynamic environment, there is an urgent need to continuously adapt the model to emerging new image classes. In other words, the CNN model must be able to gradually learn new classes from a few training images, i.e., few-shot class incremental learning. A simple method for few-shot class incremental learning is to simply fine-tune the base model using a few new class training images. However, this method leads to two serious problems: 1) catastrophic forgetting: after fine-tuning on new classes, the performance of the model on old classes drops sharply; 2) overfitting: that is, the model overfits to a few new class examples and performs poorly on a large number of test images.

[0003] In recent years, research efforts to solve the catastrophic forgetting problem have been increasing. Some work attempts to repair the feature extractor and only update the classification layer of the CNN model during the class incremental learning process. These methods may perform well in the first few class incremental learning stages, but will perform worse and worse in the long run because the feature extractor of the CNN model is not adapted to new image classes. Some other work uses the so-called knowledge distillation method, stores a set of old class examples, and forces the network to maintain its output logic for these examples during the class incremental learning stage. Although effective, research shows that this method has problems of "category data imbalance" and "forget first and then re-learn". Summary of the Invention

[0004] The purpose of the present invention is to provide a class incremental learning method and system based on analogical learning to improve the problems of catastrophic forgetting and overfitting in the prior art.

[0005] In a first aspect, an embodiment of the present application provides a class incremental learning method based on analogical learning, including the following steps:

[0006] Obtain a class incremental data set containing multiple categories, and divide the class incremental data set into training samples of multiple training stages according to categories, and ensure that the categories in the training samples of different training stages do not overlap;

[0007] Use the feature extractor in the CNN module to train the training samples in the different training stages respectively, learn the feature spaces in the different training stages, obtain the classification weight vectors of all classes in the different training stages, and store the class example sets of each class in the different training stages in the memory bank. ;

[0008] In the incremental stage, for the target class training sets of each training stage, select example images for each class to obtain the class example sets, and calculate the class centroid vectors of the target classes; similarly, calculate the class centroid vectors of all source classes in the memory bank. ;

[0009] Calculate the query matrix using the class centroid vectors of the target classes, calculate the key matrix using the centroid vectors of the source classes, calculate the value matrix using the classification weight vectors of all classes in the different training stages, and use the query matrix, key matrix, and value matrix as the inputs of the new class weight builder to obtain the classification weight vectors of the target classes.

[0010] At the end of the target class stage, adopt the scenario training strategy, use the pseudo-test set for verification to obtain the pseudo-target class classification weight vectors, and backpropagate the errors to update the classification weight vectors of all classes in the different training stages, the parameters of the new class weight builder, and the parameters of the CNN module, and perform the iterative operation for the predefined number of times; the pseudo-test set is obtained by extracting the data in the class example sets.

[0011] Define the multi-objective loss function for the classification weight vectors of the target classes, and optimize and train the class incremental learning model to obtain the trained model.

[0012] Use the trained model to test all the learned classes based on the evaluation metrics to obtain the final class incremental learning model based on analogical learning for image classification.

[0013] Based on the first aspect, in some embodiments of the present invention, calculating the class centroid vectors of all source classes in the memory bank includes the following steps:

[0014] Adopt the Holder algorithm, select E example images for each class in , calculate the class centroid vector , and save it in the memory bank to obtain the class centroid vectors of all source classes in the memory bank ; where is the only accessible target class training set in stage t.

[0015] Based on the first aspect, in some embodiments of the present invention, it further includes the following steps:

[0016] In the subsequent class incremental learning stage, the class centroid vectors are recalculated using the class example set according to the class centroid vector calculation formula to obtain updated class centroid vectors. The class centroid vector calculation formula is: , where is the class centroid vector, E is the number of example images selected for each class in , and is the i-th example image of class c.

[0017] Based on the first aspect, in some embodiments of the present invention, the query matrix is calculated using the class centroid vector of the target class, the key matrix is calculated using the centroid vector of the source class, the value matrix is calculated using the classification weight vectors of all classes in different training stages, and the query matrix, key matrix, and value matrix are used as inputs to a new class weight builder to obtain the classification weight vector of the target class, including the following steps:

[0018] In the class incremental learning stage, a new class weight builder is constructed. For each cross-attention head h, the query matrix is calculated using the class centroid vector of the target class, the key matrix is calculated using the centroid vector of the source class, and the value matrix is calculated using the classification weight vectors of all classes in different training stages;

[0019] Using the query matrix, key matrix, and value matrix as the input of the cross-attention branch h, the classification weight vector of the target class is obtained as the output of the cross-attention branch h;

[0020] The classification weight vectors of the target class output by all the cross-attention branches h are concatenated together, and the classification weight vector of the target class is generated through the MLP in the new class weight builder.

[0021] Based on the first aspect, in some embodiments of the present invention, at the end of the target class stage, a scenario training strategy is adopted, and a pseudo-test set is used for verification to obtain a pseudo-target class classification weight vector, and the error is backpropagated to update the classification weight vectors of all classes in different training stages, the parameters of the new class weight builder, and the parameters of the CNN module, and an iterative operation is performed a predefined number of times; the pseudo-test set is obtained by extracting data from the class example set, including the following steps:

[0022] Using the scenario training method to simulate N-way 1-shot scenario training, the class set is randomly divided to obtain a pseudo-target class set and a pseudo-source class set ;

[0023] Randomly select one example image from each class in the pseudo-target class set to form a pseudo-training set , and the pseudo target set The remaining images in are used as pseudo test sets ;

[0024] Using a fake test set Verify and get the pseudo target class classification weight vector ;

[0025] By classifying all the pseudo target classes into a weight vector and the classification weight matrix of the pseudo source class set Combined, further build a unified classifier for all classes ;

[0026] On the pseudo test set Test the unified classifier on , calculating the loss function, and then back-propagating the error to update the classification weight vectors of all classes in the different training stages, the parameters of the new class weight builder, and the parameters of the CNN module;

[0027] The updated CNN module is used to recalculate the feature vectors and class centroid sets of the training samples at each training stage and perform a predefined number of iterations.

[0028] Based on the first aspect, in some embodiments of the present invention, the multi-objective loss function includes a new class weight builder loss function and a CNN loss function,

[0029] The new class weight builder loss function is expressed as: ,in, is the matching loss function, is the cross entropy loss function for classification, is the hyperparameter used for balancing;

[0030] The CNN loss function is expressed as: ,in, is the cross entropy loss function, To reduce the loss of forgetfulness, To reduce overfitting loss, 、 is the hyperparameter used for balancing.

[0031] Based on the first aspect, in some embodiments of the present invention, using the trained model to test all learned categories based on evaluation indicators to obtain a final analogy-based incremental learning model for image classification includes the following steps:

[0032] Using the trained model, fine-tune the CNN module, update the new class weight builder using an optimizer, and test all learned classes based on evaluation metrics to obtain a final class incremental learning model for image classification based on analogical learning.

[0033] In a second aspect, an embodiment of the present application provides a class incremental learning system based on analogical learning, including:

[0034] A class incremental dataset partitioning module, configured to obtain a class incremental dataset containing multiple classes, and divide the class incremental dataset into training samples for multiple training stages by category, and ensure that the classes in the training samples of different training stages do not overlap;

[0035] A feature space extraction module, configured to use the feature extractor in the CNN module to train the training samples of different training stages respectively, learn the feature spaces of different training stages, obtain the classification weight vectors of all classes in different training stages, and store the class example sets of each class in different training stages into the memory bank in;

[0036] A class centroid vector calculation module, configured to, in the incremental stage, for the target class training sets of each training stage, select example images for each class to obtain a class example set, and calculate the class centroid vector of the target class; similarly calculate the class centroid vectors of all source classes in the memory bank in;

[0037] A new class weight builder module, configured to calculate a query matrix using the class centroid vector of the target class, calculate a key matrix using the centroid vectors of the source classes, calculate a value matrix using the classification weight vectors of all classes in different training stages, and use the query matrix, key matrix, and value matrix as inputs to the new class weight builder to obtain the classification weight vector of the target class;

[0038] A pseudo-test set verification module, configured to, at the end of the target class stage, adopt a scenario training strategy, verify using a pseudo-test set to obtain a pseudo-target class classification weight vector, and backpropagate the error to update the classification weight vectors of all classes in different training stages, the parameters of the new class weight builder, and the parameters of the CNN module, and perform an iterative operation for a predefined number of times; the pseudo-test set is obtained by extracting data from the class example set;

[0039] An optimization training module, configured to define a multi-objective loss function for the classification weight vector of the target class, and perform optimization training on the class incremental learning model to obtain a trained model;

[0040] A model testing module, which is used to adopt the trained model to test all learned categories based on evaluation metrics, so as to obtain a final class incremental learning model based on analogical learning for image classification.

[0041] In a third aspect, an embodiment of the present application provides an electronic device, which includes a memory for storing one or more programs; and a processor. When the one or more programs are executed by the processor, the method described in any one of the above first aspects is implemented.

[0042] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in any one of the above first aspects is implemented.

[0043] The embodiments of the present invention have at least the following advantages or beneficial effects:

[0044] The embodiments of the present invention provide a class incremental learning method and system based on analogical learning. By proposing a novel few-shot class incremental learning method (ALFSCIL) inspired by the human brain's analogical learning mechanism, this framework uses a new class weight constructor (NCCC) to calculate the analogy between new classes and all old classes, and obtains new class classifiers by mixing all old class classifiers using the calculated analogy. At the same time, a meta-analogical training module (MAT) is designed to apply a scenario training strategy to update the parameters of the CNN module and the new class weight constructor (NCCC), as well as the knowledge of each encountered class. The meta-analogical training module applies the scenario training strategy to optimize the feature representations and classifiers of all encountered classes after each round of few-shot class incremental learning stage. By combining the meta-analogical training module (MAT) and the few-shot class incremental learning method (ALFSCIL), the class incremental learning method proposed by the present invention achieves state-of-the-art performance accuracy in typical settings on benchmark image classification datasets. Description of the Drawings

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0046] Figure 1 It is a flowchart of a class incremental learning method based on analogical learning provided by an embodiment of the present invention;

[0047] Figure 2 It is the overall architecture and data flow diagram of a few-shot class incremental learning framework provided by an embodiment of the present invention;

[0048] Figure 3 Detailed configuration information diagram of the new class classifier module (NCCC) provided by the embodiments of the present invention;

[0049] Figure 4 Comparison diagram of the top-1 accuracy curves of the present invention with other sub-optimal benchmark methods on the miniImageNet and CUB200 image classification datasets based on ResNet18 provided by the embodiments of the present invention;

[0050] Figure 5 Structural block diagram of a class incremental learning system based on analogical learning provided by the embodiments of the present invention;

[0051] Figure 6 Structural block diagram of an electronic device provided by the embodiments of the present invention.

[0052] Icons: 110 - Class incremental dataset division module; 120 - Feature space extraction module; 130 - Class centroid vector calculation module; 140 - New class weight builder module; 150 - Pseudo test set verification module; 160 - Optimization training module; 170 - Model test module; 101 - Memory; 102 - Processor; 103 - Communication interface. Detailed implementation manners

[0053] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. Usually, the components of the embodiments of the present application described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations.

[0054] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application claimed, but merely represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts fall within the scope of protection of the present application.

[0055] The following will describe in detail some implementation manners of the present application with reference to the accompanying drawings. Without conflict, the following various embodiments and the various features in the embodiments can be combined with each other.

[0056] Deep convolutional neural networks (CNNs) have been successfully applied to a wide range of computer vision and pattern recognition tasks. To achieve excellent performance, CNNs need to be trained using large-scale training sets with a predefined number of image classes. Once the training process is completed, the image classes that the CNN model can recognize, as well as the network performance on a given image set, are fixed. Since CNN models are often deployed in dynamic environments that are constantly changing, there is an urgent need to continuously adapt the model to emerging new image classes. In other words, the CNN model must be able to gradually learn new classes from very few training images, i.e., few-shot class incremental learning. Driven by real-world demands, research on class incremental learning has received increasing attention in recent years. It attempts to learn new knowledge from new class training instances and retain the learned knowledge of old classes. A simple approach to few-shot class incremental learning is to simply fine-tune the base model using a few new class training images. However, this approach leads to two serious problems: 1) catastrophic forgetting: after fine-tuning on new classes, the performance of the model on old classes drops sharply; 2) overfitting: that is, the model overfits to a few new class examples and performs poorly on a large number of test images.

[0057] Inspired by the latest developments in brain cognitive science, the present invention provides a class incremental learning method based on analogical learning, which solves the problems of catastrophic forgetting and overfitting.

[0058] Please refer to Figures 1-3 , Figure 1 which is a flowchart of a class incremental learning method based on analogical learning provided by an embodiment of the present invention, Figure 2 which is the overall architecture and data flow diagram of a few-shot class incremental learning framework provided by an embodiment of the present invention, Figure 3 which is a detailed configuration information diagram of a new class classifier module (NCCC) provided by an embodiment of the present invention.

[0059] An embodiment of the present invention provides a class incremental learning method based on analogical learning, including the following steps:

[0060] Step S110: Obtain a class incremental data set containing multiple classes, and divide the class incremental data set into training samples of multiple training stages according to classes, and ensure that the classes in the training samples of different training stages do not overlap; in this embodiment, the class incremental data set includes a labeled training set, a test set, and a class label set.

[0061] Step S120: Use the feature extractor in the CNN module to train the training samples of different training stages respectively, learn the feature spaces of different training stages, obtain the classification weight vectors of all classes in different training stages, and store the class example sets of each class in different training stages into the memory bank Among them; the above-mentioned feature space discovers features extracted from the original data from the above-mentioned feature extraction, maps the original data to a higher-dimensional space, and the features in the feature space are higher-dimensional abstractions of the original data.

[0062] The above CNN module can be defined as a combination of a feature extractor parameterized by and a classification layer (such as a classification weight matrix) parameterized by W. By definition, the CNN module outputs a class prediction vector expressed as: , where x is the given input image.

[0063] Step S130: In the incremental stage, for the target class training set of each training stage, select example images for each class to obtain a class example set, and calculate the class centroid vector of the target class; similarly, calculate the class centroid vectors of all source classes in the memory bank ; the above target class training set refers to the set of images output after incremental training in each training stage. The calculated class centroid vector of the target class is the calculation of the class centroid vector of the new class, and the calculation of the class centroid vectors of all source classes is the calculation of the class centroid vectors of the old classes.

[0064] Among them, in order to save the knowledge of each observed class in stage t, the class centroid vectors of all source classes in the memory bank can be calculated. It should be noted that the calculation process of the class centroid vector of the target class is similar to that of the class centroid vectors of all source classes, but the class centroid vector of the target class is not calculated using example samples, but is calculated from all training samples of the target class. The calculation of the class centroid vectors of all source classes in the memory bank includes the following steps:

[0065] Adopt the Holder algorithm to select E example images for each class in and calculate the class centroid vector , and the calculation formula is: , where is the class centroid vector, E is the number of example images selected for each class in , is the i-th example image of class c; and save it in the memory bank [[ID=;37]] to obtain the class centroid vectors of all source classes in the memory bank ; among them, is the only accessible target class training set in stage t.

[0066] In the subsequent class incremental learning stage, whenever When an update is needed, since the training images of class c are no longer accessible, the class centroid vectors can be recalculated according to the class centroid vector calculation formula using the class example set to obtain updated class centroid vectors. The class centroid vector calculation formula is as follows:

[0067] , where is the class centroid vector, E is the number of example images selected for each class in , and is the i-th example image of class c. By updating the class centroid vectors, the knowledge of the classes can be updated in real time, thereby improving the accuracy of class incremental learning.

[0068] Step S140: Calculate a query matrix using the class centroid vector of the target class, calculate a key matrix using the centroid vector of the source class, calculate a value matrix using the classification weight vectors of all classes at different training stages, and use the query matrix, key matrix, and value matrix as inputs to a new class weight builder to obtain the classification weight vector of the target class;

[0069] Among them, the process of obtaining the classification weight vector of the target class specifically includes the following steps:

[0070] First, in the class incremental learning stage, a new class weight builder is constructed. For each cross-attention head h, calculate a query matrix using the class centroid vector of the target class, calculate a key matrix using the centroid vector of the source class, and calculate a value matrix using the classification weight vectors of all classes at different training stages; the new class classifier module (NCCC) is formed by a multi-head cross-attention (MHCA) transformer.

[0071] It should be noted that in the class incremental learning stage when t > 0, a new class weight builder NCCC module is designed, and the calculation of the above query matrix, key matrix, and value matrix is as follows:

[0072] ,

[0073] ,

[0074] ,

[0075] where, is the class centroid feature vector of all target classes, is the query matrix, is the class centroid feature vector of all source classes, is the key matrix, is the classification weight vector of all source classes, is the value matrix,<y_h^t 、 and They are three learned linear transformations of the cross-attention head h.

[0076] Then, using the query matrix, key matrix, and value matrix as the input of the cross-attention branch h, a classification weight vector for the target class is obtained from the output of the cross-attention branch h; specifically, using , and as the input, the classification weight vector for the target class output by the cross-attention branch h is as follows:

[0077]

[0078] where is the classification weight vector for the target class output by the cross-attention branch h, and d is the dimension of the classification weight vector.

[0079] Finally, all the classification weight vectors for the target class output by the cross-attention branch h are concatenated together, and a classification weight vector for the target class is generated through the MLP in the new class weight builder.

[0080] where from all cross-attention branches h are concatenated together and passed through the MLP to produce the final output :

[0081]

[0082] where each vector is is the classification weight vector for the target class c.

[0083] In the class incremental learning stage t, a new training set is available. can be used as the target class , and all the classes learned in the previous stages are used as the source classes . The class centroid of each class is calculated to form . Using , the set of class centroids of all source classes and the set of classification weights as the input, NCCC calculates the set of classification weights for all target classes.

[0084] Step S150: At the end of the target class stage, adopt a scenario training strategy, use the pseudo-test set for verification to obtain a pseudo-target class classification weight vector, and backpropagate the error to update the classification weight vectors of all classes in different training stages, the parameters of the new class weight builder, and the parameters of the CNN module, and perform an iterative operation for a predefined number of times; the pseudo-test set is obtained by extracting data from the class example set; specifically, it includes the following steps:

[0085] First, in order to enable the NCCC module to generate reasonable classification weights for unknown classes, the parameters of NCCC can be optimized using the N-way K-shot scenario training method to simulate the incremental environment and enhance its generalization ability. Using the scenario training method to simulate the N-way K-shot scenario training, the class set is randomly split to obtain the pseudo-target class set and the pseudo-source class set ; the designed Meta-Analogy Training (MAT) module of the scenario training method is used to simulate the N-way K-shot scenario training.

[0086] Then, a sample image is randomly selected from each class in the pseudo-target class set to form the pseudo-training set , and the remaining images in the pseudo-target class set are used as the pseudo-test set ; a sample image is randomly selected from each class to form the pseudo-training set , and the remaining samples in are used to form the pseudo-test set . Finally, is used to calculate the query matrix , and the class centroid vectors and classification weight vectors of all pseudo-source classes are used to calculate the key matrix and value matrix as follows:

[0087] ,

[0088] ,

[0089] .

[0090] Where, is the class centroid vector of all pseudo-source classes, is the query matrix, is the pseudo-training set, is the key matrix, is the classification weight vector of all pseudo-source classes, is the value matrix, , and are three learning linear transformations of the cross-attention head h.

[0091] Then, the pseudo-test set is used for verification to obtain the pseudo-target class classification weight vector ; the above verification means that the pseudo-test set obtained in the above steps The query matrix, key matrix, and value matrix in .

[0092] Then, by combining all the pseudo-target class classification weight vectors with the classification weight matrix of the pseudo-source class set , a unified classifier is further constructed for all classes ;

[0093] Then, the unified classifier is tested on the pseudo-test set , the loss function is calculated, and then the error is backpropagated to update the classification weight vectors of all classes, the parameters of the new class weight builder, and the parameters of the CNN module in the different training stages;

[0094] Finally, the updated CNN module is used to recalculate the feature vectors and class centroid sets of the training samples in each training stage, and an iterative operation is performed a predefined number of times.

[0095] Step S160: For the classification weight vector of the target class, a multi-objective loss function is defined, and the class incremental learning model is optimized and trained to obtain the trained model; for the classification results of the training samples obtained in step S140 at different stages, a multi-objective loss function is defined, and the optimization training task in step S150 is used at the same time.

[0096] Among them, in order to fine-tune the NCCC module, a multi-objective loss function is defined, and the multi-objective loss function includes a new class weight builder loss function and a CNN loss function,

[0097] The new class weight builder loss function is expressed as: , where is the matching loss function, is the cross-entropy loss function for classification, is the hyperparameter for balancing;

[0098] The above cross-entropy loss function for classification can be expressed as:

[0099]

[0100] where represents the softmax function, is the corresponding estimated likelihood vector;

[0101] To ensure that the newly generated classifier is not very different from the old classifier, and Denote the old and new classification weight vectors of target class c obtained in different sessions, and the matching loss can be written as:

[0102]

[0103] where is the weight vector of class c in the new classifier, is the weight vector of class c in the old classifier.

[0104] For the fine-tuning of the CNN module, use the combination of cross-entropy loss , less forgetting loss and less overfitting loss to fine-tune the CNN module. The CNN loss function is expressed as: , where is the cross-entropy loss function, is the less forgetting loss, is the less overfitting loss, , is the hyperparameter for balancing.

[0105] The less forgetting loss is defined as follows:

[0106]

[0107] where is the example sample, is the feature extractor parameter, is the example sample set.

[0108] The less overfitting loss is defined as follows:

[0109]

[0110] where is the classification weight vector of class c, is the classification weight vector of class y, where y is the label of training sample x, and the hyperparameter defines the minimum distance between training samples of different classes.

[0111] Step S170: Use the trained model to test all learned classes based on the evaluation metrics, and obtain the final class-incremental learning model based on analogical learning for image classification. The above evaluation can be to use the trained model to evaluate all learned classes based on top-1 accuracy, performance degradation rate, and average accuracy as evaluation metrics, and obtain the final class-incremental learning model based on analogical learning for image classification tasks.

[0112] Among them, in the MAT stage, the CNN module is fine-tuned. Using the trained model, the CNN module is fine-tuned, the optimizer is used to update the new class weight builder, and all learned classes are tested based on evaluation metrics to obtain a final class incremental learning model based on analogical learning for image classification.

[0113] In the above implementation process, a class incremental dataset containing multiple classes is obtained, and the dataset is divided into multiple training stages by category, and it is ensured that there is no overlap between the new classes in different training stages; the CNN module feature extractor is used to train the training samples in different stages, learn the feature spaces of different stages, obtain the classification weight vectors of all classes in different stages, and store the class example set of each class in the memory bank. In the incremental stage, for the target class training set, example images are selected for each class to obtain a class example set, and its class centroid vector is calculated. Similarly, the class centroid vectors of all source classes in the memory bank are calculated. In the memory bank, the query matrix is calculated using the class centroid vector of the target class, the key matrix is calculated using the centroid vectors of the source classes, and the value matrix is calculated using the classification weight vectors of all source classes. The three matrices obtained are used as the input of the new class weight builder to obtain the classification weight vector of the target class. At the end of the target class stage, scenario training is simulated, the classification weight of the target class obtained from the pseudo-training set is verified on the pseudo-test set, the error is backpropagated to update the classification weight set, the parameters of the new class weight builder and the CNN module, and the predefined number of iterative operations are performed; for the classification results of the training samples in different stages, a multi-objective loss function is defined to optimize the training task simultaneously; the trained model is used as an evaluation metric to evaluate all learned classes, and a final class incremental learning model based on analogical learning is obtained for the image classification task. A novel few-shot class incremental learning method (ALFSCIL) inspired by the human brain analogical learning mechanism is proposed. This framework uses a new class weight builder (NCCC) to calculate the analogy between the new class and all old classes, and obtains the new class classifier by mixing all old classifiers using the calculated analogy. At the same time, a meta-analogical training module (MAT) is designed to apply the scenario training strategy to update the parameters of the CNN module and the new class weight builder (NCCC), as well as the knowledge of each encountered class. This meta-analogical training module applies the scenario training strategy to optimize the feature representations and classifiers of all encountered classes after each round of few-shot class incremental learning stage. By combining the meta-analogical training module (MAT) and the few-shot class incremental learning method (ALFSCIL), the class incremental learning method proposed in the present invention achieves state-of-the-art performance accuracy in the typical settings of benchmark image classification datasets.

[0114] Please refer to Figure 4 , Figure 4This is a comparison graph of the top-1 accuracy curves between the present invention and other sub-optimal benchmark methods on the miniImageNet and CUB200 image classification datasets based on ResNet18 provided by the embodiments of the present invention. Figure 4 Among them, (a) shows the comparison results of the present invention on the miniImageNet dataset. It can be clearly seen that the average accuracy of the present invention is significantly improved by 2.53%. Figure 4 Among them, (b) shows the comparison results of the present invention on the CUB200 dataset. It can be clearly seen that the average accuracy of the present invention is significantly improved by 1.79%.

[0115] Based on the same inventive concept, the present invention also proposes a class-incremental learning system based on analogical learning. Please refer to Figure 5 , Figure 5 This is a structural block diagram of a class-incremental learning system based on analogical learning provided by the embodiments of the present invention. The class-incremental learning system based on analogical learning includes:

[0116] A class-incremental dataset partitioning module 110, which is used to obtain a class-incremental dataset containing multiple categories, and divide the class-incremental dataset into training samples of multiple training stages according to categories, and ensure that the categories in the training samples of different training stages do not overlap;

[0117] A feature space extraction module 120, which is used to train the training samples of different training stages respectively by using a feature extractor in a CNN module, learn the feature spaces of different training stages, obtain classification weight vectors of all classes in different training stages, and store the class instance sets of each class in different training stages in a memory bank ;

[0118] A class centroid vector calculation module 130, which is used in the incremental stage to select example images for each class of the target class training set for each training stage to obtain a class instance set, and calculate the class centroid vector of the target class; similarly calculate the class centroid vectors of all source classes in the memory bank;

[0119] A new class weight constructor module 140, which is used to calculate a query matrix by using the class centroid vector of the target class, calculate a key matrix by using the centroid vectors of the source classes, calculate a value matrix by using the classification weight vectors of all classes in different training stages, and use the query matrix, key matrix and value matrix as the inputs of the new class weight constructor to obtain the classification weight vector of the target class;

[0120] The pseudo-test set verification module 150 is used to, at the end of the target class phase, adopt a scenario training strategy, verify using the pseudo-test set, obtain a pseudo-target class classification weight vector, and backpropagate the error to update the classification weight vectors of all classes in different training phases, the parameters of the new class weight builder, and the parameters of the CNN module, and perform an iterative operation for a predefined number of times; the pseudo-test set is obtained by extracting data from the class example set;

[0121] The optimization training module 160 is used to define a multi-objective loss function for the classification weight vector of the target class and optimize and train the class incremental learning model to obtain a trained model;

[0122] The model testing module 170 is used to adopt the trained model to test all learned categories based on evaluation metrics to obtain a final class incremental learning model for image classification based on analogy learning.

[0123] In the above implementation process, the class incremental dataset division module 110 obtains a class incremental dataset containing multiple categories, and divides the dataset into multiple training phases by category, and ensures that there is no overlap between the new classes in different training phases; the feature space extraction module 120 uses the CNN module feature extractor to train the training samples in different phases, learns the feature spaces in different phases, obtains the classification weight vectors of all classes in different phases, and stores the class example set of each class in the memory bank ; the class centroid vector calculation module 130, in the incremental phase, for the target class training set, selects example images for each class to obtain a class example set, and calculates its class centroid vector. Similarly, the memory bank is calculated The centroids of all source classes; the new class weight builder module 140 calculates the query matrix using the centroid vector of the target class, calculates the key matrix using the centroid vectors of the source classes, calculates the value matrix using the classification weight vectors of all source classes, takes the three calculated matrices as the input of the new class weight builder, and obtains the classification weight vector of the target class. The pseudo-test set verification module 150 simulates scenario training at the end of the target class stage, verifies the classification weight of the target class obtained from the pseudo-training set on the pseudo-test set, updates the classification weight set, the parameters of the new class weight builder and the CNN module, and performs iterative operations for a predefined number of times; the optimization training module 160 defines a multi-objective loss function for the classification results of training samples at different stages and optimizes the training task simultaneously; the model testing module 170 uses the trained model as an evaluation metric to evaluate all learned classes, and obtains a final class incremental learning model based on analogical learning for image classification tasks. A novel few-shot class incremental learning method (ALFSCIL) inspired by the human brain's analogical learning mechanism is proposed. This framework uses a new class weight builder (NCCC) to calculate the analogy between the new class and all old classes, and obtains the new class classifier by mixing all old classifiers using the calculated analogy. At the same time, a meta-analogy training module (MAT) is designed to apply the scenario training strategy to update the parameters of the CNN module and the new class weight builder (NCCC), as well as the knowledge of each encountered class. This meta-analogy training module applies the scenario training strategy to optimize the feature representations and classifiers of all encountered classes after each few-shot class incremental learning stage. By combining the meta-analogy training module (MAT) and the few-shot class incremental learning method (ALFSCIL), the class incremental learning method proposed in the present invention achieves state-of-the-art performance accuracy in typical settings on benchmark image classification datasets.

[0124] Please refer to Figure 6 , Figure 6 which is a schematic structural block diagram of an electronic device provided by an embodiment of the present application. The electronic device includes a memory 101, a processor 102, and a communication interface 103. The memory 101, the processor 102, and the communication interface 103 are directly or indirectly electrically connected to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The memory 101 can be used to store software programs and modules, such as the program instructions / modules corresponding to a class incremental learning system based on analogical learning provided by an embodiment of the present application. The processor 102 executes various functional applications and data processing by executing the software programs and modules stored in the memory 101. The communication interface 103 can be used to communicate signaling or data with other node devices.

[0125] Among them, the memory 101 can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electric Erasable Programmable Read-Only Memory (EEPROM), etc.

[0126] The processor 102 can be an integrated circuit chip with signal processing capabilities. The processor 102 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0127] It can be understood that Figure 6 the structure shown is only schematic, and the electronic device may also include more or fewer components than those Figure 6 shown, or have a configuration different from that Figure 6 shown. Figure 6 Each component shown can be implemented using hardware, software, or a combination thereof.

[0128] In the embodiments provided in the present application, it should be understood that the disclosed systems and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0129] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part.

[0130] If the above functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0131] The above are only the preferred embodiments of the present application and are not used to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

[0132] For those skilled in the art, it is obvious that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or basic characteristics of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present application is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any reference signs in the claims should not be construed as limiting the claims involved.

Claims

1. A class incremental learning method based on analogical learning, characterized in that, It includes the following steps: Obtain a class incremental dataset including multiple categories, and divide the class incremental dataset into training samples of multiple training stages according to categories, and ensure that the categories in the training samples of different training stages do not overlap; Use the feature extractor in the CNN module to train the training samples in the different training stages respectively, learn the feature spaces in their different training stages, obtain the classification weight vectors of all classes in the different training stages, and store the class example sets of each class in the different training stages in the memory bank ; In the incremental stage, for the target class training sets of each training stage, example images are selected for each class to obtain a class example set, and the class centroid vector of the target class is calculated; similarly, the class centroid vectors of all source classes in the memory bank are calculated. ​ Calculate a query matrix using the class centroid vector of the target class, calculate a key matrix using the centroid vector of the source class, calculate a value matrix using the classification weight vectors of all classes in different training stages, and use the query matrix, key matrix, and value matrix as the input of the new class weight builder to obtain the classification weight vector of the target class; At the end of the target class stage, adopt a scenario training strategy, use a pseudo-test set for verification to obtain a pseudo-target class classification weight vector, and backpropagate the error to update the classification weight vectors of all classes in different training stages, the parameters of the new class weight builder, and the parameters of the CNN module, and perform a predefined number of iteration operations; the pseudo-test set is obtained by extracting data from the class example set; For the classification weight vector of the target class, define a multi-objective loss function and optimize and train the class incremental learning model to obtain a trained model; Adopt the trained model to test all learned categories based on evaluation metrics to obtain a final class incremental learning model based on analogical learning for image classification.

2. The class incremental learning method based on analogical learning according to claim 1, wherein The calculated memory bank The centroid vectors of all source classes therein, including the following steps: Using the Holder algorithm, for each class in , select E example images, calculate the class centroid vector , and save it in the memory bank to obtain the class centroid vectors of all source classes in the memory bank ; where is the only accessible target class training set in stage t.

3. The class incremental learning method based on analogical learning according to claim 2, characterized in that It also includes the following steps: In the subsequent class incremental learning stage, the class centroid vectors are recalculated according to the class centroid vector calculation formula using the class example set to obtain updated class centroid vectors. The class centroid vector calculation formula is as follows: , where is the class centroid vector, E is the number of example images selected for each class in , and is the i-th example image of class c.

4. The class incremental learning method based on analogical learning according to claim 3, wherein The step of calculating a query matrix using the class centroid vector of the target class, calculating a key matrix using the centroid vector of the source class, calculating a value matrix using the classification weight vectors of all classes in different training stages, and using the query matrix, key matrix, and value matrix as the input of the new class weight builder to obtain the classification weight vector of the target class includes the following steps: In the class incremental learning stage, construct a new class weight builder. For each cross-attention head h, calculate a query matrix using the class centroid vector of the target class, calculate a key matrix using the centroid vector of the source class, and calculate a value matrix using the classification weight vectors of all classes in different training stages; Use the query matrix, key matrix, and value matrix as the input of the cross-attention branch h to obtain the classification weight vector of the target class output by the cross-attention branch h; Connect the classification weight vectors of the target class output by all the cross-attention branches h together, and generate the classification weight vector of the target class through the MLP in the new class weight builder.

5. The class incremental learning method based on analogical learning according to claim 1, characterized in that The step of adopting a scenario training strategy at the end of the target class stage, using a pseudo-test set for verification to obtain a pseudo-target class classification weight vector, and backpropagating the error to update the classification weight vectors of all classes in different training stages, the parameters of the new class weight builder, and the parameters of the CNN module, and performing a predefined number of iteration operations; the pseudo-test set is obtained by extracting data from the class example set includes the following steps: Simulate N-way 1-shot scenario training using the scenario training method, and randomly divide the class set to obtain a pseudo-target class set and a pseudo-source class set ; Randomly select an example image from each class in the pseudo-target class set to form a pseudo-training set , and use the remaining images in the pseudo-target class set as a pseudo-test set ; Using a pseudo-test set for verification to obtain a pseudo-target class classification weight vector ; By combining all the classification weight vectors of the pseudo-target classes with the classification weight matrix of the pseudo-source class set to further construct a unified classifier for all classes ; Testing the unified classifier on the pseudo-test set and calculating the loss function, and then backpropagating the error to update the classification weight vectors of all classes in the different training phases, the parameters of the new class weight builder, and the parameters of the CNN module; ​ Use the updated CNN module to recalculate the feature vectors and class centroid sets of the training samples in each training stage, and perform a predefined number of iteration operations.

6. The class incremental learning method based on analogical learning according to claim 1, wherein The multi-objective loss function includes a new class weight builder loss function and a CNN loss function, The new class weight constructor loss function is expressed as: , where is the matching loss function, is the cross-entropy loss function for classification, is the hyperparameter for balancing; The CNN loss function is expressed as: , where is the cross-entropy loss function, is the less forgetting loss, is the less overfitting loss, , are hyperparameters for balancing.

7. The method for class incremental learning based on analogical learning according to claim 1, characterized in that, Using the trained model, testing all learned categories based on evaluation metrics, and obtaining a final category incremental learning model based on analogical learning for image classification, including the following steps: Using the trained model, fine-tuning the CNN module, updating the new class weight builder using an optimizer, and testing all learned categories based on evaluation metrics to obtain a final category incremental learning model based on analogical learning for image classification.

8. An analog learning-based class incremental learning system, characterized in that Including: A class incremental dataset partitioning module for obtaining a class incremental dataset containing multiple categories, dividing the class incremental dataset into training samples for multiple training stages by category, and ensuring that the categories in the training samples of different training stages do not overlap; A feature space extraction module, which is used to train the training samples in different training phases respectively by using the feature extractor in the CNN module, learn the feature spaces in different training phases, obtain the classification weight vectors of all classes in different training phases, and store the class example sets of each class in different training phases into the memory bank ; The centroids of classes calculation module is used to, in the incremental stage, for the target class training sets of each training stage, select example images for each class to obtain the class example sets, and calculate the centroid vectors of the target classes; similarly, calculate the centroid vectors of all source classes in the memory bank; A new class weight builder module for calculating a query matrix using the class centroid vector of the target class, calculating a key matrix using the centroid vector of the source class, calculating a value matrix using the classification weight vectors of all classes in different training stages, and using the query matrix, key matrix, and value matrix as inputs to the new class weight builder to obtain the classification weight vector of the target class; A pseudo-test set verification module for, at the end of the target class stage, using a scenario training strategy to verify using a pseudo-test set to obtain a pseudo-target class classification weight vector, backpropagating the error to update the classification weight vectors of all classes in different training stages, the parameters of the new class weight builder, and the parameters of the CNN module, and performing an iterative operation a predefined number of times; the pseudo-test set is obtained by extracting data from the class example set; An optimization training module for defining a multi-objective loss function for the classification weight vector of the target class and optimizing and training the class incremental learning model to obtain a trained model; A model testing module for using the trained model to test all learned categories based on evaluation metrics to obtain a final category incremental learning model based on analogical learning for image classification.

9. An electronic device, characterized in that, Including: A memory for storing one or more programs; A processor; When the one or more programs are executed by the processor, the method described in any one of claims 1-7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the method described in any one of claims 1-7 is implemented.

Images