Prototype-recalling and asymmetric knowledge aggregation based class-incremental learning method and system

Abstract

The application discloses a kind of class incremental learning method based on prototype recall and asymmetric knowledge aggregation, the class of current task is expanded by rotation transformation, and enhanced data is provided to feature extractor.For each old class, store a representative class prototype, then the old class feature generated by the combination of the extracted new class feature and the stored old class prototype, then the generated old class feature is jointly trained with the feature extracted from new data to realize prototype recall.After that, the knowledge on the classifier enhanced by self-supervised label learning is aggregated and transferred to another classifier that only recognizes unenhanced classes.The prototype recall method proposed in the application can model the feature distribution of previous data in a continuously updated representation space, thereby suppressing catastrophic forgetting.The asymmetric knowledge aggregation method proposed can learn task-independent representations and fully capture newly acquired knowledge, improving the plasticity of the incremental learner.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image classification in deep learning, and relates to a category incremental learning method and system, in particular to a category incremental learning method and system based on prototype reminiscence and asymmetric knowledge aggregation. BACKGROUND

[0002] The category incremental learning method is an important method in deep learning, which enables a deep model to have the ability to continuously learn new tasks under the condition of continuous updating of input flow. In data mining, large-scale data set processing and other problems, the category incremental learning method has great advantages. Therefore, category incremental learning is a practical and valuable technology, which enables a deep learning model to continuously learn new knowledge from dynamic data flow. However, the category incremental learning algorithm usually encounters problems such as catastrophic forgetting and poor model plasticity. In addition, the existing category incremental learning method usually relies on saving a small number of old task samples to alleviate catastrophic forgetting, but this will bring additional storage overhead and the risk of data privacy leakage.

[0003] Recently, some researches have discussed the prototype-based sample category incremental learning method (documents 1, 2, 3, 4), which to some extent solves the problems of data privacy security and memory limitation, and plays a role in inhibiting catastrophic forgetting. In order to enhance the prototype, some researches use the remembered prototype and new data to update the model, narrowing the decision boundary of the old class. Some researches also introduce Gaussian noise to limit the decision boundary of the old class. However, updating the model on continuous data flow may cause the representation of the old class to change, making the preserved prototype gradually outdated. At the same time, since the knowledge update is not considered, the feature distribution simulated by the above strategies cannot adapt to such changes, resulting in the overlap of the distributions between different classes, especially between new and old classes.

[0004] In addition to dealing with catastrophic forgetting, improving the plasticity of the model is also a problem worth considering. Previous researches (documents 1, 5, 6) have made great progress in learning generalizable and transferable representations using self-supervised label augmentation. However, these researches all ignore the influence of fully utilizing the information of new data on improving the plasticity of the model, resulting in poor model plasticity.

[0005] In summary, the category incremental learning still has great room for improvement in reducing storage costs, weakening catastrophic forgetting, and improving model plasticity.

[0006] [Document 1] Fei Zhu, Xu-Yao Zhang, Chuang Wang, Fei Yin, and Cheng Lin Liu. Prototype augmentation and self-supervision for incremental learning. In CVPR, pages 5871-5880, 2021.

[0007] [Document 2] Lu Yu, Bartlomiej Twardowski, Xialei Liu, Luis Herranz, Kai Wang, Yongmei Cheng, Shangling Jui, and Joost van de Weijer. Semantic drift compensation for class-incremental learning. In CVPR, pages 6982-6991, 2020.

[0008] [Document 3] Kai Zhu, Wei Zhai, Yang Cao, Jiebo Luo, and Zheng Jun Zha. Self-sustaining representation expansion for nonexemplar class-incremental learning. In CVPR, pages 9296-9305, 2022.

[0009] [Document 4] Marco Toldo and Mete Ozay. Bring evanescent representations to life in lifelong class incremental learning. In CVPR, pages 16732-16741, 2022

[0010] [Document 5] Guile Wu, Shaogang Gong, and Pan Li. Striking a balance between stability and plasticity for class-incremental learning. In ICCV, pages 1124-1133, 2021.

[0011] [Document 6] Hankook Lee, Sung Ju Hwang, and Jinwoo Shin. Self-supervised label augmentation via input transformations. In ICML, pages 5714-5724, 2020. SUMMARY

[0012] To solve the above technical problems, the application provides a class-incremental learning method and system based on prototype recall and asymmetric knowledge aggregation, to greater extent inhibit the catastrophic forgetting suffered by class-incremental learning, and improve the plasticity of the model.

[0013] The technical scheme adopted by the method of the application is: a class-incremental learning method based on prototype recall and asymmetric knowledge aggregation, comprising the following steps:

[0014] Step 1: expand the classes of the current task by rotation transformation, and provide the enhanced data to the feature extractor to extract new class features;

[0015] Step 2: store a representative class prototype for each old class as an old class prototype;

[0016] Step 3: combine the extracted new class features with the stored old class prototypes to generate old class features, and dynamically model the feature distribution of the old data in the continuously updated representation space;

[0017] Step 4: jointly train the generated old class features with the features extracted from the new data to realize prototype recall;

[0018] Step 5: aggregate the knowledge of the classifier learned through self-supervised label augmentation learning, and transfer it to another classifier that only recognizes unenhanced classes , to improve the plasticity of the model;

[0019] Step 6: minimize the Euclidean distance L kd between the features of the new data extracted by the current model and the features of the previous model , to constrain the feature extractor.

[0020] As a preferred, in step 1, the classes of the current task are expanded by rotation transformation, for the current task t, the classes of the current task are increased, the N-way classification problem is expanded to 4N-way classification problem, that is, the input data of each class is rotated by 90°, 180°, 270° to generate three new classes:

[0021] ​

[0022] where, represents the i-th input image, represents the image augmented by the j-th rotation;

[0023] and assign a new label to the augmented data

[0024]

[0025] where, denotes the label of the i-th input image .

[0026] input the augmented data into the feature extractor to obtain new class features:

[0027]

[0028] where, θ t denotes the parameters of the current feature extractor .

[0029] As a preference, in step 2, a prototype is calculated and stored in the deep feature space for each class:

[0030]

[0031] where, denotes the sample of the c new -th class in the current task t, denotes the number of samples of the c new -th class in the current task t;

[0032] All stored prototypes constitute a set P old saved in memory.

[0033] As a preference, in step 3, the prototype recall strategy is used to dynamically model the feature distribution of old data in the continuously updated representation space; a prototype of an old class and a feature of a new class are randomly selected and between them to perform a random bidirectional interpolation operation:

[0034]

[0035] where, is the generated old class feature in the current representation space; p e is a random number randomly drawn from [0, 1); the coefficient λ is randomly sampled from the Beta distribution.

[0036] As preferred, in step 4, the joint training of the generated old class features and the features extracted from the new data refers to inputting the generated old class features and the features extracted from the new data into the enhanced classifier and the un-enhanced classifier and calculating the cross-entropy loss:

[0037]

[0038]

[0039] wherein L ce represents the cross-entropy loss, represents the parameters of the classifier , represents the parameters of the classifier .

[0040] As preferred, the specific implementation of step 5 includes the following sub-steps:

[0041] Step 5.1: for the past tasks, input the old class features F old generated in step 3 into the classifier and process it using the classification head associated with the original class:

[0042]

[0043] wherein K = |C total | is the number of seen classes, represents the transpose of the classification head weight of the class c to which F old belongs in the classifier , represents the transpose of any classification head weight in the classifier , F old is the old class feature generated by the prototype recall, and p agg (c|F old ) represents the conditional probability of the aggregation for the old class feature;

[0044] Step 5.2: for the current task, aggregate all transformed conditional probabilities for each class:

[0045]

[0046] wherein represents the feature of the new class data after the jth enhancement, represents the transpose of the classification head weight of the class {j} to which F belongs in the classifier , represents the transpose of any classification head weight in the classifier Transpose of the weights of any class head in the class. This represents the conditional probability of aggregation for features of the new class;

[0047] Step 5.3: Transfer the aggregated knowledge to another classifier that only needs to distinguish between non-augmented classes. Its loss function L ka Represented as:

[0048]

[0049] in for The parameters are: F represents the features input to the classifier, KLD(·||·) is the Kullback-Leibler (KL) divergence; ρ agg (·|F) represents the classifier Output the conditional probability of the aggregation. Represents classifier The output.

[0050] Preferably, in step 6...

[0051]

[0052] Where ||·||2 represents L2 regularization; These represent the use of the current feature extractor. The feature extractor trained on the previous task From enhanced new types of data Extracted features;

[0053] Total loss function L total for:

[0054]

[0055] Where α and γ are the loss weights.

[0056] This invention also provides a class-based incremental learning system based on prototype recall and asymmetric knowledge aggregation, comprising:

[0057] One or more processors;

[0058] A storage device for storing one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the class incremental learning method based on prototype recall and asymmetric knowledge aggregation.

[0059] The present application proposes a new prototype recall mechanism, which combines previous class prototypes with new class features arriving, dynamically reshapes the old class feature distribution to maintain the decision boundary of previous tasks, thereby alleviating the catastrophic forgetting that occurs when deep neural network-based incremental learning is performed on dynamic data streams. In addition, in order to improve the generalization ability of the image classification model to new tasks and old tasks, the present application proposes an enhanced asymmetric knowledge aggregation method, which aggregates the overall knowledge of the current task and extracts valuable knowledge of past tasks on the basis of self-supervised label enhancement. Based on this scheme, not only can the problem of precision decline of the image classification model when performing incremental learning on the learned tasks be effectively alleviated, but also the performance of the model on new tasks can be significantly improved. The effectiveness of the algorithm proposed by the present application has been proved on CIFAR-100, TinyImageNet and ImageNet-Subset data sets.

[0060] The present application has the following advantages:

[0061] (1) The present application proposes a simple and effective prototype recall method, which can model the feature distribution of previous data in a constantly updated representation space, thereby suppressing catastrophic forgetting.

[0062] (2) The present application proposes an asymmetric knowledge aggregation method, which learns task-independent representations and completely captures newly acquired knowledge to improve the plasticity of the incremental learner.

[0063] (3) The present application achieves the most advanced performance on the existing technical benchmark, and the performance is obviously improved. BRIEF DESCRIPTION OF DRAWINGS

[0064] The technical solutions in this paper are further illustrated by using examples and specific embodiments. In addition, some drawings are also used in the process of explaining the technical solutions. For those skilled in the art, other drawings and the intention of the present application can also be obtained from these drawings without creative labor.

[0065] Figure 1 The method principle diagram of the embodiment of the present application;

[0066] Figure 2 The prototype recall principle diagram of the embodiment of the present application;

[0067] Figure 3 The asymmetric knowledge aggregation method principle diagram of the embodiment of the present application. DETAILED DESCRIPTION

[0068] For the convenience of those skilled in the art to understand and implement the present application, the present application will be further described in detail below in conjunction with the drawings and examples. It should be understood that the examples described herein are only used to illustrate and explain the present application, and are not intended to limit the present application.

[0069] See Figure 1 The embodiment provides a class incremental learning method based on prototype recall and asymmetric knowledge aggregation, and includes the following steps:

[0070] Step 1: expand the classes of the current task through rotation transformation, and provide the enhanced data to the feature extractor to extract new class features;

[0071] In an embodiment, the classes of the current task are expanded through rotation transformation. For the current task t, the classes of the current task are increased, and the N-way classification problem is expanded to a 4N-way classification problem, that is, the input data of each class is rotated by 90°, 180° and 270° to generate three new classes:

[0072]

[0073] wherein, represents the i-th input image, represents the image after the j-th rotation augmentation;

[0074] and the new label is assigned to the enhanced data

[0075]

[0076] wherein, represents the label of the i-th input image ;

[0077] The enhanced data is input into the feature extractor to obtain new class features:

[0078]

[0079] wherein, θ t represents the parameters of the current feature extractor .

[0080] Step 2: store a representative class prototype for each old class as an old class prototype;

[0081] See Figure 2 In an embodiment, a prototype is calculated and stored in the deep feature space for each class:

[0082]

[0083] wherein, denotes the number of samples of class c new in the current task t, denotes the number of samples of class c new in the current task t. All stored prototypes constitute a set P old saved in memory.

[0084] Step 3: Combine the extracted new class feature and the stored old class prototype to generate an old class feature, and dynamically model the feature distribution of the old data in the continuously updated representation space;

[0085] In an embodiment, a prototype of the old class is randomly selected and a feature of the new class is randomly selected and between them to perform a random bidirectional interpolation operation:

[0086]

[0087] wherein, is the generated old class feature in the current representation space, which has the same label as p e is a random number randomly selected from [0, 1); the coefficient λ is randomly sampled from a Beta distribution, which can be represented as λ ~ Beta(0.5, 0.5) ∈ [0, η], wherein η is a threshold value for controlling the maximum distance between the generated old random class feature and the corresponding prototype, and Beta(0.5, 0.5) represents a Beta distribution with two parent numbers of 0.5.

[0088] Step 4: Joint training of the generated old class feature and the feature extracted from the new data to realize prototype recall;

[0089] In an embodiment, the generated old class feature and the feature extracted from the new data are input into an enhanced classifier and an unenhanced classifier and the cross-entropy loss is calculated:

[0090]

[0091]

[0092] wherein, L ce represents the cross-entropy loss, represents the parameters of the classifier , represents the parameters of the classifier .

[0093] Step 5: Aggregate the classifiers learned through Self-Supervised Label Augmentation (SLA). The knowledge is transferred to another classifier that only recognizes the unenhanced classes. Improve the model's flexibility; during the testing phase, use the current feature extractor. and the refined classifier Conduct an assessment;

[0094] Please see Figure 3 In one implementation, step 5 specifically includes the following sub-steps:

[0095] Step 5.1: For past tasks, use the old class features F generated in Step 3. old The input is fed into the classifier and processed using the classification head associated with the original category:

[0096]

[0097] Where K = |C total | represents the number of categories that have been seen. Representative classifier China F old The transpose of the category head weights of category c. Representative classifier The transpose of the weights of any class head, F old The old class features generated by prototype recall, ρ agg (c|F old ) represents the conditional probability of aggregation for features of the old class;

[0098] Step 5.2: For the current task, aggregate the conditional probabilities of all transformations for each category:

[0099]

[0100] in, Represents the new class of data after the j-th enhancement. Corresponding features Representative classifier middle The category header weights of the category {j} are transposed. Representative classifier Transpose of the weights of any class head in the class. This represents the conditional probability of aggregation for features of the new class;

[0101] Step 5.3: Transfer the aggregated knowledge to another classifier that only needs to distinguish between non-augmented classes. Its loss function L ka Represented as:

[0102]

[0103] in for The parameters are: F represents the features input to the classifier, KLD(·||·) is the Kullback-Leibler (KL) divergence; ρ agg (·|F) represents the classifier Output the conditional probability of the aggregation. Represents classifier The output.

[0104] Step 6: Minimize the current model Features of the extracted new data and previous models The characteristic Euclidean distance L kd Constrained feature extractor.

[0105] In one implementation,

[0106]

[0107] Where ||·||2 represents L2 regularization; These represent the use of the current feature extractor. The feature extractor trained on the previous task From enhanced new types of data Extracted features.

[0108] Total loss function L total It can be represented as:

[0109]

[0110] Where α and γ are the loss weights, both of which are set to 15 in this embodiment.

[0111] This embodiment also provides a class-based incremental learning system based on prototype recall and asymmetric knowledge aggregation, including:

[0112] One or more processors;

[0113] A storage device for storing one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the class incremental learning method based on prototype recall and asymmetric knowledge aggregation.

[0114] The invention will be further illustrated by specific experiments.

[0115] The deep learning framework used in this experiment is Pytorch, version 1.9.0, and CUDA version is 11.3. The hardware environment of the experiment is NVIDIA GeForce RTX 3090 graphics card, and the processor is Intel(R) Xeon(R) Gold 6240C. The specific implementation process of the class-incremental learning method based on prototype recall and asymmetric knowledge aggregation is as follows:

[0116] Step 1: Build an image classification model.

[0117] The main purpose of defining the base model is to show a framework that can perform class-incremental learning. In this embodiment, a ResNet-18 deep learning model is used as the base model for image classification tasks.

[0118] Step 2: Incremental learning training of the image classification model.

[0119] (1) Data preparation: Prepare images and corresponding class labels as training and test data, divide the training set and test set by the number of tasks. For details about the data set and the division of different data sets, see the data set of the embodiment.

[0120] (2) Put the training images of the first task into the image classification model for training after self-supervised label enhancement.

[0121] (3) Incremental training;

[0122] For each new task that comes later, repeat the incremental training process until all tasks are learned. First, prototype recall is performed before learning a new task, and a prototype is calculated and stored for each class in the deep feature space. Save and freeze the model learned in the previous task.

[0123] Then generate the old class features through prototype recall, and input them into the classifier together with the features extracted from the new task data for training, and calculate the cross-entropy loss of the enhanced and the refined classifier . Use the frozen model of the previous task to perform knowledge distillation on the current updated model. Then aggregate logical output, and calculate its KL divergence with logical output for asymmetric knowledge aggregation.

[0124] Finally, the weighted sum of multiple loss functions is calculated, and the gradient of the loss is backpropagated in the backpropagation process. The network is updated through the optimization strategy of stochastic gradient descent.

[0125] Step 3: Test the class-incremental learning algorithm

[0126] In the test phase, the average incremental accuracy A of the class incremental function is calculated K and the average forgetting rate F K to measure the effect of class incremental learning. Among them, the average incremental accuracy is the average of the accuracy of the classes seen in all incremental stages, reflecting the overall incremental performance of the method, and the average forgetting rate is the average difference between the peak task accuracy after incremental learning and the final task accuracy, reflecting the ability of the method to suppress catastrophic forgetting. The higher the average incremental accuracy and the lower the average forgetting rate, the better the effect of class incremental learning. The calculation method is as follows:

[0127]

[0128]

[0129] The calculation method is as follows:

[0130]

[0131] Among them, K represents the number of incremental stages (including the initial stage), and k represents the incremental stage. The average accuracy of all classes that appear is a k , a n, refers to the accuracy of task n after training stage m.

[0132] In the specific experiment:

[0133] 1. Data set;

[0134] This experiment uses CIFAR-100, TinyImageNet and ImageNet-Subset data sets. The CIFAR-100 data set contains 60,000 32x32 size images, with a total of 100 classes. Among them, each class has 500 training images and 100 test images. The TinyImageNet data set contains 200 classes, each class has 500 training images, 50 validation images and 50 test images, and the image size is 64x64. The ImageNet-Subset data set is a sub-data set with 100 classes randomly extracted from the ImageNet-1k data set, which has about 1300 training images and 50 test images per class, and each image is 256x256 in size.

[0135] 2. Experimental setup;

[0136] The CIFAR-100 dataset and the TinyImageNet dataset have three incremental settings, 5, 10 and 20 stages respectively. The ImageNet-Subset dataset has only one incremental setting, 10 stages. All dataset classes are arranged in a fixed random order. Except for one setting on the CIFAR-100 dataset, the model is mainly trained on half of the classes in the first task, and the remaining classes are evenly divided into each stage for training in the remaining stages.

[0137] 3. Training and testing methods

[0138] Training stage: The batch size is set to 128. The model is optimized by the Adam optimizer with the weight β1 = 0.9, β2 = 0.999, ∈ = 1e -8 -4, weight decay 2e-4. The initial learning rate is 0.001, and the subsequent adjustment is carried out by the cosine annealing algorithm, with an adjustment period T = 32.

[0139] Test stage: input the test image into the model, calculate the average incremental accuracy A K and the average forgetting rate F K to measure the effect of class incremental learning, where the higher the average incremental accuracy A K , the better, and the lower the average forgetting rate F K , the better.

[0140] 4. Comparison of class incremental learning performance

[0141] To verify the effectiveness of the present application, the present application is compared with existing class incremental learning methods 1-9, and the existing class incremental learning methods mainly include:

[0142] (1) Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In CVPR, pages 2001-2010, 2017.

[0143] (2) Francisco M Castro, Manuel J Mar'in-Jim'enez, Nicol'as Guil, Cordelia Schmid, and Karteek Alahari. End-to-end incremental learning. In ECCV, pages 233-248, 2018.

[0144] (3) Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning a unified classifier incrementally via rebalancing. In CVPR, pages 831-839, 2019.

[0145] (4) James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska Barwinska, et al. Overcoming catastrophic forgetting in neural networks. PNAS, pages 3521-3526, 2017.

[0146] (5) Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In CVPR, pages 2001-2010, 2017.

[0147] (6) Yu Liu, Sarah Parisot, Gregory Slabaugh, Xu Jia, Ales Leonardis, and Tinne Tuytelaars. More classifiers, less forgetting: A generic multi-classifier paradigm for incremental learning. In ECCV, pages 699-716, 2020.

[0148] (7) Lu Yu, Bartlomiej Twardowski, Xialei Liu, Luis Herranz, Kai Wang, Yongmei Cheng, Shangling Jui, and Joost van de Weijer. Semantic drift compensation for class-incremental learning. In CVPR, pages 6982-6991, 2020.

[0149] (8) Fei Zhu, Xu-Yao Zhang, Chuang Wang, Fei Yin, and Cheng Lin Liu. Prototype augmentation and self-supervision for incremental learning. In CVPR, pages 5871-5880, 2021.

[0150] (9) Kai Zhu, Wei Zhai, Yang Cao, Jiebo Luo, and Zheng Jun Zha. Self-sustaining representation expansion for nonexemplar class-incremental learning. In CVPR, pages 9296-9305, 2022.

[0151] The average incremental accuracy results on the CIFAR-100 dataset are shown in Table 1;

[0152] Table 1

[0153]

[0154]

[0155] The average forgetting rate results on the CIFAR-100 dataset are shown in Table 2;

[0156] Table 2

[0157]

[0158] The average incremental accuracy results on the TinyImageNet and ImageNet-Subset datasets are shown in Table 3;

[0159] Table 3

[0160]

[0161]

[0162] wherein E represents the number of samples stored for each old class. As can be seen from Table 1, Table 2 and Table 3, compared with the class-incremental learning methods in recent years, the average incremental accuracy obtained by the present application is higher than that of the existing methods, the average forgetting rate is lower, and the class-incremental learning effect is obviously improved. As can be seen from the results, the overall incremental performance of the present application is higher. The reasons are mainly as follows: 1. The prototype recall method proposed in the present application can model the feature distribution of the previous data in a constantly updated representation space, thereby inhibiting catastrophic forgetting. 2. The asymmetric knowledge aggregation method proposed in the present application can learn task-independent representations and completely capture newly acquired knowledge to improve the plasticity of the incremental learner.

[0163] It should be understood that the above description of the preferred embodiments is more detailed and is not considered as limiting the scope of patent protection of the present application. Those skilled in the art can make substitutions or modifications without departing from the scope of protection of the present application, and all fall within the scope of protection of the present application. The scope of protection of the present application should be subject to the appended claims.

Claims

1. A class-incremental learning method based on prototype recall and asymmetric knowledge aggregation, characterized in that, The method comprises the following steps: Step 1: expand the classes of the current task through rotation transformation, and provide the enhanced data to the feature extractor to extract new class features; Step 2: store a representative class prototype for each old class as an old class prototype; Step 3: combine the extracted new class features and the stored old class prototypes to generate old class features, and dynamically model the feature distribution of the old data in the continuously updated representation space; Wherein, the prototype recall strategy is used to dynamically model the feature distribution of the old data in the continuously updated representation space; is a prototype of a randomly selected old class and a feature of a new class , and a random bidirectional interpolation operation is performed between and . wherein, is the old class feature generated in the current representation space; is a random number randomly drawn from [0, 1); the coefficient is randomly sampled from a Beta distribution. Step 4: jointly train the generated old class features and the features extracted from the new data to realize prototype recall; The combined training of the generated old-class features and the new data-extracted features refers to inputting the generated old-class features and the new data-extracted features into an enhanced classifier and a non-enhanced classifier , and calculating a cross-entropy loss: wherein, represents a cross-entropy loss, represent parameters of the classifier represent parameters of the classifier represent parameters of the classifier represent parameters of the classifier Step 5: Aggregating classifiers that learn through self-supervised label augmentation knowledge from one to another that only recognizes the non-augmented class improving the model's plasticity; Step 6: Minimize current model Features of the new data extracted and features of the previous model Euclidean distance of the features Constraining the feature extractor.

2. The class-incremental learning method based on prototype recall and asymmetric knowledge aggregation of claim 1, wherein: In Step 1, the classes of the current task are augmented by a rotation transformation, for the current task , the classes of the current task are augmented, expanding the N-way classification problem to a 4N-way classification problem, i.e., the input data of each class is rotated by 90°, 180°, 270° to produce three new classes: wherein, represent the first input image, represent the image augmented by the j-th rotation. and assigning new tags to the enhanced data : wherein, represents the label of the i-th input image ​​ The enhanced data is input into the feature extractor to obtain new class features: wherein, denotes the parameters of the current feature extractor .

3. The class-incremental learning method based on prototype recall and asymmetric knowledge aggregation of claim 2, wherein: In step 2, a prototype is calculated and stored in the deep feature space for each class: wherein, representing the current task in the class of samples, representing the current task in the number of samples in the class. All stored prototypes form a collection are saved in memory.

4. The prototype recall and asymmetric knowledge aggregation based class incremental learning method according to claim 1, characterized in that, The specific implementation of step 5 includes the following sub-steps: Step 5.1: For past tasks, old class features generated in Step 3 Input to the classifier, processed with the classification head associated with the original class: wherein, is the number of seen classes, represents a classifier in the transpose of the class head weights of the class c to which the instance belongs, represents a classifier the transpose of any of the class head weights, is the old class feature generated by the prototype recall, represents the conditional probability of the aggregation of the old class features; Step 5.2: for the current task, aggregate the conditional probabilities of all transformations for each class: wherein, representing the new class data after the corresponding feature, representing the classifier in the class to which the corresponding class head weight transpose, representing the classifier in any of the class head weights, representing the conditional probability of the aggregation for the new class feature; Step 5.3: Transfer of aggregated knowledge to another classifier that only needs to distinguish between non-augmented classes whose loss function is represented as: wherein is a parameter, denotes a feature input to the classifier, is the Kullback-Leibler divergence; denotes the output of the classifier outputting the aggregated conditional probabilities, denotes the output of the classifier .

5. The prototype recall and asymmetric knowledge aggregation based class incremental learning method according to claim 4, characterized in that: In step 6, wherein represents L2 regularization; , respectively represent the features extracted from the enhanced new class data using the current feature extractor and the feature extractor trained on the previous task ; Total loss function is: ； wherein and is a loss weight.

6. A system for class-incremental learning based on prototype recall and asymmetric knowledge aggregation, characterized in that, One or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the class incremental learning method based on prototype recall and asymmetric knowledge aggregation as claimed in any one of claims 1 to 5. ​

Images