Class incremental learning method for causal feature preservation and confusion factor elimination

Abstract

The invention provides a class incremental learning method for causal feature preservation and confusion factor elimination, and the method carries out one or more rounds of incremental learning based on an initially trained model comprising a backbone network, and each round comprises the steps: obtaining prototypes of a plurality of old classes; the old models are pre-trained through the first training set, a teacher model is obtained, the old model in the first round is an initially trained model, and the old models in other rounds are models obtained after front-wheel incremental learning; generating a student model based on the teacher model, and performing incremental training on the student model by using the second training set to obtain a model after incremental learning, including: calculating knowledge distillation loss according to respective classification results of the teacher model and the student model on the original images in the second training set; setting an intervention weight according to the original image features extracted by the old model and the similarity between the old prototypes; and updating the student model according to the classification loss of the student model and the distillation loss obtained by weighting the knowledge distillation loss based on the intervention weight to obtain a model after incremental learning.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and deep learning, specifically to the field of incremental learning technology, and more specifically, to an incremental learning method that preserves causal features and eliminates confusion factors. Background Technology

[0002] Class Incremental Learning (CIL) refers to the ability of a model to continuously learn new tasks without accessing old data, while maintaining its ability to recognize previous tasks, when faced with continuously input data of different classes. Its core challenge lies in preventing catastrophic forgetting, which refers to the problem of a significant decline in the model's performance in recognizing old classes after learning new classes.

[0003] To address the above issues, current mainstream technical solutions can be broadly categorized into the following four types:

[0004] (1) Regularization method: It protects key parameters in old tasks by limiting the update range of model parameters. Representative methods include Elastic Weight Consolidation (EWC) in reference [1], MAS in reference [2], and LwF in reference [3].

[0005] (2) Memory replay method: It saves some old category samples or uses a generative model to synthesize old task samples and trains them together with the new task to alleviate forgetting. For example, iCaRL in reference [4] alleviates forgetting by saving the original sample image, and GAN-based Replay in reference [5] uses generative adversarial network to generate samples to alleviate forgetting.

[0006] (3) Dynamic network architecture method: It achieves knowledge isolation by adding a dedicated sub-network structure for new tasks, such as PackNet and Dynamic Expandable Networks in reference [6].

[0007] (4) Knowledge distillation method: This method uses an old model as a teacher and uses its output to impose soft label constraints on the new model. Representative examples include BiC in reference [7] and PODNet in references [8] and [9]. Knowledge distillation is a model compression method that uses a high-performance old model as a "teacher" and uses the probability distribution (soft labels) it generates for the input samples to guide the training of the new model. Compared with traditional hard labels, these soft labels contain more relative information between classes, which can help the new model learn features and decision boundaries better, thereby improving performance while maintaining a smaller model size.

[0008] In recent years, causal reasoning methods have been introduced into incremental learning, such as DDE in reference

[10] and Distilling Colliding Effect in reference

[11] . Causal reasoning representations model the “collision causal path” between the causal graph structure and the old and new features, aiming to alleviate catastrophic forgetting in memoryless situations.

[0009] Despite the progress made by the above methods in mitigating catastrophic forgetting, the following key issues remain in practical applications:

[0010] The problem of feature drift in old knowledge. The introduction of new data alters model parameters, causing the representation of old categories in the feature space to drift, leading to a decrease in the model's accuracy in recognizing old tasks. While existing methods such as knowledge distillation (e.g., PODNet) can alleviate this drift to some extent, they essentially rely on the static output of the old model and lack active modeling capabilities.

[0011] The problem of interference between old and new features (causal collision). For example, although the DDE method introduces the idea of ​​causal modeling and attempts to extract causal features that match the new data from the old model, its strategy allows the old features to interfere with the generation process of the new features, making it difficult to learn effectively, thus causing learning conflicts between the old and new tasks.

[0012] Therefore, existing incremental learning methods generally fail to solve the balance between preserving old features and learning new features. They are prone to causing old categories to drift in the feature space, resulting in poor recognition performance for old tasks, or causing old features to interfere with the generation of new category features, making it difficult to effectively learn new tasks.

[0013] It should be noted that the background information presented here is only for illustrating relevant information about the present invention to aid in understanding the technical solution of the present invention, and does not imply that the relevant information is necessarily prior art. The relevant information was submitted and disclosed together with the present invention, and should not be considered prior art unless there is evidence that the relevant information was disclosed before the filing date of the present invention.

[0014] The references are as follows:

[0015] [1]Kirkpatrick J, Pascanu R, Rabinowitz N, et al. Overcomingcatastrophic forgetting in neural [2]networks[J]. Proceedings of the national academy of sciences, 2017, 114(13): 3521-3526.

[0016] [3]Aljundi R, Babiloni F, Elhoseiny M, et al. Memory aware synapses:Learning what (not) to forget[C] / / Proceedings of the European conference oncomputer vision (ECCV). 2018: 139-154.

[0017] [4]Li Z, Hoiem D. Learning without forgetting[J]. IEEE transactionson pattern analysis and machine intelligence, 2017, 40(12): 2935-2947.

[0018] [5]Rebuffi S A, Kolesnikov A, Sperl G, et al. icarl: Incrementalclassifier and representation learning[C] / / Proceedings of the IEEE conferenceon Computer Vision and Pattern Recognition. 2017: 2001-2010.

[0019] [6]Park J, Ji A H, Park M, et al. MalCL: Leveraging gan-basedgenerative replay to combat catastrophic forgetting in malware classification[C] / / Proceedings of the AAAI Conference on Artificial Intelligence. 2025, 39(1): 658-666.

[0020] [7] Mallya A, Lazebnik S. Packnet: Adding multiple tasks to a single network by iterative pruning[C] / / Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 2018: 7765-7773.

[0021] [8] Yoon J, Yang E, Lee J, et al. Lifelong learning with dynamically expandable networks[J]. arXiv preprint arXiv:1708.01547, 2017.

[0022] [9] Douillard A, Cord M, Ollion C, et al. Podnet: Pooled outputs distillation for small-tasks incremental learning[C] / / European Conference on Computer Vision. Cham: Springer International Publishing, 2020: 86-102.

[0023]

[10] Zhou D W, Wang Q W, Qi Z H, et al. Deep class-incremental learning: A survey[J]. arXiv preprint arXiv:2302.03648, 2023, 1(2): 6.

[0024]

[11] Hu X, Tang K, Miao C, et al. Distilling causal effect of data in class-incremental learning[C] / / Proceedings of the IEEE / CVF conference on Computer Vision and Pattern Recognition. 2021: 3957-3966. Summary of the Invention

[0025] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide a class incremental learning method that preserves causal features and eliminates confusion factors.

[0026] The objective of this invention is achieved through the following technical solution:

[0027] According to a first aspect of the present invention, an incremental learning method is provided, the method comprising obtaining an initial training model for image classification, including a backbone network, performing one or more rounds of incremental learning based on the model to obtain a model after the incremental learning is completed, each round of incremental learning comprising: S1, obtaining prototypes of multiple old classes, each prototype of an old class being calculated based on features of all samples under that class, the old class being the class already learned before the current round of incremental learning; S2, obtaining a first training set and a second training set, each first sample in the first training set comprising an image pair consisting of an original image and an augmented image after data augmentation of the original image and a label, each second sample in the second training set comprising an original image and a label; S3, pre-training the old model using the first training set to obtain a teacher model, the first round of... The old model is the model used in the initial training, and the old models in other rounds are the models after incremental learning in the previous round; S4, Generate a student model based on the teacher model, and use the second training set to incrementally train the student model based on the teacher model to obtain the model after incremental learning. The incremental training includes: S41, Calculate the knowledge distillation loss based on the classification results of the original images in the second sample by the teacher model and the student model respectively; S42, Determine the degree of confusion of the new class with the knowledge of the old class based on the original image features extracted by the backbone network of the old model and the similarity between the prototypes of each old class, and set the intervention weight of the knowledge distillation loss that is positively correlated with the degree of confusion; S43, Update the student model based on the classification loss of the student model and the distillation loss after weighting the knowledge distillation loss based on the intervention weight, and obtain the model after incremental learning.

[0028] In some embodiments of the present invention, the model for initial training further includes a first classification head. In S3, the pre-training method includes: constructing a new model with a twin structure based on the old model, which includes a shared backbone network, a first classification head, and a second classification head; inputting the image pairs of the first sample into the new model, the new model extracts the original image features and enhanced image features, and obtains a first classification result based on the original image features through the first classification head and a second classification result based on the enhanced image features through the second classification head; updating the new model based on the pre-training loss calculated by the pre-training loss function to obtain a teacher model, wherein the pre-training loss includes the difference between the original image features and enhanced image features of the first sample, the consistency loss between the first classification result and the second classification result, and the difference between the first classification result and the second classification result and the label, respectively.

[0029] In some embodiments of the present invention, the pre-training loss further includes a contrastive loss calculated using a contrastive loss function, wherein the contrastive loss includes the difference in image features of the same category in the first training set and the difference in image features of different categories; wherein the difference in image features of different categories includes: the difference in image features of two first samples of different categories in the first training set and the difference between the original image features of the first sample and the prototypes of each old class.

[0030] In some embodiments of the present invention, the pre-training loss function is as follows:

[0031] ,

[0032] in, This represents the pre-training loss calculated using the pre-training loss function. This indicates the difference between the first classification result and the label. This indicates the difference between the second classification result and the label. This represents the consistency loss between the first and second classification results. , This indicates the first classification result. This indicates the result of the second classification. Describing the L2 norm, This represents the difference between the original image features and the enhanced image features of the first sample. , Represents the features of the original image. This represents the transpose of the original image features. Indicates enhanced image features, This indicates a comparative loss.

[0033] In some embodiments of the present invention, in step S42, the setting of the intervention weight includes: calculating the similarity between the original image features extracted by the backbone network of the old model and each old class prototype, wherein the higher the similarity, the higher the degree of confusion of the new class with the knowledge of the old class, and vice versa; selecting the highest similarity from the similarity between each of the old class prototypes and the original image features, and setting the intervention weight that is positively correlated with the highest similarity.

[0034] In some embodiments of the present invention, in step S4, the generation method of generating a student model based on a teacher model includes: copying the backbone network and the first classification head of the teacher model to obtain a student model; wherein, the calculation method of knowledge distillation loss includes: processing the original image in the second sample through the backbone network of the teacher model to obtain the original image features; obtaining a first classification result based on the original image features through the first classification head of the teacher model; obtaining a second classification result based on the original image features through the second classification head; obtaining the final classification result of the teacher model for the original image in the second sample based on the average of the first classification result and the second classification result; and calculating the knowledge distillation loss using the KL divergence calculation method based on the final classification result of the teacher model for the original image in the second sample and the classification results of the student model for the original image in the second sample.

[0035] In some embodiments of the present invention, the original image is augmented using a style enhancement approach, which includes color dithering, grayscale conversion, and brightness adjustment, or a combination thereof.

[0036] According to a second aspect of the present invention, an image classification method is provided, the method comprising: acquiring an original image and a model obtained after incremental learning using the method described in the first aspect of the present invention; classifying the input original image using the model after incremental learning to obtain a classification result.

[0037] According to a third aspect of the present invention, an electronic device is provided, comprising: one or more processors; and a memory for storing executable instructions; wherein the one or more processors are configured to implement the steps of the methods of the first and / or second aspects of the present invention by executing the executable instructions.

[0038] Compared with the prior art, the advantages of the present invention are as follows:

[0039] The pre-trained teacher model of this invention provides a stable model foundation for subsequent incremental training. During the incremental training phase, the degree of confusion between the new class and the old class knowledge is determined, and intervention weights positively correlated with the degree of confusion are set. The student model is updated based on its classification loss and the distillation loss weighted by the intervention weights. Higher confusion levels result in larger intervention weights, and vice versa. By dynamically adjusting the intervention weights, this invention not only maintains the memory of old task knowledge during incremental training but also effectively learns new tasks, achieving a better balance between preserving old features and learning new features, thus improving the model's learning efficiency and stability. Attached Figure Description

[0040] The embodiments of the present invention will be further described below with reference to the accompanying drawings, wherein:

[0041] Figure 1This is a schematic diagram of the class incremental learning method according to an embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram illustrating the training process principle of the pre-training stage according to an embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram of a causal path constructed based on the causal relationship between new and old features according to an embodiment of the present invention;

[0044] Figure 4 This is a schematic diagram illustrating the incremental training process according to an embodiment of the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the invention.

[0046] As mentioned in the background section, existing incremental learning methods generally fail to solve the balance between preserving old features and learning new features. This can easily cause the extracted features of old categories to drift in the feature space, resulting in poor recognition performance for old tasks, or cause old features to interfere with the generation of new category features, making it difficult to effectively learn new tasks.

[0047] To address the aforementioned problems, the inventors propose a class of incremental learning methods that preserve causal features and eliminate confusion factors. This method involves one or more rounds of incremental learning based on an initially trained model used for image classification. Each round of incremental learning in this invention is divided into a pre-training phase and an incremental training phase.

[0048] In the pre-training phase, the old model is pre-trained using the first training set to obtain the teacher model. The first round of old model is the model used in the initial training, while the models used in subsequent rounds are the models learned from the previous round of incremental learning. The pre-trained teacher model of this invention can provide a stable model foundation for subsequent incremental training.

[0049] During the incremental training phase, based on the classification results of the teacher model and the student model for the original images of the second samples in the second training set, the knowledge distillation loss is calculated. Based on the original image features extracted from the backbone network of the old model in the second samples and the similarity between the prototypes of each old class, the degree of confusion of the new class with the knowledge of the old class is determined. Intervention weights positively correlated with the degree of confusion are set. The student model is updated based on the classification loss of the student model and the distillation loss weighted by the intervention weights, resulting in the model after incremental learning. By dynamically adjusting the intervention weights (also considered as the intervention intensity of knowledge distillation between the old and new tasks), i.e., the higher the degree of confusion, the greater the intervention weights and distillation loss, and vice versa, the student model of this invention can not only retain the memory of the knowledge of the old task to a certain extent during incremental training, but also effectively learn the new task, better achieving a balance between preserving old features and learning new features, thus improving the learning efficiency and stability of the model.

[0050] According to one embodiment of the present invention, see Figure 1 This is a schematic diagram of an incremental learning method. The method includes obtaining an initial training model for image classification, including a backbone network and a first classifier head; performing one or more rounds of incremental learning based on the model to obtain the final model after incremental learning; each round of incremental learning includes steps S1, S2, S3, and S4. Step S4 includes steps S41, S42, and S43. To better understand this invention, each step is described in detail below with reference to specific embodiments.

[0051] Step S1: Obtain the prototypes of multiple old classes. The prototype of each old class is calculated based on the features of all samples under that class. The old classes are the classes that have been learned before this round of incremental learning.

[0052] According to one embodiment of the present invention, the prototype construction method for the old class includes: after the previous round of pre-training, using the backbone network of the teacher model pre-trained in the previous round, extracting the features of all training samples of each old class in the previous task, and then calculating the average value of the features of all samples of the same old class to obtain the representative feature vector of each old class, i.e., the prototype. For example, calculating the average value of the feature vectors of all "cat" class images in the old task yields the prototype of the "cat" class.

[0053] In this process, prototypes of all old classes are constructed in the manner described above, and these prototypes are stored in a class prototype library (Prototypes). The class prototype library stores prototypes of the old class data. These prototypes not only help optimize the structure of the feature space through contrastive learning during the pre-training phase, but more importantly, they will serve as condensed representations of old knowledge for use in the incremental training phase.

[0054] Step S2: Obtain the first training set and the second training set. Each first sample in the first training set includes an image pair consisting of the original image and an augmented image after data augmentation of the original image, and a label. Each second sample in the second training set includes the original image and a label.

[0055] According to one embodiment of the present invention, to ensure that the model can effectively learn new tasks and maintain the feature representations of old tasks, the dataset needs to undergo appropriate data preprocessing and incremental learning task partitioning to obtain the first training set and the second training set for each round of incremental learning tasks. The data preprocessing and incremental learning task partitioning are described below:

[0056] 1) Data preprocessing

[0057] According to one embodiment of the present invention, data preprocessing includes standardizing each original image in the dataset. The dataset may be a CIFAR-100 or ImageNet dataset. The standardization process includes: to eliminate scale differences between image content, subtracting the mean from the pixel values ​​of each image and dividing by the standard deviation to ensure data uniformity. This processed data helps improve the training efficiency and stability of the neural network.

[0058] According to one embodiment of the present invention, data preprocessing further includes performing data augmentation on the original image using various style transformation methods (style enhancement methods) to obtain an enhanced image, wherein the original image and the corresponding enhanced image have the same category label. Style enhancement methods include, but are not limited to, contrast adjustment, color dithering, grayscale conversion, and brightness adjustment, or combinations thereof. The technical solution of this embodiment can achieve at least the following beneficial technical effects: Style enhancement is a form of data augmentation that increases the diversity of the dataset by changing the visual style of the image rather than its content. These transformations can help the model extract consistent features from images of different styles, reducing the impact of style changes on feature extraction. In this way, the model can focus on learning the semantic content of the image, rather than its stylistic features. This is particularly useful for training more robust models, especially in computer vision tasks such as image classification and object detection.

[0059] 2) Division of incremental learning tasks

[0060] According to one embodiment of the present invention, the dataset is divided into multiple sub-datasets under various stages, and a corresponding training set is constructed based on the sub-datasets under each stage. The multiple stages include the following initial task stage and incremental learning stage:

[0061] Initial Task Phase (Base Task)

[0062] According to one embodiment of the present invention, a subset of original images from a portion of the dataset is selected as a sub-dataset for the initial task phase. An initial training set is constructed using this sub-dataset, and the initial training model is trained using this initial training set. For example, in the CIFAR-100 dataset, the original images corresponding to the first 50 classes can be selected as the initial task dataset to ensure that the model can learn stable basic features in the initial stage. Each initial sample in the initial training set includes the original image and its corresponding label.

[0063] Incremental Learning (Incremental Tasks) Phase

[0064] According to one embodiment of the present invention, in one or more rounds of incremental learning, each round of incremental learning involves selecting original and enhanced images of the remaining categories in the dataset as a subset of the dataset for the corresponding new task (i.e., the categories that the model needs to newly recognize). For example, in the CIFAR-100 dataset, the first round of incremental learning selects original and enhanced images of categories 51-60 as a subset of the dataset for that round. The second round of incremental learning selects original and enhanced images of categories 61-70 as a subset of the dataset for that round.

[0065] Specifically, a first training set and a second training set are constructed based on the subset of data in each incremental learning phase. Each first sample in the first training set includes an image pair consisting of the original image and an augmented image of the original image, along with a label. Each second sample in the second training set includes the original image and a label.

[0066] It should be noted that there are no overlapping categories between different tasks to ensure that the model can learn each new task progressively.

[0067] Step S3: Use the first training set to pre-train the old model to obtain the teacher model. The old model in the first round is the model trained initially, and the old models in other rounds are the models after incremental learning in the previous round.

[0068] The inventors discovered that existing problems also include the influence of confounding features on model judgments. Current mainstream CIL methods do not explicitly model the intervention of confounding features and lack targeted suppression mechanisms for this problem. For example, in image classification, non-semantic features such as image style (e.g., background color, texture) participate in training as confounding factors. Models often incorrectly learn these "non-causal relationships" as causal dependencies, leading to a decline in model generalization ability. Therefore, according to one embodiment of the present invention, an image consisting of the original image and the corresponding enhanced image from the first training set is used to pre-train the input old model. The enhanced image is obtained by processing the original image using style enhancement. This method enables the model to effectively distinguish between content features strongly related to the task objective (e.g., object shape, structure) and easily confounding style features (e.g., background, lighting, hue) in the image, building a robust base teacher model for subsequent incremental training.

[0069] According to one embodiment of the present invention, see Figure 2 This is a schematic diagram illustrating the training process principle of the pre-training phase. The pre-training method includes steps S31, S32, and S33:

[0070] Step S31: Based on the old model, a new model with a twin structure is constructed, which includes a shared backbone network, a first classifier head, and a second classifier head.

[0071] According to one embodiment of the present invention, the new model construction method includes: copying the first classification head of the old model, and performing operations such as weight parameter replacement on the copied first classification head to obtain a second classification head, which shares the same backbone network with the first classification head, resulting in a new model with a twin structure. The backbone network can be a deep convolutional neural network (CNN), which extracts multi-level features of the image through convolution operations and activation functions.

[0072] Step S32: Input the image pair of the first sample into the new model. The new model extracts the original image features and the enhanced image features, and obtains the first classification result based on the original image features through the first classification head and the second classification result based on the enhanced image features through the second classification head.

[0073] According to one embodiment of the present invention, original image features and enhanced image features are extracted through the backbone network of a new model. The original image features and enhanced image features are then input into a first classification head and a second classification head for processing, respectively, to obtain corresponding first classification results and second classification results.

[0074] Step S33: Update the new model based on the pre-training loss calculated by the pre-training loss function to obtain the teacher model. The pre-training loss includes the difference between the original image features and the enhanced image features of the first sample, the consistency loss between the first classification result and the second classification result, and the difference between the first classification result and the second classification result and the label respectively.

[0075] According to one embodiment of the present invention, the pre-training loss further includes a contrastive loss calculated using a contrastive loss function. The contrastive loss includes differences in image features within the same category and differences in image features between different categories in the first training set. The differences in image features between different categories include: differences in image features between two first samples of different categories in the first training set, and differences between the original image features of the first samples and the prototypes of each old class. This embodiment achieves at least the following beneficial technical effects: it performs contrastive learning on differences in features within the same category, contrastive learning on differences in features between different categories, and contrastive learning on the features of the first samples output from the backbone network of the old model in the first training set with the stored prototypes of the old classes, further optimizing the intra-class compactness and inter-class separability of the feature space.

[0076] According to one embodiment of the present invention, the pre-training loss function is as follows:

[0077] ,

[0078] in, This represents the pre-training loss calculated using the pre-training loss function. This indicates the difference between the first classification result and the label. This indicates the difference between the second classification result and the label. This represents the consistency loss between the first and second classification results. , This indicates the first classification result. This indicates the result of the second classification. Describing the L2 norm, This represents the difference between the original image features and the enhanced image features of the first sample. , Represents the features of the original image. This represents the transpose of the original image features. Indicates enhanced image features, This represents the contrast loss. Specifically, the cross-entropy loss function is used to calculate the difference between the first classification result and the label, and the difference between the second classification result and the label.

[0079] The technical solution of the above-described pre-training method can achieve at least the following beneficial technical effects: 1. Supervising the model's learning of classification tasks by calculating the difference between the classification results and the true labels. 2. Forcing the model to be insensitive to style changes by calculating consistency loss, thus learning content-consistent representations. 3. Promoting the learning of complementary and non-redundant features by calculating the difference between the original image features and the enhanced image features (diversity loss), enabling the model to learn different feature representations from multiple perspectives within the same category, avoiding interference from style bias, and improving the model's generalization ability. 4. Optimizing the feature space structure generated by the backbone network by calculating the contrast loss using the old class prototype and the new class sample features, thus bringing similar features closer together and pushing dissimilar features further apart, making the extracted features from different classes more discriminative. This invention simultaneously feeds the original image and the style-enhanced image into the backbone network, combining consistency loss and diversity loss to guide the model to ignore non-essential style information during the learning process, effectively eliminating the interference of confusion factors, allowing the model to focus more on learning semantic features, thereby improving the model's accuracy and generalization ability.

[0080] Step S4: Generate a student model based on the teacher model. Using the second training set, incrementally train the student model based on the teacher model to obtain the model after incremental learning.

[0081] According to an embodiment of the present invention, in step S4, the method of generating the student model based on the teacher model includes: copying the backbone network and the first classification head of the teacher model to obtain the student model.

[0082] The inventors discovered that modeling the causal paths in the incremental learning process is crucial. By constructing and regulating the causal paths between new and old features, the model can clearly distinguish the feature relationships between new and old tasks and effectively optimize the intensity of interventions during the learning process. See also Figure 3 This diagram illustrates the causal paths constructed based on the causal relationships between old and new features. It represents the causal relationships and paths between different variables and includes the old task dataset. and new task dataset and their corresponding old features New features Old task category variables New task category variables .also, and The figures represent the complete feature sets before and after merging the old and new features, respectively. This figure illustrates how the old and new datasets influence the target variable through their respective features. Therefore, according to one embodiment of the invention, the potential impact of the new data on the old task data in the figure during incremental learning for the new task can be controlled by optimizing the intervention intensity of the old task data on the new task data. The optimization adjustment methods for the intervention intensity include:

[0083] The intervention intensity is quantified into a dynamic intervention weight. This weight is adjusted and optimized by assessing the similarity between the features of the new class sample and the pre-built prototypes of each category of the old task. Specifically, when the features of the new class sample are highly similar to the features of a certain old class prototype, it indicates a significant overlap in the feature spaces of the new and old tasks, posing a high risk of knowledge overlay. In this case, a strong intervention is determined, and a higher intervention weight is generated for this sample. Conversely, if the features of the new class sample show low similarity to all old class prototypes, a lower intervention weight is generated. This embodiment achieves at least the following beneficial technical effects: Instead of directly calculating an abstract "causal effect" value during each incremental training parameter update, the training objective is directly optimized using the aforementioned dynamic weight, making the result approach the ideal causal effect. This weight acts as a balancing factor, directly adjusting the proportion of knowledge distillation loss in the total loss during incremental training. A higher weight amplifies the loss gradient for retaining old knowledge, causing the model to focus more on maintaining consistency with the teacher model during parameter updates, thereby enhancing the causal effect of retaining old knowledge. A lower weight allows the model to focus more on optimizing the classification loss for new tasks, emphasizing the causal effects of learning new knowledge.

[0084] To illustrate, suppose the new task is to learn to recognize "sheep," while the old task includes "cows" and "dogs." When an image of the "sheep" category is input, if its features are very close to the prototype of "cow" in the feature space, it means that "sheep" and "cow" may share many low-level visual features (such as hooves, four legs, etc.). This high similarity also means that when learning "sheep," the model is very likely to confuse or overwrite memories of "cows," leading to a significantly increased risk of forgetting. Therefore, a higher intervention weight will be assigned to the knowledge distillation loss of this sample based on the highest similarity between the features of "sheep" and the prototype of "cow." Conversely, if the features of a new sample are far from all the old class prototypes, it indicates that its features are relatively unique and the risk of forgetting is low, so a lower weight will be assigned.

[0085] According to one embodiment of the present invention, the student model is incrementally trained based on the intervention weight adjustment optimization method of the above embodiment. See also... Figure 4 This is a schematic diagram illustrating the principle of incremental training. The incremental training process includes the following steps S41-S43:

[0086] Step S41: Calculate the knowledge distillation loss based on the classification results of the original images in the second sample by the teacher model and the student model, respectively.

[0087] According to one embodiment of the present invention, the calculation method of knowledge distillation loss includes: processing the original image in the second sample through the backbone network of the teacher model to obtain the original image features; obtaining a first classification result based on the original image features through the first classification head of the teacher model; obtaining a second classification result based on the original image features through the second classification head; obtaining the final classification result of the teacher model for the original image in the second sample based on the average of the first and second classification results; and calculating the knowledge distillation loss using the KL divergence calculation method based on the final classification result of the teacher model for the original image in the second sample and the classification results of the student models for the original image in the second sample. This embodiment achieves at least the following beneficial technical effects: calculating the difference between the predicted outputs of the student model and the teacher model for the same input image and using it as a knowledge distillation loss. By minimizing the knowledge distillation loss, the student model can be constrained from deviating too far from the teacher model, thereby preserving the ability to judge the old categories.

[0088] According to one embodiment of the present invention, the knowledge distillation loss is calculated as follows:

[0089] ,

[0090] in, The first sample represents the knowledge distillation loss, and KL represents the KL divergence. This represents the final classification result output by the teacher model for the second sample. This represents the classification result output by the student model for the second sample. Both the final classification result from the teacher model and the classification result output by the student model are probability distributions, including the probability that the second sample belongs to each category. The technical solution of this embodiment can achieve at least the following beneficial technical effects: By introducing a knowledge distillation strategy, this invention ensures that the new model does not lose the knowledge learned during pre-training when learning a new task through comparison between the teacher model and the student model, thereby improving the model's stability and accelerating the learning of new tasks.

[0091] Step S42: Based on the original image features extracted from the backbone network of the old model and the similarity between the prototypes of each old class, determine the degree of confusion of the new class with the knowledge of the old class, and set the intervention weight of the knowledge distillation loss that is positively correlated with the degree of confusion.

[0092] According to an embodiment of the present invention, in step S42, the setting method of intervention weight includes: calculating the similarity between the original image features extracted by the backbone network of the old model and each old class prototype, wherein the higher the similarity, the higher the degree of confusion of the new class with the old class knowledge, and vice versa; selecting the highest similarity from the similarity between each of the old class prototypes and the original image features, and setting the intervention weight positively correlated with the highest similarity.

[0093] According to one embodiment of the present invention, the intervention weight is set by: selecting the old class prototype with the highest similarity to the original image features from all old class prototypes; calculating the average Euclidean distance between the original image features and the selected old class prototype; and setting the intervention weight based on the average Euclidean distance. The method for setting the intervention weight based on the Euclidean distance is as follows:

[0094] ,

[0095] in, This represents the intervention weights corresponding to the original image features of the second sample. Indicates the average Euclidean distance. This represents the temperature coefficient. The temperature coefficient is used to control the sensitivity of the intervention weights to distance.

[0096] Step S43: Update the student model based on the classification loss of the student model and the distillation loss weighted by the intervention weights to obtain the model after incremental learning.

[0097] According to one embodiment of the present invention, the sum of the classification loss of the student model and the distillation loss weighted by the knowledge distillation loss is calculated to obtain the total incremental training loss, and the student model parameters are updated based on this total loss. The total loss is calculated as follows:

[0098] ,

[0099] in, Indicates the total loss. This represents the classification loss of the student model for the second sample. The classification loss of the student model can be calculated using the cross-entropy loss function based on the classification result and label of the student model for the second sample. The technical solution of this embodiment can achieve at least the following beneficial technical effects: the above method can intelligently identify potential knowledge conflict points, and strengthen the "review" of old knowledge when the conflict is large (i.e., increase the intervention weight to increase the proportion of distillation loss); while when the conflict risk is small, the proportion of classification loss is higher, making the model more focused on learning new knowledge. This mechanism realizes refined and adaptive control of intervention weights in the learning process of new and old knowledge, improving the incremental training effect.

[0100] According to an embodiment of the present invention, an incremental learning system is provided. The system includes a pre-training module, a knowledge distillation module, and an incremental training module. The pre-training module executes step S3 in the above embodiment, the knowledge distillation module executes steps S41-S42 in the above embodiment, and the incremental training module executes step S43 in the above embodiment.

[0101] According to an embodiment of the present invention, an image classification method is provided, the method comprising: acquiring an original image and a model obtained after incremental learning using the method described in the above embodiment; classifying the input original image using the model after incremental learning to obtain a classification result.

[0102] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order, as long as the required function can be achieved.

[0103] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0104] Computer-readable storage media can be tangible devices that hold and store instructions for use by an instruction execution device. Computer-readable storage media can include, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.

[0105] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. An incremental learning method, comprising obtaining an initially trained model for image classification, including a backbone network, performing one or more rounds of incremental learning based on the model, and obtaining a model after the incremental learning is completed, wherein each round of incremental learning includes: S1. Obtain the prototypes of multiple old classes. The prototype of each old class is calculated based on the features of all samples under that class. The old classes are the classes that have been learned before this round of incremental learning. S2. Obtain the first training set and the second training set. Each first sample in the first training set includes an image pair consisting of the original image and an augmented image after data augmentation of the original image, and a label. Each second sample in the second training set includes the original image and a label. S3. Use the first training set to pre-train the old model to obtain the teacher model. The old model in the first round is the model trained initially, and the old models in other rounds are the models after incremental learning in the previous round. S4. Generate a student model based on the teacher model. Using the second training set, incrementally train the student model based on the teacher model to obtain the incrementally learned model. The incremental training includes: S41. Calculate the knowledge distillation loss based on the classification results of the teacher model and the student model for the original images in the second sample. S42. Based on the original image features extracted from the backbone network of the old model and the similarity between the prototypes of each old class, determine the degree of confusion of the new class with the knowledge of the old class, and set the intervention weight of the knowledge distillation loss that is positively correlated with the degree of confusion. S43. Update the student model based on the classification loss of the student model and the distillation loss after weighting the knowledge distillation loss based on the intervention weights, to obtain the model after incremental learning.

2. The method according to claim 1, characterized in that, The initially trained model also includes a first classification head. In S3, the pre-training methods include: A new model with a twin structure is constructed based on the old model, which includes a shared backbone network, a first classifier head, and a second classifier head. The image pair of the first sample is input into the new model. The new model extracts the original image features and the enhanced image features. The first classification head obtains the first classification result based on the original image features, and the second classification head obtains the second classification result based on the enhanced image features. The new model is updated based on the pre-training loss calculated by the pre-training loss function to obtain the teacher model. The pre-training loss includes the difference between the original image features and the enhanced image features of the first sample, the consistency loss between the first classification result and the second classification result, and the difference between the first classification result and the second classification result and the label respectively.

3. The method according to claim 2, characterized in that, The pre-training loss also includes a contrastive loss calculated using a contrastive loss function, which includes the difference in image features of the same category and the difference in image features of different categories in the first training set; The differences in image features across different categories include: differences in image features between two first samples of different categories in the first training set, and differences between the original image features of the first samples and the prototypes of each old category.

4. The method according to claim 3, characterized in that, The pre-training loss function is as follows: ， in, This represents the pre-training loss calculated using the pre-training loss function. This indicates the difference between the first classification result and the label. This indicates the difference between the second classification result and the label. This represents the consistency loss between the first and second classification results. , This indicates the first classification result. This indicates the result of the second classification. Describing the L2 norm, This represents the difference between the original image features and the enhanced image features of the first sample. , Represents the features of the original image. This represents the transpose of the original image features. Indicates enhanced image features, This indicates a comparative loss.

5. The method according to claim 1, characterized in that, In S42, the intervention weight is set in the following ways: Calculate the similarity between the original image features extracted by the backbone network of the old model and the prototypes of each old class. The higher the similarity, the higher the degree of confusion between the new class and the old class knowledge, and vice versa. Select the highest similarity among all old class prototypes and the original image features, and set the intervention weight that is positively correlated with the highest similarity.

6. The method according to claim 2, characterized in that, In S4, the generation method of the student model based on the teacher model includes: copying the backbone network and the first classification head of the teacher model to obtain the student model; wherein, the calculation method of knowledge distillation loss includes: The original image in the second sample is processed by the backbone network of the teacher model to obtain the original image features. The first classification head of the teacher model obtains the first classification result based on the original image features, and the second classification head obtains the second classification result based on the original image features. The final classification result of the teacher model for the original image in the second sample is obtained by averaging the first and second classification results. The knowledge distillation loss is calculated using the KL divergence method based on the final classification results of the teacher model on the original images in the second sample and the classification results of the student models on the original images in the second sample.

7. The method according to claim 1, characterized in that, The original image is augmented using style enhancement techniques, which include color dithering, grayscale conversion, and brightness adjustment, or a combination thereof.

8. An image classification method, characterized in that the method include: Acquire the original image and the model after incremental learning using the method described in any one of claims 1-7; The model after incremental learning is used to classify the original input image to obtain the classification result.

9. A computer-readable storage medium, characterized in that, It stores a computer program that can be executed by a processor to implement the steps of the method according to any one of claims 1-8.

10. An electronic device, characterized in that, include: One or more processors; as well as Memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of the method according to any one of claims 1-8 by executing the executable instructions.

Images