A quasi-incremental learning method based on causal learning and dynamic parameter selection

Abstract

The embodiments of the present disclosure provide a class incremental learning method based on causal learning and dynamic parameter selection. Applied to the field of image data processing technology, it includes: obtaining a training image set, the training image set contains each training image and the real label of the training image, and constructing an input data sequence according to the training image set; constructing a teacher model and a student model to extract features, and obtain output features and corresponding logits outputs, thereby constructing an old class knowledge causal path and a new class knowledge causal path with causal relationships; based on knowledge distillation and old class regularization methods, the old class knowledge causal path and the new class knowledge causal path are respectively used to calculate the old class positive causal effect and the new class positive causal effect, and the overall causal effect is obtained by summing them up. In this way, the present invention uses causal reasoning technology and causal structure methods to alleviate catastrophic forgetting, and provides new ideas and solutions for alleviating the problem of catastrophic forgetting.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image data processing, and in particular to a class incremental learning method based on causal learning and dynamic parameter selection. Background Art

[0002] With the rapid development of Internet technology, deep learning models have achieved remarkable results in various fields. However, these deep learning models often encounter a very common problem, namely catastrophic forgetting, which seriously affects the accuracy of deep learning models.

[0003] As a technique for analyzing causal relationships, causal learning helps the model better understand and retain existing knowledge by exploring the causal effects between features and targets, reduces the conflict between new and old knowledge, and effectively solves the problem of catastrophic forgetting. Therefore, it is necessary to develop a method to find causal paths to alleviate catastrophic forgetting from a quasi-incremental model. Summary of the invention

[0004] The present disclosure provides a quasi-incremental learning method based on causal learning and dynamic parameter selection, which alleviates catastrophic forgetting by utilizing causal reasoning technology and causal structure methods, and solves the technical problem that data forgetting is prone to occur in existing methods.

[0005] According to a first aspect of the present disclosure, a class incremental learning method based on causal learning and dynamic parameter selection is provided, comprising the following steps:

[0006] Acquire a training image set, wherein the training image set includes each training image and a true label of the training image, and construct an input data sequence according to the training image set;

[0007] Constructing a teacher model and a student model, using the teacher model and the student model to extract features of the input data sequence, obtaining output features and corresponding logits outputs, and constructing an old class knowledge causal path and a new class knowledge causal path with causal relationships based on the output features and the corresponding logits outputs;

[0008] The difference between the logits output values ​​when the old data exists and when the old data does not exist is calculated according to the old class knowledge causal path; the logits output of the old class data is subjected to knowledge distillation according to the teacher model and the student model, and regularization is performed to obtain the old class knowledge distillation result and the old class regularization result; the optimized old class positive causal effect is obtained according to the difference, the old class knowledge distillation result and the old class regularization result;

[0009] According to the new class knowledge causal path, the predicted value of the teacher model score where the old data exists and the predicted value of the teacher model score where the old data does not exist are calculated respectively, and the difference between the two is calculated to obtain the new class positive causal effect of the single data, and the new class positive causal effect is obtained by updating and summing; the new class positive causal effect is simplified according to the collision causal effect to obtain the simplified new class positive causal effect; the output feature mean on the new class data is subjected to knowledge distillation according to the teacher model and the student model, and regularization is performed to obtain the new class knowledge distillation result and the new class regularization result;

[0010] A new class positive causal effect is optimized according to the positive causal effect, the new class knowledge distillation result and the new class regularization result;

[0011] The old category positive causal effect and the new category positive causal effect are added together to obtain the overall causal effect.

[0012] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein a training image set is obtained, wherein the training image set includes each training image and a true label of the training image, and a process of obtaining an input data sequence according to the training image set is:

[0013] Input image samples are collected to obtain input sample data, labels are assigned to the input sample data, a training data set is constructed, and the training data set is divided into old class data and new class data to obtain an input data sequence.

[0014] According to the above aspects and any possible implementation, a further implementation is provided, wherein the old class knowledge causal path construction process is:

[0015] The old class data and the new class data are input into the teacher model for processing, and the old and new class extraction features of the teacher model are obtained as output, and the logits of the new and old class features output on the old task category are obtained according to the old and new class extraction features of the teacher model;

[0016] The new class data is input into the student model for processing, and the new class extraction features of the student model are output. The old class logits output is obtained based on the logits output on the old task category by the new and old class features and the new class extraction features.

[0017] According to the above aspects and any possible implementation, a further implementation is provided, wherein the process of constructing the causal path of the new type of knowledge is as follows:

[0018] The old class data and the new class data are input into the teacher model for processing, and the old and new class extraction features of the teacher model are output, and the collision causal effect of the old and new class extraction features of the teacher model and the new class data is constructed;

[0019] The new class data is input into the student model for processing, and the new class logits output is output.

[0020] According to the above aspects and any possible implementation, an implementation is further provided, wherein the calculation formula of the overall causal effect is:

[0021] ;

[0022] in, is the overall causal effect, For a new type of positive causal effect, For the old category positive causal effect.

[0023] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein the method further includes:

[0024] Constructing a linear regression model, and performing linear fitting on the extracted features according to the linear regression model to obtain regression coefficients;

[0025] The importance of the extracted features is obtained according to the regression coefficients and normalized, and the importance of the extracted features is evaluated.

[0026] According to the above aspects and any possible implementation, an implementation is further provided, wherein the relationship between the teacher model and the student model is:

[0027] ;

[0028] Among them, T t is the updated teacher model, For the teacher model, is the student model and η is a hyperparameter.

[0029] Compared with the prior art, the present invention has achieved the following beneficial effects:

[0030] The present invention proposes a class incremental learning method based on causal learning and dynamic parameter selection (CLPS-CIL), which uses causal reasoning technology to analyze the causal relationship between new and old knowledge respectively, explores the causal effect between features and targets, thereby alleviating the conflict between new and old knowledge and the accumulation of deviations. At the same time, the causal structure method is used to calculate and select important feature parameters, and these parameters are used to guide the Fisher information matrix to update the model parameters to protect key knowledge and achieve reasonable forgetting. In addition, the present invention prevents the model from forgetting or losing important knowledge that has been learned when continuously learning new categories, thereby effectively controlling the degree of knowledge forgetting, and then finding the causal path to alleviate catastrophic forgetting in the class incremental model, providing a new idea and solution for alleviating the problem of catastrophic forgetting.

[0031] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, among which:

[0033] Figure 1 A schematic diagram of a flow chart of a class incremental learning method based on causal learning and dynamic parameter selection according to an embodiment of the present disclosure is shown;

[0034] Figure 2 A schematic diagram of the causal effect path of old class knowledge of a class incremental learning method based on causal learning and dynamic parameter selection according to an embodiment of the present disclosure is shown;

[0035] Figure 3 A schematic diagram of a causal effect path of new class knowledge of a class incremental learning method based on causal learning and dynamic parameter selection according to an embodiment of the present disclosure is shown;

[0036] Figure 4 A schematic diagram of the overall causal effect path of a quasi-incremental learning method based on causal learning and dynamic parameter selection according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0038] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0039] Please refer to Figure 1 As shown, this embodiment provides a class incremental learning method based on causal learning and dynamic parameter selection, including the following steps:

[0040] S101 , obtaining a training image set, wherein the training image set includes each training image and a true label of the training image, and obtaining an input data sequence according to the training image set.

[0041] In the class of incremental learning, its goal is to learn from a continuous sequence of tasks. Continuously learn a single task model , and at the same time make the model's performance on all tasks reach the best. The training data set is , , represents the input sample, Represents a label, each task Contains input data and tags The labels of different tasks are mutually exclusive, i.e. .

[0042] S102. Construct a teacher model and a student model, and construct a causal path of old class knowledge and a causal path of new class knowledge with causal relationship based on the teacher model, the student model and the input data sequence.

[0043] After acquiring knowledge information, this embodiment learns new and old knowledge from continuous data. First, through the old knowledge with causal relationship, this embodiment learns new and old knowledge from continuous data. and ) Learn the causal relationship old category knowledge that actively alleviates knowledge conflicts and bias accumulation, so as to better retain important old knowledge and adapt to classification. This embodiment establishes , and The positive causal path is as follows: Figure 2 As shown, and Represent samples of new and old task data respectively, and are the features extracted by the teacher model and the student model respectively. It is the teacher model extraction and Features, Represents the logits of the extracted new and old class features output on the old task category, Represents the final logits output.

[0044] Likewise, this embodiment learns new types of knowledge from continuous data with causal effects that actively mitigate knowledge conflicts and bias accumulation. Figure 3 Established The causal path is used to alleviate the impact of new classes when the model is learning. It can be seen that if Received The impact of Will affect , thereby helping to adapt to new types of knowledge.

[0045] S103. Calculate the old-category positive causal effect and the new-category positive causal effect based on the old-category knowledge causal path and the new-category knowledge causal path respectively, and calculate the overall causal effect based on the old-category positive causal effect and the new-category positive causal effect.

[0046] In order to better estimate the positive causal effect of adaptation on the old class, Defined as old data The difference between the logits when it exists and when it does not exist is:

[0047] (1)

[0048] because There is no parent node, so equation (1) can be transformed into equation (2). express Operation means assigning a certain value to the variable without considering the parent class node. Combining the total probability and Bayesian probability calculation rules, formula (2) is simplified to formula (3), and we can get:

[0049] (2)

[0050] (3)

[0051] because yes arrive The only intermediate path of , so (3) is further simplified as:

[0052] (4)

[0053] From formula (4), it can be seen that it reveals the teacher model How to model for students Make an impact, It can be seen as a prediction constraint between the teacher model and the student model. Indicates that the teacher model is based on the old data Pre-training on old data and not on old data The probability difference of pre-training on , obviously this difference must not be zero.

[0054] In order to maintain the mastery of previously learned old category knowledge in the learning task, this embodiment performs distillation in the following manner:

[0055] (5)

[0056] in: represents the number of samples belonging to the old category, is the training data set, and Respectively represent the logits output of the teacher model and the student model for the old class samples. In order to better maintain the memory of the old data knowledge and enhance the knowledge distillation process, this embodiment also adopts a causal regularization method, and the regularization formula is:

[0057] (6)

[0058] in: In order to better reflect the constraint relationship between the teacher and student models, this embodiment defines the relationship between the teacher model and the student model as:

[0059] (7)

[0060] in: is a hyperparameter, and in this embodiment, the value is 4. For the training data set, in formula (7), the knowledge distillation of the first term is crucial, which is the key to retaining the knowledge of the old data, while the second term is to further refine the teacher-student model, forcing the model to keep the memory of the old knowledge as much as possible when learning new tasks, and further illustrate Figure 2 The causal path in can effectively encourage new and old data and right positive causal effect.

[0061] When constructing new types of knowledge, this embodiment will be the new type of positive causal effect Defined as old data The difference between score predictions when present and absent:

[0062] (8)

[0063] (9)

[0064] (10)

[0065] in: express operate, For old data knowledge The score predictions of the trained teacher model are is the score prediction for training the teacher model without knowledge of old data. There is no parent class node, so equation (8) holds; because in class incremental learning, the teacher model is trained without old data knowledge, The value of is approximately zero, so equation (10) holds.

[0066] In order to better achieve and To learn new types of knowledge with positive causal effects, using pass right This embodiment combines formula (10) to produce a collision causal effect. Updated to be the sum of the causal effects for each sample prediction score:

[0067] (11)

[0068] (12)

[0069] (13)

[0070] (14)

[0071] in: It is new data The number of samples, For the The score prediction of samples, is the corresponding feature extracted by the encoder of the teacher model. Using the full probability formula in formula (12), we can get formula (13). Figure 3 In, due to is from arrive , so equation (14) holds.

[0072] Obviously, in formula (14) The second term is simplified to be the predicted score of the teacher model given a new sample:

[0073] (15)

[0074] Bayes’ theorem is applied in equation (15). Figure 3 In the feature extraction From And subject to The main impact comes from , using causal Markov to simplify:

[0075] (16)

[0076] Substituting formula (16) into formula (15) and simplifying it, we get:

[0077] (17)

[0078] Substituting formula (17) into (14), we obtain:

[0079] (18)

[0080] (19)

[0081] In formula (18), It is regarded as a weight factor, which is composed of the teacher model feature space and It is determined by the positional relationship between them.

[0082] For the convenience of implementation, this embodiment selects the truncated top-K samples and converts them into The calculation is for The weighted sum of K-Nearest-Neighbours, when the teacher model is in the feature space and The smaller the distance between them, the larger the weight factor will be if and only if To enhance the knowledge distillation process, the following distillations are performed:

[0083] (20)

[0084] in: is the number of samples of the new category, L KL is the KL divergence loss function, and The student model and the teacher model are the samples The softened probability distribution of and are the means of the output features of the model and the teacher model on the new categories, respectively.

[0085] In addition, to further maximize , The estimates are as follows:

[0086] (twenty one)

[0087] in: Yes The prediction score of It's a label. is the cross entropy loss function, The formula is as follows:

[0088] (twenty two)

[0089] in: The corresponding sample The K-Nearest-Neighbours (KNN) sample set, is the weight, which is 0.95 according to experience. is the prediction score of the current model, The neighbor nodes get the prediction scores. By optimizing equation (21), we can learn new types of knowledge with causal effects in continuous data.

[0090] S103. The causal effect in class incremental learning is obtained based on the old class knowledge learning and the new class knowledge learning with causal relationship.

[0091] In the incremental learning process, the model needs to effectively absorb new knowledge without forgetting the old knowledge. However, due to the weight update mechanism of the neural network, the model may inadvertently adjust the parameters that are critical to the old task when optimizing the new task, resulting in a decrease in the performance of the old task. In addition, the conflict between new and old knowledge may aggravate the forgetting phenomenon. For example, suppose a model was originally trained to recognize cats and was later asked to learn to recognize dogs. If the new task is learned in a way that conflicts with the old task and makes contradictory classification decisions on certain features, it may be difficult for the model to maintain accurate recognition of cats and dogs at the same time.

[0092] To this end, this embodiment introduces a causal effect method to identify and quantify the strength of the causal relationship between features in new and old tasks. Through this analysis, the model can adjust its weights more intelligently to ensure that the original causal structure is not destroyed when learning new knowledge, thereby achieving harmonious coexistence of new and old knowledge.

[0093] This embodiment adapts to new classes and preserves old classes through active causal effects from a continuous data stream, so this embodiment is based on a teacher model (T t-1 ) and the student model adapted to new tasks , the teacher model is updated once each iteration to adapt to the distribution of new data:

[0094] (twenty three)

[0095] Among them, η is a hyperparameter. According to experiments, The model will have better results.

[0096] In this embodiment, the overall goal is expressed as In order to better obtain positive causal effects from old and new class data, we use represents the positive causal effect of adapting to the new class, represents the positive causal effect of adapting to the old class, as Figure 4 As shown, the overall representation is as follows:

[0097] (twenty four)

[0098] In addition, in order to more accurately protect the old knowledge of the model, this embodiment evaluates and adjusts the importance of parameters by combining causal reasoning. Specifically, the importance of each feature is evaluated by linear regression, which is regarded as a simplification of the Structural Equation Model (SEM). For each feature , this example fits a linear regression model to predict the target variable , and get the regression coefficient , which reflects the characteristic For the target variable Degree of contribution:

[0099] (25)

[0100] in: is the number of features, is the intercept term, is the regression coefficient, which is the embodiment of common knowledge; therefore, the feature Importance Defined as the absolute value of its regression coefficient:

[0101] (26)

[0102] To ensure the stability and comparability of the values, the importance of all feature parameters is normalized:

[0103] (27)

[0104] At the same time, in order to better solve the parameter updating problem of the model, this embodiment adopts the parameter updating strategy of Fisher information matrix and causal reasoning. The core is to use the information matrix to quantify the sensitivity of the parameters to the loss function and identify those parameter knowledge that are crucial to the current task.

[0105] First initialize the Fisher matrix The diagonal elements of are zero, then, for each parameter in the model , the diagonal elements of the information matrix are calculated by :

[0106] (28)

[0107] in: is the training dataset, is the loss function, It is data samples.

[0108] Next, we use the normalized important features To adjust the Fisher matrix The diagonal elements of , to reflect the real importance:

[0109] (29)

[0110] In this way, those parameters that are judged to be more important in the causal analysis will have larger diagonal elements of the Fisher matrix and thus be more protected in the subsequent learning process. At the end of each task, the cumulative Fisher matrix is ​​updated to reflect the information of all previous tasks. The diagonal elements of the cumulative Fisher matrix On Task Updated to:

[0111] (30)

[0112] in: is the diagonal element of the adjusted Fisher matrix at the end of task j. The goal of incorporating causal reasoning is to protect the model from forgetting the knowledge of previous tasks during learning while allowing it to learn new knowledge. Evaluating feature importance through causal reasoning and identifying which features are most critical for predicting the target helps to more finely adjust the Fisher matrix, making the model more robust in the continuous learning process.

[0113] Example

[0114] In order to verify the effectiveness of the proposed method, this embodiment carried out experimental verification on the image classification datasets Cifar100, 5-Datasets, Tiny-ImageNet dataset and the challenging datasets ImageNet-R, OmniBenchmark, VTAB, and ObjectNet.

[0115] (1) The data set used in the experiment of this embodiment is:

[0116] Cifar100 dataset: Cifar100 contains 60,000 32×32 RGB images of 100 categories. Each class has 500 training images and 100 test images. This example divides the 100 classes equally into 10 and 20 incremental batches.

[0117] 5-Datasets: 5-Datasets consists of five image classification datasets: Cifar10, MNIST, Fashion-MNIST, SVHN, and notMNIST. In the experiment, the model was trained sequentially according to SVHN, MNIST, Cifar10, NotMNIST, and Fashion-MNIST. Although it is not difficult to train each dataset individually, due to the different task types of each dataset, the model is easy to forget, and their sequential training is quite challenging even with the ImageNet pre-trained model.

[0118] Tiny-ImageNet dataset: It has 200 classes, image size is 64 x 64 pixels, and each class has 500 training images, 50 validation images, and 50 test images.

[0119] VTAB dataset: It is a diverse and challenging set of tasks containing different classes from multiple complex domains, used to measure general and practical visual task evaluation.

[0120] OminiBenchmark dataset: Compared with ImageNet-1000, OminiBenchmark is a large-scale benchmark dataset covering more complex fields.

[0121] ObjectNet dataset: It is a large real-world test set for object recognition where the object background, rotation, and imaging viewpoint are random.

[0122] ImageNet-R dataset: ImageNet-R has renderings of 200 ImageNet classes with 30,000 images. It includes categories such as art, cartoons, graffiti, embroidery, graphics, origami, paintings, patterns, plastic objects, plush objects, sculptures, sketches, tattoos, toys, etc.

[0123] (2) Evaluation indicators determined in this embodiment

[0124] This embodiment reports two main indicators in the experiment: the degree of forgetting after learning task t (FGT, the lower the better) and the average accuracy (ACC) after learning task t of continuous learning.

[0125] (3) Process of this Example

[0126] When using ViT-B / 16 as the backbone, this example uses 3 epochs for Cifar100 with a learning rate of 0.001; uses 5 epochs for 5-Datasets with a learning rate of 0.001, and reports the results of three independent runs. When using DieT-S / 16 as the backbone, the buffer size is set to 2000, and the 100 classes are divided into 10 tasks and 20 tasks for Cifar100, and the 200 classes are divided into 5 tasks and 10 tasks for Tiny-ImageNet. When using ViT-B / 16IN21K as the backbone, OmniBenchmark, ImageNet-R, ObjectNet, and VTAB are evenly divided into 10 tasks, the batch size is 32, the learning rate is 0.001, and the learning rates of the fully connected layers are 0.004, 0.005, 0.006, and 0.0009, respectively. The implementation of CLPS-CIL methods is based on Pytorch. All implementations are run on the RTX 4080 GPU, using SGD as the optimizer for all methods and the backbone.

[0127] (4) Comparison method of this embodiment

[0128] Several competitive incremental-like methods are considered: ER, BiC, LUCIR, PODNET, DDE, DER++, CLSER, FOSTER, MEMO, GDumb, Co2L, BaCE, OCM, LiDER, DualHSIC, L2P, BEEF, HAT, OWM, SLDA, PASS, MUC, LwF, EEIL, iCaRL, A-GEM, GD, HAL, ROW, Sup, MORE, EWT, IL2M.

[0129] (5) Performance evaluation and analysis experimental results

[0130] The experimental results on ViT-B / 16 compared with various comparison methods are shown in Tables 1, 2, and 3.

[0131] Table 1 Comparison with competitive baselines at 10 steps on Cifar100 (all methods use pre-trained ViT-B / 16 as backbone)

[0132] ;

[0133] ;

[0134]

[0135] In Table 1, the Cifar100 dataset is divided into 10 steps. When the buffer size is 100, the average accuracy and forgetting performance are 83.79% and 6.94% respectively, which are 8.98% and 17.84% higher than the best results in the comparison method; when the buffer size is 500, the average accuracy and forgetting performance are 86.85% and 3.53% respectively, which are 2.26% and 9.05% higher than the best results in the comparison method; it can be seen that under a smaller buffer size, the CLPS-CIL of the present invention achieves relatively excellent jumping performance.

[0136] Table 2 Comparison with competitive baselines at 20 steps on Cifar100 (all methods use pre-trained ViT-B / 16 as backbone)

[0137] ;

[0138]

[0139] In Table 2, the Cifar100 dataset is divided into 20 steps. When the buffer size is 100, the average accuracy and forgetting performance are 79.20% and 11.81% respectively, which are 13.32% and 22.24% higher than the best results in the comparison method; when the buffer size is 500, the average accuracy and forgetting performance are 84.85% and 5.93% respectively, which are 2.39% and 9.85% higher than the best results in the comparison method; it can be seen that at 20 steps, with a smaller buffer size, the CLPS-CIL of the present invention also achieves relatively excellent jumping performance.

[0140] Table 3 Comparison with competitive baselines on 5-Datasets datasets (all methods use pre-trained ViT-B / 16 as the skeleton)

[0141] ;

[0142]

[0143] In Table 3, it can be seen from the experimental results in the table that on the 5-Datasets dataset, when the buffer size is 250, the average accuracy and forgetfulness are 91.29% and 0.36% respectively, which are 3.79% and 8.05% higher than the best results of all the compared methods; when the buffer size is 500, the average accuracy and forgetfulness are 91.42% and 0.32% respectively, which are 1.62% and 4.9% higher than the best results of all the compared methods. According to the experimental results, the average accuracy and forgetfulness of the method of the present invention are very similar when the buffer size is 250 and 500, which shows that the method of the present invention has strong adaptability and shows excellent performance on smaller buffers.

[0144] It can be seen from the experimental results in Table 1, Table 2, and Table 3 that the method of the present invention is superior to all the comparative methods in all experimental settings, especially under smaller buffers, where the average accuracy is improved by up to 13.32%, and the forgetting degree is reduced by 22.24% at best. At the same time, on the 5-Datasets dataset, the performance obtained under smaller buffers is close to that under larger buffers, which fully demonstrates that the method of the present invention has stronger adaptability and anti-forgetting ability under smaller buffers, thereby further verifying that causal analysis can further reduce the conflict and bias accumulation problems between new and old knowledge, and can effectively alleviate the occurrence of catastrophic forgetting.

[0145] (6) Ablation analysis results

[0146] This example considers four versions of ablation experiments on the Cifar100 dataset, sets the buffer size to 100, and reports the average of three random runs. (a): Let , indicating that the elimination of The distillation term in (9) is eliminated from the causal perspective. Figure 2 All causal paths of are deleted; (b): Let , which means by eliminating Figure 3 The causal path in , which is the second term in equation (24), degenerates it into cross entropy loss; , combining (a) and (b), that is (d) order , indicates that the calculation part of parameter importance is considered to be eliminated. The results are shown in Table 4.

[0147] Table 4 Ablation experiment results (buffer size is 100, using pre-trained ViT-B / 16 as the skeleton and reporting the results of three independent runs)

[0148]

[0149] From the ablation experiment in Table 4, it can be seen that when all causal paths are considered to be eliminated, the performance of the model will drop significantly, which proves that the causal method proposed in this embodiment has good anti-forgetting ability and is conducive to the learning and adaptation of knowledge; of course, when trying to remove the causal path of new or old class adaptation, the model performance also drops significantly, and the average accuracy and forgetting rate drop by 22% and 30.88% respectively, which proves that all causal paths are indispensable in the CLPS-CIL method. In addition. The introduced parameter importance calculation can also effectively prevent the decline of model performance. Therefore, the calculation of parameter importance can help retain old knowledge.

[0150] (7) Analysis of the impact of hyperparameters on the KNN model

[0151] The specific results are shown in Table 5:

[0152] Table 5 Hyperparameter analysis results (buffer size is 100, using pre-trained ViT-B / 16 as the backbone on the Cifar100 dataset and reporting the results of three independent runs)

[0153]

[0154] An experimental analysis was conducted on the hyperparameter K in the K-Nearest-Neighbours (KNN) sample set, where K refers to the number of KNNs. From the experimental results in Table 5, it can be seen that in the experimental setting environment of this embodiment, when the value of k is 5, that is, when the number of KNNs is 5, the performance of the model is optimal.

[0155] (8) Analysis of the impact of hyperparameter λ on the model

[0156] The specific results are shown in Table 6:

[0157] Table 6 Hyperparameter analysis results (buffer size is 100, using pre-trained ViT-B / 16 as the backbone on the Cifar100 dataset and reporting the results of three independent runs)

[0158]

[0159] This embodiment conducts an experimental analysis on the hyperparameter λ in formula (7). From the experimental results in Table 6, it can be seen that in the experimental setting environment of this embodiment, when the value of λ is 4, it can effectively use formula (7) to retain old class knowledge and prevent the model knowledge from being forgotten.

[0160] (9) Analysis of the impact of hyperparameter η on the model

[0161] The specific results are shown in Table 7:

[0162] Table 7 Hyperparameter analysis results (buffer size is 100, using pre-trained ViT-B / 16 as the backbone on the Cifar100 dataset and reporting the results of three independent runs)

[0163]

[0164] This embodiment experimentally analyzes the hyperparameter η in formula (22). From the experimental results in Table 7, it can be seen that in the experimental setting environment of this embodiment, when the value of η is 0.9, the update of the teacher model and the student model makes the overall performance of the model reach the best.

[0165] (10) Performance evaluation on DeiT-S / 16 with less overlapping information

[0166] In order to reduce the information overlap between the pre-training data and the class incremental learning dataset, this example uses DieT-S / 16 as the backbone for experimental verification. For the Cifar100 100 dataset, 100 classes are divided into 10 and 20 tasks (C100-T10 and C100-T200); for Tiny-ImageNet, 200 classes are divided into 5 and 10 tasks (T-5T and T-10T). The specific experimental results are shown in Table 8.

[0167] Table 8 Average accuracy after learning all tasks (buffer size is 2000, and results of five independent runs are reported)

[0168] ;

[0169]

[0170] According to the results shown in Table 8, on DeiT-S / 16 with less overlapping information, the performance of the CLPS-CIL method is better than all the compared class-incremental methods. This proves that even in models with less overlapping information, CLPS-CIL can still effectively learn new knowledge while maintaining the stability of old class knowledge, thereby improving the performance of the model.

[0171] (11) Experimental analysis of challenging datasets

[0172] On the challenging datasets OmniBenchmark, ImageNet-R, ObjectNet, and VTAB, ViT-B / 16IN21K is used as the backbone model, and the experimental settings are followed to further evaluate the method proposed in this embodiment. Among them, all classes on the four datasets are evenly divided into 10 tasks for verification. The specific results are shown in Table 9.

[0173] Table 9 Accuracy comparison results of the methods on four challenging datasets and several typical class imbalance problems (the random seed is set to 1993 in the experiment)

[0174]

[0175] In Table 9, CLPS-CIL is compared with several distillation-based incremental methods. It can be seen that on the challenging datasets OmniBenchmark, ImageNet-R, ObjectNet, and VTAB, the accuracy of the CLPS-CIL method is improved by 3.58%, 5.54%, 7.95%, and 5.22% respectively compared with the best results among the comparison methods, achieving relatively excellent performance. This fully proves that the CLPS-CIL method can also have strong learning and adaptability on complex datasets, and effectively alleviate the occurrence of catastrophic forgetting.

[0176] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0177] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0178] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A class incremental learning method based on causal learning and dynamic parameter selection, characterized in that: The following steps are involved: Acquire a training image set, wherein the training image set includes each training image and a true label of the training image, and construct an input data sequence according to the training image set; The process of obtaining a training image set, which includes each training image and the true label of the training image, and constructing an input data sequence according to the training image set is as follows: Collect input image samples to obtain input sample data, assign labels to the input sample data, construct a training data set, and divide the training data set into old class data and new class data to obtain an input data sequence; Constructing a teacher model and a student model, using the teacher model and the student model to extract features of the input data sequence, obtaining output features and corresponding logits outputs, and constructing an old class knowledge causal path and a new class knowledge causal path with causal relationships based on the output features and the corresponding logits outputs; The process of constructing the causal path of old knowledge is as follows: The old class data and the new class data are input into the teacher model for processing, and the old and new class extraction features of the teacher model are obtained as output, and the logits of the new and old class features output on the old task category are obtained according to the old and new class extraction features of the teacher model; The new class data is input into the student model for processing, and the new class extraction features of the student model are output, and the old class logits are output according to the logits output on the old task category by the new and old class features and the new class extraction features; The process of constructing the causal path of the new type of knowledge is as follows: The old class data and the new class data are input into the teacher model for processing, and the old and new class extraction features of the teacher model are obtained as output, and the collision causal effect of the old and new class extraction features of the teacher model and the new class data is constructed; Input the new class data into the student model for processing, and output the new class logits output; The difference between the logits output values ​​when the old data exists and when the old data does not exist is calculated according to the old class knowledge causal path; the logits output of the old class data is subjected to knowledge distillation according to the teacher model and the student model, and regularization is performed to obtain the old class knowledge distillation result and the old class regularization result; the optimized old class positive causal effect is obtained according to the difference, the old class knowledge distillation result and the old class regularization result; According to the new class knowledge causal path, the predicted value of the teacher model score where the old data exists and the predicted value of the teacher model score where the old data does not exist are calculated respectively, and the difference between the two is calculated to obtain the new class positive causal effect of the single data, and the new class positive causal effect is obtained by updating and summing; the new class positive causal effect is simplified according to the collision causal effect to obtain the simplified new class positive causal effect; the output feature mean on the new class data is subjected to knowledge distillation according to the teacher model and the student model, and regularization is performed to obtain the new class knowledge distillation result and the new class regularization result; A new class positive causal effect optimized according to the new class positive causal effect, the new class knowledge distillation result and the new class regularization result; The old category positive causal effect and the new category positive causal effect are added together to obtain the overall causal effect.

2. The incremental learning method based on causal learning and dynamic parameter selection according to claim 1, characterized in that: The calculation formula for the overall causal effect is: ； in, is the overall causal effect, For a new type of positive causal effect, For the old category positive causal effect.

3. The incremental learning method based on causal learning and dynamic parameter selection according to claim 1, characterized in that: The method further comprises: Constructing a linear regression model, and performing linear fitting on the extracted features according to the linear regression model to obtain regression coefficients; The importance of the extracted features is obtained according to the regression coefficients and normalized, and the importance of the extracted features is evaluated.

4. The incremental learning method based on causal learning and dynamic parameter selection according to claim 1, characterized in that: The relationship between the teacher model and the student model is: ； in, is the updated teacher model, For the teacher model, For student models, is a hyperparameter.

Images