A self-supervised class-incremental learning method based on similarity prototype calibration

Abstract

The present application relates to a kind of self-supervised class incremental learning method based on similarity prototype calibration, including the following steps: calculating each new class prototype;Obtain the set of most similar class and the weight of each prototype in the set;The new class prototype and old class prototype are weighted calculation, to obtain the calibrated prototype;Calculate the target logit parameter of each transformation;The knowledge of old task original class is aggregated, to obtain the first aggregation inference probability, and all transformations are aggregated for the current task, to obtain the second aggregation inference probability;First self-distillation loss and second self-distillation loss are calculated and minimized;Target loss function is calculated and minimized, to obtain the optimized classification model, wherein the optimized classification model is used to classify input image.The present application effectively reduces the overlap between new and old classes, reduces the distinction between retention and update network, while fully utilizing the knowledge learned from new class, significantly improves the plasticity of model.

Description

Technical Field

[0001] The present invention belongs to the technical field of image classification, and in particular relates to a self-supervised incremental learning method based on similarity prototype calibration. Background Art

[0002] In class-incremental learning (CIL), a model is trained on a series of tasks consisting of a set of new classes. Among existing CIL methods, retraining-based approaches perform well in mitigating catastrophic forgetting by preserving old examples and using them to learn new tasks. However, blindly preserving old examples raises concerns about privacy violations and memory usage. Therefore, retraining-based CIL methods are not applicable to many real-world situations. In contrast, example-free class-incremental learning can retain knowledge of old classes without storing any examples.

[0003] Some current work proposes preserving old categories without storing old samples. For example, Wu et al. proposed a GAN-based method, Kirkpatrick et al. proposed a regularization-based method, and Yan et al. proposed a network expansion-based method. Furthermore, Zhu et al. proposed an incremental learning method based on prototype enhancement and self-supervised learning, while Zhu et al. and Shi et al. proposed a prototype-based example-free incremental learning method. These methods preserve categories by memorizing old categories in a deep feature space and jointly classifying prototypes and features of new categories. This strategy effectively memorizes old knowledge while protecting privacy.

[0004] However, the biggest problem with existing prototype-based incremental learning methods is the lack of features from old categories, which can lead to decision boundary collapse. Even though some work has considered enhancing the prototype by adding Gaussian noise or oversampling to enrich features from old categories, training with a continuous data stream may still cause the representation of old category features to inevitably change, rendering the prototype obsolete. At the same time, existing incremental learning methods based on example-free classes often perform well on new classes because they have sufficient training datasets. However, when new class data appears, the prototypes of old classes become insufficient. This is because old classes lack sufficient training data compared to new classes, which also leads to potential data imbalance. Summary of the Invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides a self-supervised incremental learning method based on similarity prototype calibration. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0006] The embodiment of the present invention provides a self-supervised incremental learning method based on similarity prototype calibration, comprising the steps of:

[0007] S1. Extract the data features of each new category of the current task and use the data features to calculate the prototype of each new category;

[0008] S2. Obtaining a set of most similar categories and a weight of each prototype in the set using the similarity between each new category prototype and the old category prototype;

[0009] S3. Perform weighted calculation on the new category prototype and the old category prototype using the weight of each prototype to obtain a calibrated prototype;

[0010] S4, calculating the target logit parameter of each transformation using the affinity between several augmented images obtained by transforming the original image;

[0011] S5. Aggregate the knowledge of the original category of the old task using the target logit parameter and the calibrated prototype to obtain a first aggregated inference probability, and aggregate all transformations of each new category from the original image to the augmented image for the current task to obtain a second aggregated inference probability;

[0012] S6. Calculate a first self-distillation loss based on the first aggregated inference probability and the single inference probability of classifying the original image features, calculate a second self-distillation loss based on the second aggregated inference probability and the single inference probability of classifying the original image features, and minimize the first self-distillation loss and the second self-distillation loss;

[0013] S7. Calculate a target loss function based on the first self-distillation loss and the second self-distillation loss and minimize the target loss function to obtain an optimized classification model, wherein the optimized classification model is used to classify the input image.

[0014] In one embodiment of the present invention, step S2 includes:

[0015] Calculate the similarity between each new category prototype and the old category prototype:

[0016]

[0017] Among them, s c,q represents the similarity between the prototype of the old category c and the prototype of the new category q, Represents the old category prototype, represents the new category prototype, τ represents the temperature parameter used to control the sharpness of the weight distribution;

[0018] The similarities are arranged in descending order as Γ c =argtopk{s c,q}, and select a preset number of new category prototypes and old category prototypes in descending order of similarity as the set of most similar categories;

[0019] The softmax function is used to calculate the weight of each prototype in the set:

[0020]

[0021] Among them, ω c,q Represents the weight of the classification prototype, s c,q represents the similarity between the prototype of the old category c and the prototype of the new category q, s l,q Represents the similarity between the prototype of the old category l and the prototype of the new category q.

[0022] In one embodiment of the present invention, the weighted calculation formula in step S3 is:

[0023]

[0024] in, represents the calibrated prototype, Represents the old category prototype, represents the prototype of the new category h, ω h represents the weight of the prototype of the new category h, k represents the number of categories, λ c represents a trade-off parameter.

[0025] In one embodiment of the present invention, step S4 includes:

[0026] The original image is transformed into several augmented images, and the original image is input into the feature extractor to obtain the original logit parameters of each transformation:

[0027]

[0028] Among them, s j represents the original logit parameter of the jth transformation, ω j represents the weight of the j-th transformation classifier, F j represents the features of the j-th transformation;

[0029] Calculate the affinity between several augmented images and obtain the affinity matrix:

[0030]

[0031] Among them, A(i,j) represents the affinity between the i-th augmented image and the j-th augmented image, i and j represent the indexes of different augmented images, s i represents the original logit parameter of the i-th transformation;

[0032] Deleting the diagonal elements from the affinity matrix yields the association matrix:

[0033] A=A⊙(1-I)

[0034] Where I is the identity matrix and ⊙ is element-wise multiplication;

[0035] Normalize each row of the association matrix to obtain the normalized affinity matrix of the augmented image i, while keeping all elements on the diagonal zero. The normalized affinity matrix of the augmented image i is:

[0036]

[0037] Where M represents the number of augmented images in a sample, and N represents the number of samples;

[0038] The target logit parameters for each transformation are calculated using the normalized affinity matrix:

[0039]

[0040] in, represents the target logit parameter of the j-th transformation, represents the normalized affinity matrix.

[0041] In one embodiment of the present invention, the first aggregate inference probability is:

[0042]

[0043] in, Represents the inference probability of the old class feature, F old represents the features generated by prototype calibration, c represents the categories that have been seen, K represents the total number of categories that have been seen, Indicates the classifier in stage T Medium F old The transpose of the target logit parameter for category c, Indicates the classifier in stage T Medium F old The transpose of the target logit parameter for any class k.

[0044] In one embodiment of the present invention, the second aggregate inference probability is:

[0045]

[0046] in, Represents the aggregate inference probability of new class features after different transformations, Represents the augmented image after j transformations Features, Represents the augmented image The feature set extracted from the image, M represents the number of augmented images, Indicates the classifier in stage T middle The transpose of the target logit parameter of category c after j transformations, Indicates the classifier in stage T middle The transpose of the target logit parameter for any category k after j transformations.

[0047] In one embodiment of the present invention, step S6 includes:

[0048] The first self-distillation loss is calculated using the KL divergence between the first aggregated inference probability and the single inference probability of classifying the original image features:

[0049]

[0050] in, represents the first self-distillation loss, is the KL divergence, Represents the classifier after different transformations The first aggregate inference probability of the output, Represents a classifier The output, It is a feature extractor Parameters;

[0051] The second self-distillation loss is calculated using the KL divergence between the second aggregated inference probability and the single inference probability of classifying the original image features:

[0052]

[0053] in, represents the second self-distillation loss, is the KL divergence, Represents the classifier after different transformations The second aggregate inference probability of the output, Represents a classifier The output, It is a feature extractor Parameters;

[0054] The first self-distillation loss and the second self-distillation loss are minimized to regularize the feature extractor.

[0055] In one embodiment of the present invention, the objective loss function is:

[0056]

[0057]

[0058] in, is the target loss function, represents the cross entropy loss, and denote the loss of new data and the loss of old class features obtained by prototype calibration, respectively. Represents the current model trained on new data and the previous model trained with the previous data The characteristic Euclidean distance between represents the first self-distillation loss, represents the second self-distillation loss, ||·||2 represents L2 regularization, α and γ represent loss weights, represents a classifier that distinguishes between augmented images and original images, Represents a classifier Parameters, represents the classifier only for the original image, φ t Represents a classifier Parameters, represents the i-th input image, represents the i-th augmented image, θ t Represents the current feature extractor The parameter θ t-1 Represents the previous feature extractor Parameters, Represents a feature extractor for old categorical data.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] 1. The present invention utilizes the set of most similar categories between the new category prototype and the old category prototype and the weight of each prototype, and uses the weight to perform weighted calculation to obtain a calibrated prototype. By aggregating the new prototypes with the highest similarity to the old prototypes, the evolution of the representation of the old category prototype is tracked. By adjusting the features of the old category to the current representation space, the overlap between the new and old categories is effectively reduced, which not only alleviates the distinction between the retained and updated networks, but also has practical significance for the model performance.

[0061] 2. The present invention is based on an asymmetric aggregation self-distillation strategy for batch knowledge integration, which uses affinity to calculate logit parameters, aggregates the knowledge of the original categories of the old tasks, and aggregates all transformations of each new category for the current task. Asymmetric aggregation self-distillation is achieved by selectively extracting the most valuable information from the new categories and aggregating it with existing knowledge in an asymmetric manner. By integrating the knowledge learned in the same batch, it selectively aggregates valuable knowledge from enhanced images, makes full use of the knowledge learned from new categories, and significantly improves the plasticity of the model, making it more adaptable to real-world scenarios and can efficiently adapt to new tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 A flowchart of a self-supervised incremental learning method based on similarity prototype calibration provided by an embodiment of the present invention;

[0063] Figure 2 The overall framework diagram of the self-supervised incremental learning method based on similarity prototype calibration provided by an embodiment of the present invention;

[0064] Figure 3 A schematic diagram of asymmetric polymerization self-distillation provided by an embodiment of the present invention;

[0065] Figure 4 A schematic diagram of the importance of different enhancement types in different categories provided by an embodiment of the present invention;

[0066] Figure 5 A schematic diagram illustrating changes in classification accuracy when different methods provided in embodiments of the present invention are used for learning tasks on CIFAR-100 and TinyImageNet. DETAILED DESCRIPTION

[0067] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0068] Example 1

[0069] This embodiment proposes a class incremental learning method based on similarity-based prototype calibration, which is a Non Exemplar Class-Incremental Learning (NECIL) algorithm and a novel self-supervised learning method, namely, a self-distillation algorithm based on asymmetric aggregation to perform self-supervised learning on the model. Through the prototype calibration method, the biased old category prototype is fused with the weighted new category prototype to reshape the feature space of the old prototype to enhance distinguishability. In addition, in order to improve the transferability of the model, the overall knowledge of the current task is self-supervised learning based on the asymmetric aggregation self-distillation method.

[0070] First, a brief description and analysis of incremental learning is given. The incremental learning task can be considered as training the model with T tasks in sequence. The entire dataset is a task-specific dataset. The union of , where t∈{1,…,T}, is the input data, is the true label. Specifically, the model of this embodiment consists of two parts: a feature extractor with parameter θ and the parameters are Unified classifier In the incremental learning setting, the dataset S t It is only available during the training task t. The testing phase follows the strategy of single-head evaluation. In the incremental learning setting, the model uses a dataset S of previous data t-1 For training, when a new task t arrives, the model only uses the dataset S t Training without S t-1 However, due to catastrophic forgetting, although the model can adaptively achieve good performance on task t, it will also forget the knowledge learned from task t-1. In step t, the goal is to minimize the new dataset S t The predefined loss on the parameter θ′ is used to find the parameter θ′:

[0071]

[0072] in, Represents the dataset S from task t t The expected value of the sample (x,y) sampled in, is the cross entropy loss function, Represents the i-th image Input to the feature extractor with parameter θ middle.

[0073] Like other class-incremental methods, this embodiment seeks to classify all observed classes without accessing old data.

[0074] See Figure 1 and Figure 2 , Figure 1 A flowchart of a self-supervised incremental learning method based on similarity prototype calibration provided by an embodiment of the present invention is provided. Figure 2 The overall framework diagram of the self-supervised incremental learning method based on similarity prototype calibration provided by an embodiment of the present invention is as follows: Figure 2 The left side shows an overview of the self-supervised incremental learning method based on similarity prototype calibration. Figure 2 The right side is a detailed description of the prototype calibration scheme, where the prototype calibration scheme calibrates the biased prototypes by fusing the old category prototypes and the weighted prototype representation.

[0075] The self-supervised incremental learning method based on similarity prototype calibration includes the following steps:

[0076] S1. Extract the data features of each new category of the current task, and use the data features to calculate the prototype of each new category.

[0077] Specifically, the expanded category information and enhanced data of the current task are provided to the feature extractor to obtain the data features of each new category, and the data features are averaged to obtain the prototype of each new category.

[0078] S2. Using the similarity between each new category prototype and the old category prototype, obtain a set of most similar categories and a weight of each prototype in the set.

[0079] The prototype of the old category is calibrated by migrating its statistical information through the feature distribution of the new category. The focus of prototype calibration is to reduce the feature distribution difference of the prototype, aiming to alleviate overfitting on the old dataset, obtain a more accurate prototype representation, and further generate a classifier with better generalization ability. Compared with the prototype features, because there is enough training data, the new category has more accurate feature distribution statistics. Inspired by this, the prototype of the old category can be calibrated by migrating the statistical information of the new category prototype to reduce the feature difference. Specifically, for the prototype of the old category c, the similarity between the new category (taking q as an example) and the old category prototype is:

[0080]

[0081] Among them, s c,q represents the similarity between the prototype of the old category c and the prototype of the new category q, Represents the old category prototype, represents the new category prototype, τ represents the temperature parameter used to control the sharpness of the weight distribution;

[0082] Then, the ranking of similarity s is calculated and the prototypes of the old category c are arranged in descending order of similarity scores as Γ c =argtopk{s c,q}, and select a preset number of top-k new category prototypes and old category prototypes in descending order of similarity as the set of most similar categories. Use the softmax function to calculate the weight of each prototype in the set:

[0083]

[0084] Among them, ω c,q Represents the weight of the classification prototype, s c,q represents the similarity between the prototype of the old category c and the prototype of the new category q, s l,qRepresents the similarity between the prototype of the old category l and the prototype of the new category q.

[0085] S3. Perform weighted calculation on the new category prototype and the old category prototype using the weight of each prototype to obtain a calibrated prototype.

[0086] Specifically, the features of the new category and the features of the old category obtained by migration are weighted. Therefore, the biased calibration prototype is calculated using the following weighted formula:

[0087]

[0088] in, represents the calibrated prototype, Represents the old category prototype, represents the prototype of the new category h, ω h represents the weight of the prototype of the new category h, k represents the number of categories, λ c represents a trade-off parameter.

[0089] The similarity-based prototype calibration method of this embodiment recalculates the prototype of the old category by fusing the biased prototype with the selected calibration item, thereby ensuring a more accurate representation of the prototype of the old category.

[0090] Integrate the knowledge learned from different transformations into another classifier. To make the model easily adaptable to new data and thus offset the problem of catastrophic forgetting, this example designs a self-distillation method (AASD) to transfer the knowledge of different data enhancements into a single inference, aiming to fully utilize the information learned from the dataset.

[0091] See Figure 3 , Figure 3 Schematic diagram of asymmetric polymerization self-distillation provided by an embodiment of the present invention. The asymmetric polymerization self-distillation method includes steps S4, S5, and S6.

[0092] S4. Calculate the target logit parameter of each transformation using the affinity between several augmented images obtained by transforming the original image.

[0093] First, the original image is transformed to obtain several augmented images, and the original image is input into the feature extractor to obtain the original logit parameters of each transformation.

[0094] Specifically, N samples are used as input data, each sample has an original image, and the original image is transformed (for example, rotated) to obtain M augmented images. The input data of each class of the original image and augmented image is defined as in, Represents the image set after the original image has been augmented (i.e. transformed) j times, Represents the jth augmented image, and the label of each augmented image is It is understood that an original image is transformed j times to obtain j augmented images.

[0095] The original logit parameters for each transformation are defined as:

[0096]

[0097] Among them, s j represents the original logit parameter of the jth transformation, ω j represents the weight of the j-th transformation classifier, F j represents the features of the j-th transformation.

[0098] In self-supervised learning, not all augmentation information is symmetrically affected by semantic information learning. Figure 4 As shown, Figure 4 This is a schematic diagram of the importance of different enhancement types in different categories provided by an embodiment of the present invention. Figure 4 In the example above, when classifying "mobile phone" and "remote control," a 180-degree rotation increases the difficulty because rotation changes semantic information, not color. Meanwhile, when classifying "blue can" and "green can," color information is more valuable than rotation information. Therefore, symmetrically aggregating all branches does not accurately measure the importance of different knowledge to feature representation learning.

[0099] In order to make different transformations converge better, recalculate the original logit parameter s j , aggregate the knowledge of each enhancement by batch weighted propagation of all samples. Given N samples, one of which contains M augmented images, the knowledge propagation is weighted by calculating the affinity between the M augmented images, and the affinity matrix is ​​obtained:

[0100]

[0101] Among them, A(i,j) represents the affinity between the i-th augmented image and the j-th augmented image, i and j both represent the index of the augmentation number, s i represents the original logit parameter of the i-th transformation.

[0102] To avoid self-reinforcement, remove the diagonal elements from the affinity matrix A and obtain the association matrix:

[0103] A=A⊙(1-I)

[0104] Where I is the identity matrix and ⊙ is element-wise multiplication.

[0105] Then, each row of the association matrix of the affinity matrix A is normalized to obtain the normalized affinity matrix of the augmented image i At the same time, keep the elements on the diagonal to 0, where the normalized affinity matrix of the augmented image i can be expressed as the softmax function of each row of the association matrix:

[0106]

[0107] Where M represents the number of augmented images in a sample, and N represents the number of samples.

[0108] The normalized affinity matrix reflects the relationship between different enhancements in the current batch dataset and indirectly explains which enhancement information is more valuable for feature representation learning. If the i-th sample is similar to the j-th sample, it has a higher affinity. Then predict s j will have a greater weight to propagate to s i .

[0109] The target logit parameters for each transformation are recalculated using the normalized affinity matrix:

[0110]

[0111] in, represents the target logit parameter of the j-th transformation, represents the normalized affinity matrix.

[0112] S5. Aggregate the knowledge of the original category of the old task using the target logit parameter and the calibrated prototype to obtain a first aggregated inference probability, and aggregate all transformations of each new category from the original image to the augmented image for the current task to obtain a second aggregated inference probability.

[0113] Specifically, for the old task, only the knowledge of the original class is aggregated into the first aggregated inference probability:

[0114]

[0115] in, Represents the inference probability of the old class feature, F old represents the features generated by prototype calibration, c represents the categories that have been seen, K represents the total number of categories that have been seen, Indicates the classifier in stage T Medium F old The transpose of the target logit parameter for category c, Indicates the classifier in stage T Medium F oldThe transpose of the target logit parameter for any class k.

[0116] For the task at the current stage, all transformations of each class are aggregated into the second aggregate inference probability:

[0117]

[0118] in, Represents the aggregate inference probability of new class features after different transformations, Represents the augmented image after j transformations Features, Represents the augmented image The feature set extracted from the image, M represents the number of augmented images, Indicates the classifier in stage T middle The transpose of the target logit parameter of category c after j transformations, Indicates the classifier in stage T middle The transpose of the target logit parameter for any category k after j transformations.

[0119] S6. Calculate a first self-distillation loss based on the first aggregated inference probability and the single inference probability for classifying the original image features, calculate a second self-distillation loss based on the second aggregated inference probability and the single inference probability for classifying the original image features, and minimize the first self-distillation loss and the second self-distillation loss.

[0120] Specifically, the first self-distillation loss is calculated using the KL divergence between the first aggregated inference probability and the single inference probability of classifying the original image features:

[0121]

[0122] in, represents the first self-distillation loss, is the KL divergence, Represents the classifier after different transformations The first aggregate inference probability of the output, Represents a classifier The output, It is a feature extractor Parameters.

[0123] The second self-distillation loss is calculated using the KL divergence between the second aggregated inference probability and the single inference probability of classifying the original image features:

[0124]

[0125] in, represents the second self-distillation loss, is the KL divergence, Represents the classifier after different transformations The second aggregate inference probability of the output, Represents a classifier The output, The output of represents the single inference probability, It is a feature extractor Parameters.

[0126] The self-distillation loss is minimized to regularize the feature extractor.

[0127] S7. Calculate a target loss function based on the first self-distillation loss and the second self-distillation loss and minimize the target loss function to obtain an optimized classification model, wherein the optimized classification model is used to classify the input image.

[0128] Specifically, first, minimize the current model trained on new data and the previous model trained with the previous data The characteristic Euclidean distance between:

[0129]

[0130] get After that, calculate the cross entropy loss to optimize the classifier and Then, the overall target loss function is obtained, and the target loss function is minimized to obtain a trained classification model. The trained classification model includes a feature extractor and a unified classifier connected in sequence. The trained classification model is used to classify the input image to obtain better classification results. Among them, the target loss function is:

[0131]

[0132] in, is the total loss function (i.e., the target loss function), represents the cross entropy loss, and denote the loss of new data and the loss of old class features obtained by prototype calibration, respectively. Represents the current model trained on new data and the previous model trained with the previous data The characteristic Euclidean distance between represents the first self-distillation loss, represents the second self-distillation loss, ||·||2 represents L2 regularization, α and γ represent loss weights, represents a classifier that distinguishes between augmented images and original images, Represents a classifier Parameters, represents the classifier only for the original image, φ t Represents a classifier Parameters, represents the i-th input image, represents the i-th augmented image, θ t Represents the current feature extractor The parameter θ t-1 Represents the previous feature extractor Parameters, Represents a feature extractor for old categorical data.

[0133] See Figure 5 , Figure 5 A schematic diagram illustrating changes in classification accuracy when different methods provided in embodiments of the present invention are used for learning tasks on CIFAR-100 and TinyImageNet.

[0134] In order to evaluate the performance of the method of this embodiment, experiments were conducted on the CIFAR-100 and TinyImageNet datasets, respectively, with three different incremental stages (5, 10, and 20 stages, where the number of stages represents the number of times data or tasks are introduced during the learning process).

[0135] The results are compared with the best methods of current incremental learning without examples. The main evaluation indicators are classification accuracy and forgetting rate. Figure 5 It can be seen that the accuracy of the method of this embodiment (the method of this embodiment is the blue broken line, and the broken lines of other colors are methods in the prior art) on the two data sets is significantly improved. At the same time, it can be clearly seen from the decrease in the broken line that when the number of training tasks increases, the method of this embodiment can effectively offset the catastrophic forgetting problem.

[0136] This embodiment uses the set of categories with the most similar similarities between the new category prototype and the old category prototype and the weight of each prototype, and uses the weight to perform weighted calculation to obtain a calibrated prototype. By aggregating the new prototypes with the highest similarity to the old prototypes, the evolution of the representation of the old category prototype is tracked. By adjusting the features of the old category to the current representation space, the overlap between the new and old categories is effectively reduced, which not only alleviates the distinction between the retained and updated networks, but also has practical significance for the model performance.

[0137] This embodiment is based on an asymmetric aggregation self-distillation strategy for batch knowledge integration, uses affinity to calculate logit parameters, aggregates the knowledge of the original categories of the old tasks, and aggregates all transformations of each new category for the current task. Asymmetric aggregation self-distillation is achieved by selectively extracting the most valuable information from the new categories and aggregating it with the existing knowledge in an asymmetric manner. By integrating the knowledge learned in the same batch, it selectively aggregates valuable knowledge from the enhanced images, makes full use of the knowledge learned from the new categories, and significantly improves the plasticity of the model, making it more adaptable to real-world scenarios and able to efficiently adapt to new tasks.

[0138] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A self-supervised incremental learning method based on similarity prototype calibration, characterized in that: Including steps: S1. Extract the data features of each new category of the current task and use the data features to calculate the prototype of each new category; S2. Obtaining a set of most similar categories and a weight of each prototype in the set using the similarity between each new category prototype and the old category prototype; S3. Perform weighted calculation on the new category prototype and the old category prototype using the weight of each prototype to obtain a calibrated prototype; S4, calculating the target logit parameter of each transformation using the affinity between several augmented images obtained by transforming the original image; S5. Aggregate the knowledge of the original category of the old task using the target logit parameter and the calibrated prototype to obtain a first aggregated inference probability, and aggregate all transformations of each new category from the original image to the augmented image for the current task to obtain a second aggregated inference probability; S6. Calculate a first self-distillation loss based on the first aggregated inference probability and the single inference probability of classifying the original image features, calculate a second self-distillation loss based on the second aggregated inference probability and the single inference probability of classifying the original image features, and minimize the first self-distillation loss and the second self-distillation loss; S7. Calculate a target loss function based on the first self-distillation loss and the second self-distillation loss and minimize the target loss function to obtain an optimized classification model, wherein the optimized classification model is used to classify the input image.

2. The self-supervised incremental learning method based on similarity prototype calibration according to claim 1, characterized in that Step S2 includes: Calculate the similarity between each new category prototype and the old category prototype: Among them, s c,q represents the similarity between the prototype of the old category c and the prototype of the new category q, Represents the old category prototype, represents the new category prototype, τ represents the temperature parameter used to control the sharpness of the weight distribution; The similarities are arranged in descending order as Γ c =argtopk{s c,q }, and select a preset number of new category prototypes and old category prototypes in descending order of similarity as the set of most similar categories; The softmax function is used to calculate the weight of each prototype in the set: Among them, ω c,q Represents the weight of the classification prototype, s c,q represents the similarity between the prototype of the old category c and the prototype of the new category q, s l,q Represents the similarity between the prototype of the old category l and the prototype of the new category q.

3. The self-supervised incremental learning method based on similarity prototype calibration according to claim 1, characterized in that The formula for weighted calculation in step S3 is: in, represents the calibrated prototype, Represents the old category prototype, represents the prototype of the new category h, ω h represents the weight of the prototype of the new category h, k represents the number of categories, λ c represents a trade-off parameter.

4. The self-supervised incremental learning method based on similarity prototype calibration according to claim 1, characterized in that Step S4 includes: The original image is transformed into several augmented images, and the original image is input into the feature extractor to obtain the original logit parameters of each transformation: Among them, s j represents the original logit parameter of the jth transformation, ω j represents the weight of the j-th transformation classifier, F j represents the features of the j-th transformation; Calculate the affinity between several augmented images and obtain the affinity matrix: Among them, A(i,j) represents the affinity between the i-th augmented image and the j-th augmented image, i and j represent the indexes of different augmented images, s i represents the original logit parameter of the i-th transformation; Deleting the diagonal elements from the affinity matrix yields the association matrix: A=A⊙(1-I) Where I is the identity matrix and ⊙ is element-wise multiplication; Normalize each row of the association matrix to obtain the normalized affinity matrix of the augmented image i, while keeping all elements on the diagonal zero. The normalized affinity matrix of the augmented image i is: Where M represents the number of augmented images in a sample, and N represents the number of samples; The target logit parameters for each transformation are calculated using the normalized affinity matrix: in, represents the target logit parameter of the j-th transformation, represents the normalized affinity matrix.

5. The self-supervised incremental learning method based on similarity prototype calibration according to claim 1, characterized in that The first aggregate inference probability is: in, Represents the inference probability of the old class feature, F old represents the features generated by prototype calibration, c represents the categories that have been seen, K represents the total number of categories that have been seen, Indicates the classifier in stage T Medium F old The transpose of the target logit parameter for category c, Indicates the classifier in stage T Medium F old The transpose of the target logit parameter for any class k.

6. The self-supervised incremental learning method based on similarity prototype calibration according to claim 1, characterized in that The second aggregate inference probability is: in, Represents the aggregate inference probability of new class features after different transformations, Represents the augmented image after j transformations Features, Represents the augmented image The feature set extracted from the image, M represents the number of augmented images, Indicates the classifier in stage T middle The transpose of the target logit parameter of category c after j transformations, Indicates the classifier in stage T middle The transpose of the target logit parameter for any category k after j transformations.

7. The self-supervised incremental learning method based on similarity prototype calibration according to claim 1, characterized in that: Step S6 includes: The first self-distillation loss is calculated using the KL divergence between the first aggregated inference probability and the single inference probability of classifying the original image features: in, represents the first self-distillation loss, is the KL divergence, Represents the classifier after different transformations The first aggregate inference probability of the output, Represents a classifier The output, It is a feature extractor Parameters; The second self-distillation loss is calculated using the KL divergence between the second aggregated inference probability and the single inference probability of classifying the original image features: in, represents the second self-distillation loss, is the KL divergence, Represents the classifier after different transformations The second aggregate inference probability of the output, Represents a classifier The output, It is a feature extractor Parameters; The first self-distillation loss and the second self-distillation loss are minimized to regularize the feature extractor.

8. The self-supervised incremental learning method based on similarity prototype calibration according to claim 7, characterized in that: The objective loss function is: in, is the target loss function, represents the cross entropy loss, and denote the loss of new data and the loss of old class features obtained by prototype calibration, respectively. Represents the current model trained on new data and the previous model trained with the previous data The characteristic Euclidean distance between represents the first self-distillation loss, represents the second self-distillation loss, ||·||2 represents L2 regularization, α and γ represent loss weights, represents a classifier that distinguishes between augmented images and original images, Represents a classifier Parameters, represents the classifier only for the original image, φ t Represents a classifier Parameters, represents the i-th input image, represents the i-th augmented image, θ t Represents the current feature extractor The parameter θ t-1 Represents the previous feature extractor Parameters, Represents a feature extractor for old categorical data.

Images