Small sample continuous learning method and system based on continuous knowledge protection decomposition

Abstract

The present invention belongs to the field of artificial intelligence and deep learning. In order to solve the problems of imbalance and poor adaptability of new and old knowledge, a small sample continuous learning method and system based on continuous knowledge protection decomposition are provided. Among them, the small sample continuous learning method based on continuous knowledge protection decomposition includes replaying sample data of each category and the backbone network of the continuous learning model, obtaining knowledge sensitive components and redundant capacity components through singular value decomposition; freezing the pre-trained linear weight matrix corresponding to the knowledge sensitive component during the incremental adaptive training process; at the same time, using the redundant capacity component to construct a learnable adapter and update the pre-trained linear weight matrix; reacquiring small sample replay data, and performing singular value decomposition and incremental adaptive training operations again based on the updated pre-trained linear weight matrix. It achieves the balance and efficient adaptation of new and old knowledge through the dynamic update of the covariance matrix, and improves the accuracy of the classification results.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and deep learning, and in particular, relates to a small sample continuous learning method and system based on continuous knowledge protection decomposition. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Few-shot continuous learning aims to preserve existing knowledge while learning new categories with limited data. However, this field faces the challenge of catastrophic forgetting, that is, when learning new categories, the model easily overwrites previous knowledge. In addition, the scarcity of labeled samples means that there is less labeled data available for learning new categories, which increases the risk of overfitting and further weakens the generalization ability of the model. In practical applications, such as adaptive recommendation systems and robot navigation in dynamic environments, few-shot continuous learning algorithms need to ensure that the model can effectively adapt to new information under limited resources without affecting the stability of existing knowledge. To address the above challenges, existing methods can be divided into the following categories: (1) Data replay methods: Alleviate the forgetting problem by storing or generating data of learned categories, but usually bring storage costs or unstable generation quality problems; (2) Optimization-based methods: Use meta-learning, special loss functions or geometric constraints to improve learning effects from limited data, but it is difficult to balance stability and adaptability; (3) Dynamic adaptation techniques: Introduce dynamic architectures or learnable adapters to adapt to new categories, but may increase model complexity and inference overhead.

[0004] Although existing methods have improved performance to a certain extent, they still have significant limitations in balancing stability and adaptability. First, completely fine-tuning the model is prone to catastrophic forgetting. Therefore, most methods choose to freeze the backbone network to protect existing knowledge, but this limits the model's ability to adapt to new tasks and easily leads to insufficient learning ability of the model for new categories. Secondly, some methods introduce additional task-specific learnable modules and only fine-tune them during the adaptation phase. Although such methods improve the model's adaptability to new tasks, they inevitably increase the number of model parameters and reasoning complexity, and are prone to catastrophic forgetting of existing knowledge, thereby reducing the accuracy of the classification results. Summary of the invention

[0005] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a small sample continuous learning method and system based on continuous knowledge protection decomposition, which adopts the method of continuous knowledge protection decomposition and can achieve the balance and efficient adaptation of new and old knowledge through the dynamic update of the covariance matrix, thereby improving the accuracy of the classification results.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A first aspect of the present invention provides a small sample continuous learning method based on continuous knowledge protection decomposition.

[0008] A small sample continuous learning method based on continuous knowledge preservation decomposition, comprising:

[0009] Obtain the playback sample data of each category and extract the playback sample features. Based on the continuous learning model backbone network, calculate the covariance matrix of the activation values ​​corresponding to the playback sample features in the current linear layer of the continuous learning model; multiply the current pre-trained linear weight matrix with the covariance matrix of the activation values ​​of the current linear layer, and then divide the multiplication result into two parts: knowledge sensitive components and redundant capacity components through singular value decomposition;

[0010] In the incremental adaptive training process, the pre-trained linear weight matrix corresponding to the knowledge-sensitive component is frozen, and it is multiplied with the training sample features of the current incremental adaptive training stage to obtain the knowledge-sensitive features; at the same time, the redundant capacity component is used to construct a learnable adapter, which is multiplied with the training sample features of the current incremental adaptive training stage to obtain the redundant capacity features; the knowledge-sensitive features and the redundant capacity features are then superimposed to obtain the output features obtained by the current linear layer transformation and passed to the next layer of the continuous learning model; after supervised training, the updated pre-trained linear weight matrix is ​​obtained;

[0011] Re-acquire the small sample playback data, and perform singular value decomposition and incremental adaptation training operations again based on the updated pre-trained linear weight matrix until the continuous learning model meets the set requirements and stops learning, so as to use the trained continuous learning model to perform classification tasks.

[0012] As an implementation method, the covariance matrix of the activation values ​​is expressed as:

[0013] ,

[0014] in, represents the covariance matrix; Represents the activation value of the playback sample feature at the current layer; express Transpose; is the number of playback samples; Represents the input dimension of the pre-trained linear weight matrix.

[0015] As an implementation method, the process of singular value decomposition is:

[0016] ,

[0017] in, represents singular value decomposition; and is an orthogonal matrix, is a diagonal matrix containing the singular values ​​arranged from largest to smallest ; represents the covariance matrix; represents the pre-trained linear weight matrix; represents the rank of the pre-trained linear weight matrix; represents the left singular vector; is a matrix No. OK; Represents the transpose of a matrix.

[0018] As an implementation method, the first Rr singular vectors with larger singular values ​​are used as knowledge sensitive components; the last r singular vectors with smaller corresponding singular values ​​are used as redundant capacity components; where R represents the rank of the pre-trained linear weight matrix, which is the number of non-zero singular values, , Represents the input dimension of the pre-trained linear weight matrix; represents the output dimension of the pre-trained linear weight matrix; r is a preset hyperparameter used to control the number of redundant components.

[0019] As an implementation, the redundant capacity components are used to construct two learnable low-rank matrices and As an adapter:

[0020]

[0021]

[0022]

[0023]

[0024]

[0025] in, Represents the pre-trained linear weight matrix corresponding to the knowledge sensitive component; represents the pre-trained linear weight matrix; and is an orthogonal matrix; represents the inverse of the covariance matrix; represents the left singular vector; represents singular values; is a matrix No. OK; Represents the transpose of a matrix; Indicates the last List, and Represent the adapter matrix respectively; during the adaptation process, by only updating and Freeze way to retain what you have learned.

[0026] As an implementation method, the expression of the updated pre-trained linear weight matrix is:

[0027]

[0028] in, represents the updated pre-trained linear weight matrix; Represents the pre-trained linear weight matrix corresponding to the knowledge sensitive component; and A matrix representing the updated parameters of the adapter corresponding to the redundant capacity component.

[0029] As an implementation mode, the training sample features of the current incremental adaptive training stage are extracted from the training sample data of the current incremental adaptive training stage; the training sample data of the current incremental adaptive training stage is composed of initial training sample data and playback sample data.

[0030] A second aspect of the present invention provides a small sample continuous learning system based on continuous knowledge protection decomposition.

[0031] A small sample continuous learning system based on continuous knowledge preservation decomposition, comprising:

[0032] The weight decomposition module is used to obtain the playback sample data of each category and extract the playback sample features. Based on the continuous learning model backbone network, the covariance matrix of the activation values ​​corresponding to the playback sample features in the current linear layer of the continuous learning model is calculated; the current pre-trained linear weight matrix is ​​multiplied by the covariance matrix of the activation values ​​of the current linear layer, and the multiplication result is divided into two parts: knowledge sensitive components and redundant capacity components through singular value decomposition;

[0033] The weight update module is used to freeze the pre-trained linear weight matrix corresponding to the knowledge-sensitive component during the incremental adaptive training process, and multiply it with the training sample features of the current incremental adaptive training stage to obtain the knowledge-sensitive features; at the same time, the redundant capacity component is used to construct a learnable adapter, and it is multiplied with the training sample features of the current incremental adaptive training stage to obtain the redundant capacity features; the knowledge-sensitive features and the redundant capacity features are then superimposed to obtain the output features obtained by the current linear layer transformation and passed to the next layer of the continuous learning model to obtain the updated pre-trained linear weight matrix;

[0034] The adaptive training module is used to reacquire small sample playback data, and perform singular value decomposition and incremental adaptive training operations again based on the updated pre-trained linear weight matrix until the continuous learning model meets the set requirements and stops learning, so as to use the trained continuous learning model to perform classification tasks.

[0035] A third aspect of the present invention provides a computer-readable storage medium.

[0036] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the small sample continuous learning method based on continuous knowledge protection decomposition as described above.

[0037] A fourth aspect of the present invention provides an electronic device.

[0038] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the small sample continuous learning method based on continuous knowledge protection decomposition as described above are implemented.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] (1) The present invention proposes a continuous knowledge protection decomposition method, which divides the model weights into knowledge sensitive components and redundant capacity components through singular value decomposition guided by the covariance matrix. The components are used to protect the representation space of learned knowledge and the redundant space available for learning new knowledge, respectively. This method effectively solves the contradiction between the traditional method of freezing the backbone network to limit adaptability and the complete fine-tuning to cause catastrophic forgetting, thereby improving the accuracy of the classification results.

[0041] (2) The present invention guides the weight decomposition by replaying the covariance matrix of the samples, closely aligning the principal components with the classification capabilities of representative samples, thereby allocating additional feature space for new knowledge learning. This overcomes the defect of the traditional weight partitioning method based on singular value decomposition that cannot focus on the model's classification ability for the learned categories, improves the adaptability of the model, maintains the generalization of the model, and ensures the accuracy of the classification task execution results.

[0042] (3) In the continuous learning process of the present invention, different layers have different redundant capacity requirements for new knowledge, and the redundant capacity requirements change as the continuous learning progresses. The present invention adjusts the weight decomposition strategy in real time to adapt to the evolving model capabilities by recalculating the covariance matrix before the start of a new training phase, thereby achieving dynamic optimization of the balance between new and old knowledge and ensuring the accuracy of subsequent classification tasks.

[0043] (4) The present invention restores the original structure of the model by reintegrating the adapter with the frozen weights after fine-tuning it. While maintaining the reasoning efficiency, it ensures the efficient integration of new and old knowledge, thus solving the problem of traditional methods increasing the reasoning overhead by introducing additional parameters or modules to improve the adaptability of the model.

[0044] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0046] Figure 1 This is a flow chart of a small sample continuous learning method based on continuous knowledge protection decomposition according to an embodiment of the present invention;

[0047] Figure 2 It is a schematic diagram of the structure of a small sample continuous learning system based on continuous knowledge protection decomposition according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0049] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0050] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0051] Terminology explanation:

[0052] Continuous learning with small samples:

[0053] The goal is to learn a model that can continuously learn knowledge from a series of new categories using only a small number of labeled training samples while retaining the knowledge gained from previous categories. The data of each incremental adaptation stage can be represented as In each training stage t, the model receives the training set , which contains the input sample and its corresponding tags Initial training phase It is also called the basic training phase, and its label set There are sufficient labels and samples to enable the model to obtain good basic representation capabilities and lay the foundation for subsequent continuous learning. In the training process, the model is gradually introduced to new categories, each category has only a small number of labeled samples, and there is no overlap between the category sets of different training stages (such as stage t and stage t'). During the new phase of training, only limited replay samples from the learned categories can be accessed (usually only one replay sample for a category) to maintain the learned knowledge, and the complete data of the previous training phase cannot be directly obtained. In the model evaluation phase after the training phase t, the model needs to be tested on all learned categories to evaluate the model's ability to learn new categories and maintain the learned categories.

[0054] Embodiment 1

[0055] according to Figure 1 As shown, this embodiment provides a small sample continuous learning method based on continuous knowledge protection decomposition, including:

[0056] Step 1: Obtain the playback sample data of each category and extract the playback sample features. Based on the continuous learning model backbone network, calculate the covariance matrix of the activation values ​​corresponding to the playback sample features in the current linear layer of the continuous learning model; multiply the current pre-trained linear weight matrix with the covariance matrix of the activation value of the current linear layer, and then divide the multiplication result into two parts: knowledge sensitive component and redundant capacity component through singular value decomposition;

[0057] Step 2: During the incremental adaptive training process, freeze the pre-trained linear weight matrix corresponding to the knowledge-sensitive component, multiply it with the training sample features of the current incremental adaptive training stage to obtain the knowledge-sensitive features; at the same time, use the redundant capacity component to build a learnable adapter, multiply it with the training sample features of the current incremental adaptive training stage to obtain the redundant capacity features; then superimpose the knowledge-sensitive features and the redundant capacity features to obtain the output features obtained by the current linear layer transformation and pass them to the next layer of the continuous learning model to obtain the updated pre-trained linear weight matrix;

[0058] Step 3: Re-acquire the small sample playback data, and perform singular value decomposition and incremental adaptation training operations again based on the updated pre-trained linear weight matrix until the continuous learning model meets the set requirements and stops learning, so as to use the trained continuous learning model to perform the classification task.

[0059] The continuous knowledge preserving decomposition method for continuous learning of small samples in this embodiment can continuously separate redundant components from basic components related to the acquired knowledge and allocate additional space for new knowledge. Since the output feature space of the linear projection layer in the neural network usually contains a large between-class variance and a small within-class variance, the principal components of the covariance matrix projected to the output space can capture the most influential and discriminative elements that contribute to classification. The covariance matrix of the output features of the intermediate layer of the model is constructed based on the playback samples, and the product of the covariance matrix and the weight matrix is ​​subjected to singular value decomposition. Such weight decomposition guided by the covariance matrix can ensure that the decomposed principal components are closely aligned with the classification capabilities of representative samples of the learned knowledge.

[0060] To achieve the above purpose, in step 1, in each incremental training phase To get started, collect a small sample dataset of replays. ,in Represents a union; each old category contains only one sample. Each old category contains only one sample. Input the replayed sample into the model backbone network The activation values ​​of the intermediate layers of the model are calculated in , and the covariance matrix is ​​calculated before each linear layer :

[0061] ,

[0062] in, represents the covariance matrix; Represents the activation value of the playback sample feature at the current layer; express Transpose; is the number of playback samples; Represents the input dimension of the pre-trained linear weight matrix.

[0063] Among them, the process of singular value decomposition of the product of the covariance matrix of each activation value and the linear weight matrix is:

[0064] ,

[0065] in, represents singular value decomposition; and is an orthogonal matrix, is a diagonal matrix containing the singular values ​​arranged from largest to smallest ; represents the covariance matrix; represents the pre-trained linear weight matrix; represents the rank of the pre-trained linear weight matrix; represents the left singular vector; is a matrix No. OK; Represents the transpose of a matrix.

[0066] In the specific implementation process, the first Rr singular vectors with larger singular values ​​are used as knowledge sensitive components; the last r singular vectors with smaller corresponding singular values ​​are used as redundant capacity components; where R represents the rank of the pre-trained linear weight matrix, which is the number of non-zero singular values, , Represents the input dimension of the pre-trained linear weight matrix; Represents the output dimension of the pre-trained linear weight matrix.

[0067] Generally, r is set to 64; the selection of the r value can be determined based on actual applications or experimental results, usually through cross-validation or model performance evaluation.

[0068] Among them, the knowledge-sensitive component and the redundant capacity component are complementary. The redundant capacity component can minimize the interference to existing capabilities to ensure the stability of old categories and the plasticity of new tasks.

[0069] In the specific implementation of step 2, the redundant capacity components are used to construct two learnable low-rank matrices and As an adapter:

[0070]

[0071]

[0072]

[0073]

[0074]

[0075] in, Represents the pre-trained linear weight matrix corresponding to the knowledge sensitive component; represents the pre-trained linear weight matrix; and is an orthogonal matrix; represents the inverse of the covariance matrix; represents the left singular vector; represents singular values; is a matrix No. OK; Represents the transpose of a matrix; Indicates the last List, and Represent the adapter matrix respectively; during the adaptation process, by only updating and Freeze way to retain what you have learned.

[0076] During the incremental training process of the tth stage, each training stage will contain a new set of training samples and corresponding labels. The input data may be images, text, or other forms of samples. These data first pass through the input layer of the backbone network, pass to the linear layer layer by layer, and finally pass through the adapter adjustment. For each input sample, the features of the i-th training sample are processed by the linear layer in the network. The current linear layer is calculated as follows:

[0077] AX

[0078] in, represents the pre-trained linear weight matrix corresponding to the knowledge sensitive component; B and A are a set of learnable adapter matrices; is the input feature of the current linear layer; It is the output feature obtained by the current linear layer transformation and passed to the next layer of the network.

[0079] In this process, the role of the adapter is to fine-tune the weights of the network, especially when adding new categories to the training process, the adapter will perform adaptive fine-tuning based on the original model. The adapter effectively integrates new knowledge into the original classification capabilities by calculating the combination of input features and the new weight matrix.

[0080] The loss function is usually used to measure the difference between the model output and the true label. In incremental training, the loss function calculation usually depends on the output of the network and the true label. (That is, the actual label of the input sample). Assume that we use the cross entropy loss function as the loss function in common classification tasks, and its calculation formula is:

[0081]

[0082] in, Represents the loss value of the i-th training sample; Represents the true label of the i-th sample in category c (usually 0 or 1, indicating whether it belongs to the category); Represents the predicted probability of the i-th sample in category c, which is usually obtained through the softmax layer of the model.

[0083] After calculating the loss, the back-propagation algorithm is used to calculate the gradient of each model parameter. The gradient represents the rate of change of the loss function with respect to the adapter parameters. Through back-propagation, the gradient of the loss with respect to the weight matrices A and B is calculated, and the model parameters are updated according to these gradients to minimize the loss.

[0084] The expression of the updated pre-trained linear weight matrix is: ;

[0085] in, represents the updated pre-trained linear weight matrix; W Represents the pre-trained linear weight matrix corresponding to the knowledge sensitive component; and Represents the adapter update parameter matrix corresponding to the redundant capacity components.

[0086] Specifically, in step 3, the covariance matrix is ​​recalculated before each new adaptation phase begins. Repeat the above decomposition process to integrate the newly acquired capabilities into the knowledge-sensitive components, so that this method can adapt to the evolving model capabilities in continuous learning of small samples. In the adaptation phase, it will be updated to include all previously learned categories:

[0087] .

[0088] In summary, the above framework applies continuous knowledge protection decomposition at specific layers of the backbone model, which can effectively protect the learned knowledge while improving the adaptation performance to new tasks without changing the model architecture or increasing the reasoning overhead.

[0089] Compared with the previous technologies, the continuous knowledge protection decomposition method proposed in the present invention has the following significant beneficial effects:

[0090] (1) Higher average accuracy: The continuous knowledge-preserving decomposition method proposed in this paper performs well in multiple incremental learning continuous training stages, especially on the miniImageNet and CUB-200 datasets, showing higher accuracy than existing methods. For example, on the miniImageNet dataset, the average accuracy of the continuous decomposition adaptation method proposed in this paper is 90.66%, far exceeding existing methods such as SAVC (67.05%) and FeSSSS (68.23%). On the CUB-200 dataset, the average accuracy of the continuous decomposition adaptation method proposed in this paper is 79.00%, which is also higher than many competing methods, such as KA-Net (73.17%).

[0091] (2) Stronger knowledge retention capability: In incremental learning tasks that gradually introduce new categories, the present invention can effectively adapt to new categories and retain memory of previous knowledge. For example, in the last continuous learning phase of miniImageNet, the accuracy of the continuous decomposition adaptation method proposed by the present invention was 86.23%, significantly better than other methods, proving that the method has good adaptability when new categories are introduced and can effectively reduce catastrophic forgetting.

[0092] (2) Strong generalization ability: The present invention not only maintains high performance on the miniImageNet dataset, but also performs well on the more challenging CUB-200 dataset, with an average accuracy of 79.00%. This result shows that the continuous decomposition and adaptation method proposed in the present invention has good generalization ability, can adapt to different incremental learning tasks, and performs robustly on multiple datasets.

[0093] In summary, the continuous knowledge-preserving decomposition method of the present invention shows superior performance in incremental learning and small-sample classification tasks, can effectively cope with the gradual introduction of new categories, and surpasses existing advanced technologies on multiple datasets.

[0094] Table 1 Comparison of the present invention with other methods on miniImageNet

[0095]

[0096] in:

[0097] DSN: It is a small sample continuous learning algorithm based on the Dynamic Support Network (DSN), which uses the dynamic support network node expansion method to support the adaptive update network of the feature space.

[0098] Data-free: It is a small sample continuous learning algorithm based on data free replay. It synthesizes the learned data through the generator to keep the learned knowledge from being forgotten without accessing the real data.

[0099] MetaFSCIL: is a Few-shot Class-incremental Learning (FSCIL) algorithm based on meta-learning. It uses meta-objectives to learn and quickly adapt to new tasks without forgetting what has been learned.

[0100] LIMIT: It is a small-sample continuous learning algorithm that learns multi-phase incremental tasks (LIMIT). It constructs a generalizable feature space for unseen tasks by using base class data to construct pseudo small-sample continuous learning tasks.

[0101] FACT: It is a small sample continuous learning algorithm based on forward compatible training (FACT). It squeezes the embedding space of known categories by assigning virtual cluster centers to provide feature space for the embedding of new categories.

[0102] CABD: is a small-sample continuous learning algorithm based on class-aware bilateral distillation (CABD), which is used to adaptively learn knowledge from two complementary teachers.

[0103] TEEN: It is a small sample continuous learning algorithm based on the training-free calibration strategy (Training-frEE calibration), which enhances the discriminability of new categories by fusing the new cluster centers (i.e., the average features of the categories) with the weighted basic prototypes.

[0104] C-FSCIL: is a Constrained Few-shot Class-Incremental Learning (C-FSCIL) algorithm that can trade off the accuracy of learning new classes and the computational memory cost.

[0105] Regularizer: It is a small sample continuous learning algorithm based on the subspace regularizer mechanism, which encourages the weight vector of the new category to be close to the subspace spanned by the weight of the existing category to reduce catastrophic forgetting and overfitting.

[0106] ALICE: It is a small sample incremental learning algorithm based on Augmented Angular Loss Incremental Classification (ALICE), which uses angle penalty loss instead of the commonly used cross entropy loss to obtain well-clustered features.

[0107] SAVC: It is a semantic-aware virtual contrastive model (SAVC) that promotes the separation between new classes and base classes by introducing virtual classes into supervised contrastive learning.

[0108] NC-FSCIL: It is a Neural Collapse-inspired Feature-classifier Alignment for Few-Shot Class Incremental Learning (NC-FSCIL), which alleviates the misalignment between old class features and classifiers faced by small-shot continuous learning through pre-assigned classifiers.

[0109] FeSSSS: is a method based on the Few-shot Self-Supervised Syetem (FeSSSS), which uses the advances brought by self-supervised learning to correct overfitting and catastrophic forgetting, and trains lightweight feature fusion and classifiers on a series of features that appear in supervised and self-supervised models.

[0110] Mamba-FSCIL: It is a Few-shot Class-incremental Learning (FSCIL) algorithm based on a selective state-space model (i.e., Mamba model). A projector of a dual selective state-space model is designed to adaptively adjust its projection parameters according to the input context to adapt to new categories while reducing interference to existing categories.

[0111] CPE-CLIP: It is a small-sample incremental learning algorithm that performs continuous parameter-efficient finetuning (CPE) on the CLIP model (based on Contrastive Language-Image Pre-training, CLIP), leveraging the extensive knowledge gained by CLIP in large-scale pre-training and its effectiveness in generalizing new concepts.

[0112] Table 2 Comparison of the present invention with other methods on CUB-200

[0113]

[0114] in:

[0115] Data-free: It is a small sample continuous learning algorithm based on data free replay. It synthesizes the learned data through the generator to keep the learned knowledge from being forgotten without accessing the real data.

[0116] MetaFSCIL: is a Few-shot Class-incremental Learning (FSCIL) algorithm based on meta-learning. It uses meta-objectives to learn and quickly adapt to new tasks without forgetting what has been learned.

[0117] FeSSSS: is a method based on the Few-shot Self-Supervised Syetem (FeSSSS), which uses the advances brought by self-supervised learning to correct overfitting and catastrophic forgetting, and trains lightweight feature fusion and classifiers on a series of features that appear in supervised and self-supervised models.

[0118] DSN: It is a small sample continuous learning algorithm based on the Dynamic Support Network (DSN), which uses the dynamic support network node expansion method to support the adaptive update network of the feature space.

[0119] FACT: It is a small sample continuous learning algorithm based on forward compatible training (FACT). It squeezes the embedding space of known categories by assigning virtual cluster centers to provide feature space for the embedding of new categories.

[0120] ALICE: It is a small sample incremental learning algorithm based on Augmented Angular Loss Incremental Classification (ALICE), which uses angle penalty loss instead of the commonly used cross entropy loss to obtain well-clustered features.

[0121] LIMIT: It is a small-sample continuous learning algorithm that learns multi-phase incremental tasks (LIMIT). It constructs a generalizable feature space for unseen tasks by using base class data to construct pseudo small-sample continuous learning tasks.

[0122] NC-FSCIL: It is a Neural Collapse-inspired Feature-classifier Alignment for Few-Shot Class Incremental Learning (NC-FSCIL), which alleviates the misalignment between old class features and classifiers faced by small-shot continuous learning through pre-assigned classifiers.

[0123] Mamba-FSCIL: It is a Few-shot Class-incremental Learning (FSCIL) algorithm based on a selective state-space model (i.e., Mamba model). A projector of a dual selective state-space model is designed to adaptively adjust its projection parameters according to the input context to adapt to new categories while reducing interference to existing categories.

[0124] CPE-CLIP: It is a small-sample incremental learning algorithm that performs continuous parameter-efficient finetuning (CPE) on the CLIP model (based on Contrastive Language-Image Pre-training, CLIP), leveraging the extensive knowledge gained by CLIP in large-scale pre-training and its effectiveness in generalizing new concepts.

[0125] CEC+: It is a small-sample incremental learning algorithm based on Continuously Evolved Classifiers (CEC). It updates classification weights and test features through a knowledge-guided relation refinement module, and designs a pseudo-incremental relation refinement learning method, using a novel concept mining strategy to mine global and local concepts.

[0126] KANet: is a small-shot incremental learning algorithm based on the Knowledge Adaptation Network that summarizes data-specific knowledge from training data and fuses it into a general representation.

[0127] The continuous knowledge protection decomposition method of the present invention has broad application prospects, especially in products and systems that need to efficiently process incremental learning tasks. For example, the following application areas:

[0128] Intelligent vision system: In visual perception systems such as smart cameras, self-driving cars, and drones, this invention can improve the system's ability to continuously learn under limited annotated data, while avoiding catastrophic forgetting, maintaining the memory of existing knowledge, and optimizing the accuracy of tasks such as target detection and behavior analysis.

[0129] Robotics and Autonomous Systems: In the fields of robotic navigation, autonomous drones, smart manufacturing and automated production, this invention improves the flexibility and stability of the system by helping the system quickly adapt to new tasks in a changing environment while retaining the ability to perform existing tasks, making it suitable for long-term operation and multi-tasking scenarios.

[0130] Augmented Reality and Mobile Applications: The invention can be integrated into augmented reality applications, smartphone camera applications, social media filters, etc. to help improve image processing and computer vision capabilities. Through continuous learning and adaptation, the invention can enhance the user experience and maintain efficient and accurate visual effects in dynamic environments.

[0131] The small sample continuous learning method based on continuous knowledge protection decomposition of the present invention is widely used in the following products:

[0132] Intelligent vision products: This invention can be applied to devices such as smart cameras, self-driving cars and security monitoring systems. It can enhance visual perception capabilities through continuous learning and support the rapid learning of new categories and effective retention of existing knowledge without a large amount of labeled data.

[0133] Robotics and Automation Equipment: This invention is applicable to autonomous systems such as robot navigation, autonomous drones and automated production, helping to improve visual perception, task adaptation and decision-making capabilities, enabling the system to maintain efficient learning and stable performance in a dynamic environment.

[0134] Augmented Reality and Mobile Applications: The present invention can be integrated into augmented reality systems, smartphone applications, social media filters and other fields to help optimize image processing, target recognition and content generation functions, improve user experience, and enhance the system's ability to adapt to environmental changes.

[0135] Embodiment 2

[0136] like Figure 2 As shown, in this embodiment, a small sample continuous learning system based on continuous knowledge protection decomposition is provided, including:

[0137] The weight decomposition module 201 is used to obtain the playback sample data of each category and extract the playback sample features. Based on the continuous learning model backbone network, the covariance matrix of the activation values ​​corresponding to the playback sample features in the current linear layer of the continuous learning model is calculated; the current pre-trained linear weight matrix is ​​multiplied by the covariance matrix of the activation values ​​of the current linear layer, and the multiplication result is divided into two parts, namely, the knowledge sensitive component and the redundant capacity component, by singular value decomposition;

[0138] The weight updating module 202 is used to freeze the pre-trained linear weight matrix corresponding to the knowledge sensitive component during the incremental adaptive training process, and multiply it with the training sample feature of the current incremental adaptive training stage to obtain the knowledge sensitive feature; at the same time, the redundant capacity component is used to construct a learnable adapter, and it is multiplied with the training sample feature of the current incremental adaptive training stage to obtain the redundant capacity feature; and then the knowledge sensitive feature and the redundant capacity feature are superimposed to obtain the output feature obtained by the current linear layer transformation and passed to the next layer of the continuous learning model to obtain the updated pre-trained linear weight matrix;

[0139] The adaptive training module 203 is used to reacquire the small sample playback data, and perform singular value decomposition and incremental adaptive training operations again based on the updated pre-trained linear weight matrix until the continuous learning model meets the set requirements and stops learning, so as to use the trained continuous learning model to perform classification tasks.

[0140] It should be noted here that the various modules in this embodiment correspond one-to-one to the various steps in Example 1, and the specific implementation process is the same, which will not be described in detail here.

[0141] Embodiment 3

[0142] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps in the small sample continuous learning method based on continuous knowledge protection decomposition as described above are implemented.

[0143] Embodiment 4

[0144] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the small sample continuous learning method based on continuous knowledge protection decomposition as described above are implemented.

[0145] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products of the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram and the combination of the processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0146] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A small sample continuous learning method based on continuous knowledge protection decomposition, characterized in that: include: Obtain the playback sample data of each category and extract the playback sample features. Based on the continuous learning model backbone network, calculate the covariance matrix of the activation values ​​corresponding to the playback sample features in the current linear layer of the continuous learning model; multiply the current pre-trained linear weight matrix with the covariance matrix of the activation values ​​of the current linear layer, and then divide the multiplication result into two parts: knowledge sensitive components and redundant capacity components through singular value decomposition; In the incremental adaptation training process, the pre-trained linear weight matrix corresponding to the knowledge sensitive component is frozen, and it is multiplied with the training sample features of the current incremental adaptation training stage to obtain the knowledge sensitive features; at the same time, the redundant capacity component is used to construct a learnable adapter, which is multiplied with the training sample features of the current incremental adaptation training stage to obtain the redundant capacity features; After superimposing the knowledge-sensitive features and the redundant capacity features, the output features obtained by the current linear layer transformation are obtained and passed to the next layer of the continuous learning model to obtain the updated pre-trained linear weight matrix; Re-acquire the small sample playback data, and perform singular value decomposition and incremental adaptation training operations again based on the updated pre-trained linear weight matrix until the continuous learning model meets the set requirements and stops learning, so as to use the trained continuous learning model to perform classification tasks.

2. The small sample continuous learning method based on continuous knowledge protection decomposition as claimed in claim 1, characterized in that: The expression of the covariance matrix of the activation values ​​is: ， in, represents the covariance matrix; Represents the activation value of the playback sample feature at the current layer; express Transpose; is the number of playback samples; Represents the input dimension of the pre-trained linear weight matrix.

3. The small sample continuous learning method based on continuous knowledge protection decomposition as claimed in claim 1, characterized in that: The process of singular value decomposition is: ， in, represents singular value decomposition; and is an orthogonal matrix, is a diagonal matrix containing the singular values ​​arranged from largest to smallest ; represents the covariance matrix; represents the pre-trained linear weight matrix; represents the rank of the pre-trained linear weight matrix; represents the left singular vector; is a matrix No. OK; Represents the transpose of a matrix.

4. The small sample continuous learning method based on continuous knowledge protection decomposition as claimed in claim 1, characterized in that: The first Rr singular vectors with larger singular values ​​are used as knowledge sensitive components; the last r singular vectors with smaller corresponding singular values ​​are used as redundant capacity components; where R represents the rank of the pre-trained linear weight matrix, which is the number of non-zero singular values, , Represents the input dimension of the pre-trained linear weight matrix; represents the output dimension of the pre-trained linear weight matrix; r is a preset hyperparameter used to control the number of redundant components.

5. The small sample continuous learning method based on continuous knowledge protection decomposition as claimed in claim 4, characterized in that: Two learnable low-rank matrices constructed from redundant capacity components and , and as an adapter: in, Represents the pre-trained linear weight matrix corresponding to the knowledge sensitive component; represents the pre-trained linear weight matrix; and is an orthogonal matrix; represents the inverse of the covariance matrix; represents the left singular vector; represents singular values; is a matrix No. OK; Represents the transpose of a matrix; Indicates the last List, and Represent the adapter matrix respectively; during the adaptation process, by only updating and Freeze way to retain what you have learned.

6. The small sample continuous learning method based on continuous knowledge protection decomposition as claimed in claim 1, characterized in that: The expression of the updated pre-trained linear weight matrix is: in, represents the updated pre-trained linear weight matrix; Represents the pre-trained linear weight matrix corresponding to the knowledge sensitive component; and Represents the adapter update parameter matrix corresponding to the redundant capacity components.

7. The small sample continuous learning method based on continuous knowledge protection decomposition as claimed in claim 1, characterized in that: The training sample features of the current incremental adaptive training phase are extracted from the training sample data of the current incremental adaptive training phase; the training sample data of the current incremental adaptive training phase is composed of initial training sample data and playback sample data.

8. A small sample continuous learning system based on continuous knowledge protection decomposition, characterized in that: include: The weight decomposition module is used to obtain the playback sample data of each category and extract the playback sample features. Based on the continuous learning model backbone network, the covariance matrix of the activation values ​​corresponding to the playback sample features in the current linear layer of the continuous learning model is calculated; the current pre-trained linear weight matrix is ​​multiplied by the covariance matrix of the activation values ​​of the current linear layer, and the multiplication result is divided into two parts: knowledge sensitive components and redundant capacity components through singular value decomposition; The weight update module is used to freeze the pre-trained linear weight matrix corresponding to the knowledge sensitive component during the incremental adaptation training process, and multiply it with the training sample features of the current incremental adaptation training stage to obtain the knowledge sensitive features; at the same time, the redundant capacity component is used to construct a learnable adapter, and it is multiplied with the training sample features of the current incremental adaptation training stage to obtain the redundant capacity features; After superimposing the knowledge-sensitive features and the redundant capacity features, the output features obtained by the current linear layer transformation are obtained and passed to the next layer of the continuous learning model to obtain the updated pre-trained linear weight matrix; The adaptive training module is used to reacquire small sample playback data, and perform singular value decomposition and incremental adaptive training operations again based on the updated pre-trained linear weight matrix until the continuous learning model meets the set requirements and stops learning, so as to use the trained continuous learning model to perform classification tasks.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the small sample continuous learning method based on continuous knowledge protection decomposition as described in any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the small sample continuous learning method based on continuous knowledge protection decomposition as described in any one of claims 1 to 7 are implemented.

Images