Small sample continuous learning method and system based on adaptive adaptation layer selection

Abstract

The invention belongs to the field of artificial intelligence and deep learning, and provides a small sample continuous learning method and system based on adaptive adaptation layer selection. The small sample continuous learning method based on adaptive adaptation layer selection comprises the following steps: pre-training a singular value of a linear weight; freezing a pre-training linear weight matrix corresponding to the knowledge sensitive component; adaptively determining an adaptation layer; multiplying the adapter matrix based on the adaptation layer by the training sample features of the current incremental adaptation training stage to obtain redundancy capacity features; and the small sample playback data is obtained again, singular value decomposition, adaptive layer adaptive determination and incremental adaptive training operation are performed in sequence based on the updated pre-training linear weight matrix, and learning is stopped until the continuous learning model reaches a set requirement, so that the trained continuous learning model is utilized to execute a classification task. According to the method, dynamic adapter selection for minimizing old knowledge interference is realized, and the classification accuracy of a small sample continuous learning model is guaranteed.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and deep learning, and in particular to a small sample continuous learning method and system based on adaptive adaptation layer selection. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Small-sample continuous learning can learn new categories with limited data while protecting existing knowledge. In practical applications, such as adaptive recommendation systems and robot navigation in dynamic environments, small-sample continuous learning algorithms need to ensure that the model can effectively adapt to new information under limited resources without affecting the stability of existing knowledge. Existing methods usually use fixed network layers for fine-tuning or inserting adapters at different adaptation stages, failing to consider that the ability of different network layers to retain old knowledge changes dynamically during the continuous learning process. With the continuous introduction of new tasks, the adaptability and sensitivity of different layers to old knowledge will also change, making it difficult for manual or fixed layer selection strategies to achieve the optimal balance between adapting to new tasks and retaining old knowledge, thereby affecting the performance of small-sample continuous learning models and ultimately weakening the classification accuracy of small-sample continuous learning models. Summary of the invention

[0004] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a small sample continuous learning method and system based on adaptive adaptation layer selection, which realizes dynamic adapter selection to minimize the interference of old knowledge and ensures the classification accuracy of the small sample continuous learning model.

[0005] In order to achieve the above object, the present invention adopts the following technical solution: A first aspect of the present invention provides a small sample continuous learning method based on adaptive adaptation layer selection.

[0006] A small sample continuous learning method based on adaptive adaptation layer selection, comprising: Based on the playback sample data and the continuous learning model backbone network, calculate the covariance matrix of the activation values ​​corresponding to the playback sample data in the linear layer of the continuous learning model; multiply the pre-trained linear weight matrix with the covariance matrix of the activation value, and then decompose it into knowledge sensitive components and redundant capacity components through singular value decomposition; Freeze the pre-trained linear weight matrix corresponding to the knowledge sensitive component, multiply it with the training sample features of the current incremental adaptation training stage to obtain the knowledge sensitive features; at the same time, use the redundant capacity component to construct a learnable adapter, and adaptively determine the adaptation layer according to the pre-calculated adapter sensitivity ratio of each linear layer; multiply the adapter matrix based on the adaptation layer with the training sample features of the current incremental adaptation training stage to obtain the redundant capacity features; After superimposing the knowledge-sensitive features and the redundant capacity features, the output features obtained by the linear layer transformation are passed to the next layer of the continuous learning model to obtain the updated pre-trained linear weight matrix; Re-acquire the small sample playback data, and perform singular value decomposition, adaptive determination of the adaptation layer, and incremental adaptation training operations in sequence based on the updated pre-trained linear weight matrix until the continuous learning model meets the set requirements and stops learning, so as to use the trained continuous learning model to perform classification tasks.

[0007] As an implementation method, the adapter sensitivity ratios of all linear layers are arranged in ascending order, and K layers with the smallest adapter sensitivity ratios are selected for adapter construction; wherein K is a positive integer greater than or equal to 1.

[0008] As an implementation method, the adapter sensitivity ratio is:

[0009] in, is the adapter sensitivity ratio size; It is Layer singular value diagonal matrix The last A small non-zero singular value, that is, the singular value corresponding to the first component in the redundant capacity component; correspond The smallest non-zero singular value in .

[0010] As an implementation method, the covariance matrix of the activation values ​​is expressed as: , in, represents the covariance matrix; Represents the activation value of the playback sample feature at the current layer; express Transpose; is the number of playback samples; Represents the input dimension of the pre-trained linear weight matrix.

[0011] As an implementation method, the process of singular value decomposition is: , in, represents singular value decomposition; and is an orthogonal matrix, is a diagonal matrix containing the singular values ​​arranged from largest to smallest ; represents the covariance matrix; represents the pre-trained linear weight matrix; represents the rank of the pre-trained linear weight matrix; represents the left singular vector; is a matrix No. OK; Represents the transpose of a matrix.

[0012] As an implementation method, the expression of the updated pre-trained linear weight matrix is: ; in, represents the updated pre-trained linear weight matrix; Represents the pre-trained linear weight matrix corresponding to the knowledge sensitive component; and Represents the adapter update parameter matrix corresponding to the redundant capacity components.

[0013] As an implementation mode, the training sample features of the current incremental adaptive training stage are extracted from the training sample data of the current incremental adaptive training stage; the training sample data of the current incremental adaptive training stage is composed of initial training sample data and playback sample data.

[0014] A second aspect of the present invention provides a small sample continuous learning system based on adaptive adaptation layer selection.

[0015] A small sample continuous learning system based on adaptive adaptation layer selection, comprising: The weight decomposition module is used to calculate the covariance matrix of the activation values ​​corresponding to the replay sample data in the linear layer of the continuous learning model based on the replay sample data and the continuous learning model backbone network; multiply the pre-trained linear weight matrix with the covariance matrix of the activation value, and then decompose it into knowledge sensitive components and redundant capacity components through singular value decomposition; The adaptation layer determination module is used to freeze the pre-trained linear weight matrix corresponding to the knowledge sensitive component, multiply it with the training sample feature of the current incremental adaptation training stage to obtain the knowledge sensitive feature; at the same time, the redundant capacity component is used to construct a learnable adapter, and the adaptation layer is adaptively determined according to the pre-calculated adapter sensitivity ratio of each linear layer; the adapter matrix based on the adaptation layer is multiplied with the training sample feature of the current incremental adaptation training stage to obtain the redundant capacity feature; The weight update module is used to superimpose the knowledge-sensitive features and the redundant capacity features to obtain the output features obtained by the linear layer transformation and pass them to the next layer of the continuous learning model to obtain the updated pre-trained linear weight matrix; The adaptation training module is used to reacquire small sample playback data, and perform singular value decomposition, adaptive determination of the adaptation layer and incremental adaptation training operations in sequence based on the updated pre-trained linear weight matrix until the continuous learning model meets the set requirements and stops learning, so as to use the trained continuous learning model to perform classification tasks.

[0016] A third aspect of the present invention provides a computer-readable storage medium.

[0017] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the small sample continuous learning method based on adaptive adaptation layer selection as described above.

[0018] A fourth aspect of the present invention provides an electronic device.

[0019] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the small sample continuous learning method based on adaptive adaptation layer selection as described above are implemented.

[0020] Compared with the prior art, the present invention has the following beneficial effects: (1) Dynamic adaptation layer selection mechanism: Different from the traditional method of manually selecting the adaptation layer, the present invention introduces the quantitative indicator of adapter sensitivity ratio (ASR) to dynamically evaluate the ability of each network layer to retain old knowledge and adapt to new tasks. By sorting and selecting the K layers with the lowest ASR values ​​as the adaptation layer, the present invention realizes dynamic adapter selection that minimizes the interference of old knowledge.

[0021] (2) Knowledge-sensitive decomposition and redundant capacity modeling: Singular value decomposition (SVD) is used to decompose network weights into knowledge-sensitive components and redundant capacity components, which are used to retain existing knowledge and adapt to new tasks, respectively. The knowledge-sensitive components are frozen to protect old knowledge, while the redundant capacity components are built as adapters and fine-tuned to learn new knowledge, effectively solving the problem of balancing adaptability and stability when freezing weights or fully fine-tuning in traditional methods.

[0022] (3) Dynamic evaluation and continuous adjustment: The present invention can perform real-time dynamic evaluation of the network layer at each incremental stage, recalculate the ASR and select the adaptation layer based on the latest playback data to adapt to the task requirements at different stages, thus overcoming the problem that the existing methods lack dynamic adjustment capabilities in the fixed layer selection strategy.

[0023] (4) Low computational and storage overhead: Compared with data playback methods that require a large amount of storage space or dynamic expansion methods that introduce additional model parameters, the present invention avoids modifying the model architecture or significantly increasing the inference overhead by utilizing the redundant capacity in the existing network structure, and has higher efficiency and practicality.

[0024] (5) Adaptation layer optimization and knowledge integration: After the adaptation is completed, the fine-tuned redundant capacity components are re-merged with the frozen knowledge-sensitive components to restore the original structure of the weight matrix. The new knowledge is integrated into the knowledge-sensitive components by continuously updating the covariance matrix, ensuring that the model can better adapt to new tasks in the future.

[0025] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0027] Figure 1 This is a flow chart of a small sample continuous learning method based on adaptive adaptation layer selection according to an embodiment of the present invention; Figure 2 It is a structural diagram of a small sample continuous learning system based on adaptive adaptation layer selection according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0029] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0030] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0031] Terminology explanation: Continuous learning with small samples: The goal is to learn a model that can continuously learn knowledge from a series of new categories using only a small number of labeled training samples while retaining the knowledge gained from previous categories. The data of each incremental adaptation stage can be represented as In each training stage t, the model receives the training set , which contains the input sample and its corresponding tags Initial training phase It is also called the basic training phase, and its label set There are sufficient labels and samples to enable the model to obtain good basic representation capabilities and lay the foundation for subsequent continuous learning. In the training process, the model is gradually introduced to new categories, each category has only a small number of labeled samples, and there is no overlap between the category sets of different training stages (such as stage t and stage t'). During the new phase of training, only limited replay samples from the learned categories can be accessed (usually only one replay sample for a category) to maintain the learned knowledge, and the complete data of the previous training phase cannot be directly obtained. In the model evaluation phase after the training phase t, the model needs to be tested on all learned categories to evaluate the model's ability to learn new categories and maintain the learned categories.

[0032] Embodiment 1 according to Figure 1 As shown, this embodiment provides a small sample continuous learning method based on adaptive adaptation layer selection, including: Step 1: Based on the playback sample data and the continuous learning model backbone network, calculate the covariance matrix of the activation values ​​corresponding to the playback sample data in the linear layer of the continuous learning model; multiply the pre-trained linear weight matrix with the covariance matrix of the activation value, and then decompose it into knowledge sensitive components and redundant capacity components through singular value decomposition; Step 2: Freeze the pre-trained linear weight matrix corresponding to the knowledge sensitive component, multiply it with the training sample features of the current incremental adaptation training stage to obtain the knowledge sensitive features; at the same time, use the redundant capacity component to construct a learnable adapter, and adaptively determine the adaptation layer according to the pre-calculated adapter sensitivity ratio of each linear layer; multiply the adapter matrix based on the adaptation layer with the training sample features of the current incremental adaptation training stage to obtain the redundant capacity features; Step 3: After superimposing the knowledge-sensitive features and the redundant capacity features, the output features obtained by the linear layer transformation are obtained and passed to the next layer of the continuous learning model to obtain the updated pre-trained linear weight matrix; Step 4: Re-acquire the small sample playback data, and perform singular value decomposition, adaptive determination of the adaptation layer, and incremental adaptation training operations in sequence based on the updated pre-trained linear weight matrix (i.e., repeat steps 1 to 3) until the continuous learning model meets the set requirements and stops learning, so as to use the trained continuous learning model to perform the classification task.

[0033] To achieve the above purpose, in step 1, in each incremental training phase To get started, collect a small sample dataset of replays. ,in, Represents a union; each old category contains only one sample. Input the replayed sample into the model backbone network The activation values ​​of the intermediate layers of the model are calculated in , and the covariance matrix is ​​calculated before each linear layer : , in, represents the covariance matrix; Represents the activation value of the playback sample feature at the current layer; express Transpose; is the number of playback samples; Represents the input dimension of the pre-trained linear weight matrix.

[0034] Among them, the process of singular value decomposition of the product of the covariance matrix of each activation value and the linear weight matrix is: , in, represents singular value decomposition; and is an orthogonal matrix, is a diagonal matrix containing the singular values ​​arranged from largest to smallest ; represents the covariance matrix; represents the pre-trained linear weight matrix; represents the rank of the pre-trained linear weight matrix; represents the left singular vector; is a matrix No. OK; Represents the transpose of a matrix.

[0035] In the specific implementation process, the first Rr singular vectors with larger singular values ​​are used as knowledge sensitive components; the last r singular vectors with smaller corresponding singular values ​​are used as redundant capacity components; where R represents the rank of the pre-trained linear weight matrix, which is the number of non-zero singular values, , Represents the input dimension of the pre-trained linear weight matrix; Represents the output dimension of the pre-trained linear weight matrix.

[0036] Generally, r is set to 64; the selection of the r value can be determined based on actual applications or experimental results, usually through cross-validation or model performance evaluation.

[0037] Among them, the knowledge-sensitive component and the redundant capacity component are complementary. The redundant capacity component can minimize the interference to existing capabilities to ensure the stability of old categories and the plasticity of new tasks.

[0038] In the specific implementation of step 2, after weight decomposition, the weights of the model are divided into two parts: "knowledge-sensitive components" and "redundant capacity components". Knowledge-sensitive components are used to retain existing knowledge, while redundant capacity components provide flexible space for learning new knowledge. Although all network layers contain redundant capacity components, not every layer has the same ability to protect existing knowledge, nor does every layer have the same redundant capacity to adapt to new tasks. When fine-tuning the adapter, if the adapter is assigned to layers that are more sensitive to existing knowledge, the original knowledge structure may be destroyed, leading to catastrophic forgetting. Therefore, before each incremental adaptation stage, it is necessary to accurately select the layer that is most suitable for adapting to the new task while ensuring minimal interference with existing knowledge.

[0039] In an embodiment of the present invention, an adaptive adaptation layer selection strategy is proposed, that is, the adaptation layer is adaptively determined according to the size of the pre-calculated adapter sensitivity ratio (ASR) of each linear layer.

[0040] The adapter sensitivity ratio quantifies the sensitivity of the redundant capacity components separated in each layer to the existing knowledge. Lower ASR indicates that the redundant capacity components have singular values ​​closer to the minimum, and the adapter built with these components is less likely to interfere with the existing knowledge. On the contrary, higher ASR indicates that the adapter contains some important components that have non-negligible contributions to the existing knowledge. Therefore, layers with lower ASR should be preferentially used for adapting new tasks. Before each new incremental adaptation training starts, adaptive layer selection is performed by the following steps: (1) Obtaining the singular value matrix: In the tth adaptation stage, perform knowledge protection weight decomposition on each linear layer in the backbone network to obtain the singular value diagonal matrix of each layer , for example The singular value diagonal matrix obtained after the knowledge-preserving singular value decomposition of the layer linear weight matrix is ; (2) Adapter sensitivity ratio calculation: Calculate the adapter sensitivity ratio of each linear layer based on the singular value of the redundant capacity component calculated. :

[0041] in It is Layer singular value diagonal matrix The last A small non-zero singular value, that is, the singular value corresponding to the first component in the redundant capacity component; correspond The smallest non-zero singular value in .

[0042] (3) Adapter sensitivity ratio sorting: Arrange the N ASR values ​​in descending order to obtain: ， , ...,

[0043] in is the position of the layer where the minimum ASR value among the N ASR values ​​is located, is the corresponding minimum ASR value; is the position of the layer where the second-to-last smallest ASR value among the N ASR values ​​is located, is the corresponding second smallest ASR value; is the position of the layer where the largest ASR value among the N ASR values ​​is located, is the corresponding maximum ASR value.

[0044] (4) Select the K layers with the smallest sensitivity as the adaptation layers: Select the last K layers with the smallest ASR value to construct the adapter based on knowledge protection decomposition. The positions of the selected layers are as follows:

[0045] K is the number of adaptation layers set manually, which is used to control the number of learnable parameters and is set to 6 by default. is a set of K layers with the least impact on the preservation of existing knowledge selected by the adaptive layer selection mechanism proposed by the present invention before the start of the t-th adaptation phase; is the position number of the layer with the smallest ASR value; The position number of the layer with the second smallest ASR value; The need for the position of the Kth smallest layer of ASR value. The adapters constructed by these K layers are used for parameter update in the tth stage, and the remaining adapters are not adapted.

[0046] The redundant capacity components are used to construct two learnable low-rank matrices and As an adapter:

[0047]

[0048]

[0049]

[0050]

[0051] in, Represents the pre-trained linear weight matrix corresponding to the knowledge sensitive component; represents the pre-trained linear weight matrix; and is an orthogonal matrix; represents the inverse of the covariance matrix; represents the left singular vector; represents singular values; is a matrix No. OK; Represents the transpose of a matrix; Indicates the last List, and Represent the adapter matrix respectively; during the adaptation process, by only updating and Freeze way to retain what you have learned.

[0052] In step 3, during the incremental training of the tth stage, each training stage contains a new set of training samples and corresponding labels. The input data may be images, text, or other forms of samples. These data first pass through the input layer of the backbone network, pass to the linear layer layer by layer, and finally pass through the adapter. For each input sample, the features of the i-th training sample are processed by the linear layer in the network. The current linear layer is calculated as follows:

[0053] in, represents the pre-trained linear weight matrix corresponding to the knowledge sensitive component; B and A are a set of learnable adapter matrices; is the input feature of the current linear layer; It is the output feature obtained by the current linear layer transformation and passed to the next layer of the network.

[0054] In this process, the role of the adapter is to fine-tune the weights of the network, especially when adding new categories to the training process, the adapter will perform adaptive fine-tuning based on the original model. The adapter effectively integrates new knowledge into the original classification capabilities by calculating the combination of input features and the new weight matrix.

[0055] The loss function is usually used to measure the difference between the model output and the true label. In incremental training, the loss function calculation usually depends on the output of the network and the true label. (That is, the actual label of the input sample). Assume that we use the cross entropy loss function as the loss function in common classification tasks, and its calculation formula is:

[0056] in, Represents the loss value of the i-th training sample; Represents the true label of the i-th sample in category c (usually 0 or 1, indicating whether it belongs to the category); Represents the predicted probability of the i-th sample in category c, which is usually obtained through the softmax layer of the model.

[0057] After calculating the loss, the back-propagation algorithm is used to calculate the gradient of each model parameter. The gradient represents the rate of change of the loss function with respect to the adapter parameters. Through back-propagation, the gradient of the loss with respect to the weight matrices A and B is calculated, and the model parameters are updated according to these gradients to minimize the loss.

[0058] The expression of the updated pre-trained linear weight matrix is:

[0059] in, represents the updated pre-trained linear weight matrix; W Represents the pre-trained linear weight matrix corresponding to the knowledge sensitive component; and Represents the adapter update parameter matrix corresponding to the redundant capacity components.

[0060] Specifically, in step 4, the covariance matrix is ​​recalculated before each new adaptation phase begins. Repeat the above decomposition process to integrate the newly acquired capabilities into the knowledge-sensitive components, so that this method can adapt to the evolving model capabilities in continuous learning of small samples. In the adaptation phase, it will be updated to include all previously learned categories:

[0061] Then, the updated playback data is used to re-decompose the knowledge protection weights of the N linear weight layers. After decomposition, the above adaptive layer selection mechanism is executed to obtain a new adaptive layer as follows: 。

[0062] in is a set of K layers with the least impact on the preservation of existing knowledge, selected by the adaptive layer selection mechanism proposed by the present invention before the start of the t+1th adaptation phase; To take advantage of the latest replay dataset The position number of the layer with the smallest ASR value calculated after knowledge protection decomposition; For the latest playback dataset The serial number of the position of the second-to-last layer with the smallest ASR value calculated after knowledge protection decomposition; For the latest playback dataset The position of the Kth smallest layer of the ASR value calculated after knowledge protection decomposition is required. The adapter constructed by these K layers is used to update parameters in the t+1th stage, and the remaining adapters are not updated.

[0063] In summary, the above framework can effectively protect the learned knowledge while improving the adaptation performance to new tasks without changing the model architecture or increasing the inference overhead by applying continuous knowledge protection decomposition in the backbone model's adaptive selection of the layer that has the least impact on the existing knowledge.

[0064] Compared with the previous technologies, the continuous knowledge protection decomposition method proposed in the present invention has the following significant beneficial effects: (1) Dynamic adaptation layer selection mechanism: Different from the traditional method of manually selecting the adaptation layer, the present invention introduces the quantitative indicator of adapter sensitivity ratio (ASR) to dynamically evaluate the ability of each network layer to retain old knowledge and adapt to new tasks. By sorting and selecting the K layers with the lowest ASR values ​​as the adaptation layer, the present invention realizes dynamic adapter selection that minimizes the interference of old knowledge.

[0065] (2) Knowledge-sensitive decomposition and redundant capacity modeling: Singular value decomposition (SVD) is used to decompose network weights into knowledge-sensitive components and redundant capacity components, which are used to retain existing knowledge and adapt to new tasks, respectively. The knowledge-sensitive components are frozen to protect old knowledge, while the redundant capacity components are built as adapters and fine-tuned to learn new knowledge, effectively solving the problem of balancing adaptability and stability when freezing weights or fully fine-tuning in traditional methods.

[0066] (3) Dynamic evaluation and continuous adjustment: The present invention can perform real-time dynamic evaluation of the network layer at each incremental stage, recalculate the ASR and select the adaptation layer based on the latest playback data to adapt to the task requirements at different stages, thus overcoming the problem that the existing methods lack dynamic adjustment capabilities in the fixed layer selection strategy.

[0067] (4) Low computational and storage overhead: Compared with data playback methods that require a large amount of storage space or dynamic expansion methods that introduce additional model parameters, the present invention avoids modifying the model architecture or significantly increasing the inference overhead by utilizing the redundant capacity in the existing network structure, and has higher efficiency and practicality.

[0068] (5) Adaptation layer optimization and knowledge integration: After the adaptation is completed, the fine-tuned redundant capacity components are re-merged with the frozen knowledge-sensitive components to restore the original structure of the weight matrix. The new knowledge is integrated into the knowledge-sensitive components by continuously updating the covariance matrix, ensuring that the model can better adapt to new tasks in the future.

[0069] In summary, the present invention effectively balances the learning of new knowledge and the retention of old knowledge without modifying the model architecture by introducing an adaptive adaptation layer selection mechanism and a weight decomposition strategy, significantly improving the adaptability and generalization ability of small sample continuous learning. This innovative method is significantly superior to existing technologies in terms of model efficiency, knowledge protection, and dynamic adaptability.

[0070] Table 1 Comparison of the present invention with other methods on miniImageNet

[0071] in: DSN: It is a small sample continuous learning algorithm based on the Dynamic Support Network (DSN), which uses the dynamic support network node expansion method to support the adaptive update network of the feature space.

[0072] Data-free: It is a small sample continuous learning algorithm based on data free replay. It synthesizes the learned data through the generator to keep the learned knowledge from being forgotten without accessing the real data.

[0073] MetaFSCIL: is a Few-shot Class-incremental Learning (FSCIL) algorithm based on meta-learning. It uses meta-objectives to learn and quickly adapt to new tasks without forgetting what has been learned.

[0074] FeSSSS: is a method based on the Few-shot Self-Supervised Syetem (FeSSSS), which uses the advances brought by self-supervised learning to correct overfitting and catastrophic forgetting, and trains lightweight feature fusion and classifiers on a series of features that appear in supervised and self-supervised models.

[0075] C-FSCIL: is a Constrained Few-shot Class-Incremental Learning (C-FSCIL) algorithm that can trade off the accuracy of learning new classes and the computational memory cost.

[0076] LIMIT: It is a small-sample continuous learning algorithm that learns multi-phase incremental tasks (LIMIT). It constructs a generalizable feature space for unseen tasks by using base class data to construct pseudo small-sample continuous learning tasks.

[0077] FACT: It is a small sample continuous learning algorithm based on forward compatible training (FACT). It squeezes the embedding space of known categories by assigning virtual cluster centers to provide feature space for the embedding of new categories.

[0078] TEEN: It is a small sample continuous learning algorithm based on the training-free calibration strategy (Training-frEE calibration), which enhances the discriminability of new categories by fusing the new cluster centers (i.e., the average features of the categories) with the weighted basic prototypes.

[0079] ALICE: It is a small sample incremental learning algorithm based on Augmented Angular Loss Incremental Classification (ALICE), which uses angle penalty loss instead of the commonly used cross entropy loss to obtain well-clustered features.

[0080] CABD: is a small-sample continuous learning algorithm based on class-aware bilateral distillation (CABD), which is used to adaptively learn knowledge from two complementary teachers.

[0081] NC-FSCIL: It is a neural collapse-inspired feature-classifier alignment for Few-Shot Class Incremental Learning (NC-FSCIL), which alleviates the misalignment between old class features and classifiers faced by small-shot continuous learning through pre-assigned classifiers.

[0082] Mamba-FSCIL: It is a Few-shot Class-incremental Learning (FSCIL) algorithm based on a selective state-space model (i.e., Mamba model). A projector of a dual selective state-space model is designed to adaptively adjust its projection parameters according to the input context to adapt to new categories while reducing interference to existing categories.

[0083] CEC+: It is a small-sample incremental learning algorithm based on Continuously Evolved Classifiers (CEC). It updates classification weights and test features through a knowledge-guided relationship refinement module, designs a pseudo-incremental relationship refinement learning method, and uses a novel concept mining strategy to mine global and local concepts.

[0084] KANet: is a small-shot incremental learning algorithm based on the Knowledge Adaptation Network that summarizes data-specific knowledge from training data and fuses it into a general representation.

[0085] CPE-CLIP: It is a small-sample incremental learning algorithm that performs continuous parameter-efficient finetuning (CPE) on the CLIP model (based on Contrastive Language-Image Pre-training, CLIP), leveraging the extensive knowledge gained by CLIP in large-scale pre-training and its effectiveness in generalizing new concepts.

[0086] PL-FSCIL: is a prompt learning for few-shot class-incremental learning method that combines visual prompts with pre-trained visual Transformers to effectively address the challenges faced in small-shot continuous learning.

[0087] PriViLege: It is a pre-trained vision and language transformers with prompting functions and knowledge distillation. It effectively solves the challenges of catastrophic forgetting and overfitting in large models through new pre-training knowledge adjustment and two loss functions.

[0088] As can be seen from Table 1, the present invention has the following advantages: (1) Higher average accuracy: The adaptive layer selection method proposed in this paper performs well in multiple incremental learning continuous training stages, especially on the CIFAR-100 dataset, showing higher accuracy than existing methods. On the CIFAR-100 dataset, the average accuracy of the adaptive layer selection method proposed in this paper is 88.62%, far exceeding existing methods such as PL-FSCIL (72.66%) and PriViLege (88.41%).

[0089] (2) Stronger knowledge retention capability: In the incremental learning task of gradually introducing new categories, the present invention can effectively adapt to new categories and maintain memory of previous knowledge. For example, in the last continuous learning stage of CIFAR-100, the accuracy of the continuous decomposition adaptation method proposed by the present invention is 86.22%, which is significantly better than other methods, proving that the method has good adaptability when new categories are introduced and can effectively reduce catastrophic forgetting.

[0090] The adaptive adaptation layer selection method of the present invention shows superior performance in incremental learning and small sample classification tasks, can effectively cope with the gradual introduction of new categories, and surpasses the existing advanced technology on the CIFAR-100 dataset.

[0091] The present invention has broad application prospects, especially in products and systems that need to efficiently process incremental learning tasks. For example, the following application areas: Intelligent vision system: In visual perception systems such as smart cameras, self-driving cars, and drones, this invention can improve the system's ability to continuously learn under limited annotated data, while avoiding catastrophic forgetting, maintaining the memory of existing knowledge, and optimizing the accuracy of tasks such as target detection and behavior analysis.

[0092] Robotics and Autonomous Systems: In the fields of robotic navigation, autonomous drones, smart manufacturing and automated production, this invention improves the flexibility and stability of the system by helping the system quickly adapt to new tasks in a changing environment while retaining the ability to perform existing tasks, making it suitable for long-term operation and multi-tasking scenarios.

[0093] Augmented Reality and Mobile Applications: The invention can be integrated into augmented reality applications, smartphone camera applications, social media filters, etc. to help improve image processing and computer vision capabilities. Through continuous learning and adaptation, the invention can enhance the user experience and maintain efficient and accurate visual effects in dynamic environments.

[0094] The small sample continuous learning method based on adaptive adaptation layer selection of the present invention is widely used in the following products: Intelligent vision products: This invention can be applied to devices such as smart cameras, self-driving cars and security monitoring systems. It can enhance visual perception capabilities through continuous learning and support the rapid learning of new categories and effective retention of existing knowledge without a large amount of labeled data.

[0095] Robotics and Automation Equipment: This invention is applicable to autonomous systems such as robot navigation, autonomous drones and automated production, helping to improve visual perception, task adaptation and decision-making capabilities, enabling the system to maintain efficient learning and stable performance in a dynamic environment.

[0096] Augmented Reality and Mobile Applications: The present invention can be integrated into augmented reality systems, smartphone applications, social media filters and other fields to help optimize image processing, target recognition and content generation functions, improve user experience, and enhance the system's ability to adapt to environmental changes.

[0097] Embodiment 2 like Figure 2 As shown, in this embodiment, a small sample continuous learning system based on adaptive adaptation layer selection is provided, including: The weight decomposition module 201 is used to calculate the covariance matrix of the activation values ​​corresponding to the replay sample data in the linear layer of the continuous learning model based on the replay sample data and the continuous learning model backbone network; multiply the pre-trained linear weight matrix by the covariance matrix of the activation value, and then decompose it into knowledge sensitive components and redundant capacity components through singular value decomposition; Adaptation layer determination module 202, which is used to freeze the pre-trained linear weight matrix corresponding to the knowledge sensitive component, multiply it with the training sample feature of the current incremental adaptation training stage to obtain the knowledge sensitive feature; at the same time, use the redundant capacity component to construct a learnable adapter, and adaptively determine the adaptation layer according to the pre-calculated adapter sensitivity ratio of each linear layer; multiply the adapter matrix based on the adaptation layer with the training sample feature of the current incremental adaptation training stage to obtain the redundant capacity feature; The weight updating module 203 is used to superimpose the knowledge-sensitive features and the redundant capacity features to obtain the output features obtained by the linear layer transformation and pass them to the next layer of the continuous learning model to obtain the updated pre-trained linear weight matrix; The adaptation training module 204 is used to reacquire the small sample playback data, and perform singular value decomposition, adaptive determination of the adaptation layer and incremental adaptation training operations in sequence based on the updated pre-trained linear weight matrix until the continuous learning model meets the set requirements and stops learning, so as to use the trained continuous learning model to perform classification tasks.

[0098] It should be noted here that the various modules in this embodiment correspond one-to-one to the various steps in Example 1, and the specific implementation process is the same, which will not be described in detail here.

[0099] Embodiment 3 This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps in the small sample continuous learning method based on adaptive adaptation layer selection as described above are implemented.

[0100] Embodiment 4 This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the small sample continuous learning method based on adaptive adaptation layer selection as described above are implemented.

[0101] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products of the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram and the combination of the processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0102] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A small sample continuous learning method based on adaptive adaptation layer selection, characterized in that: include: Based on the playback sample data and the continuous learning model backbone network, calculate the covariance matrix of the activation values ​​corresponding to the playback sample data in the linear layer of the continuous learning model; Multiply the pre-trained linear weight matrix by the covariance matrix of the activation value, and then decompose it into knowledge-sensitive components and redundant capacity components through singular value decomposition; Freeze the pre-trained linear weight matrix corresponding to the knowledge sensitive component, multiply it with the training sample features of the current incremental adaptation training stage to obtain the knowledge sensitive features; at the same time, use the redundant capacity component to construct a learnable adapter, and adaptively determine the adaptation layer according to the pre-calculated adapter sensitivity ratio of each linear layer; multiply the adapter matrix based on the adaptation layer with the training sample features of the current incremental adaptation training stage to obtain the redundant capacity features; After superimposing the knowledge-sensitive features and the redundant capacity features, the output features obtained by the linear layer transformation are passed to the next layer of the continuous learning model to obtain the updated pre-trained linear weight matrix; Re-acquire the small sample playback data, and perform singular value decomposition, adaptive determination of the adaptation layer, and incremental adaptation training operations in sequence based on the updated pre-trained linear weight matrix until the continuous learning model meets the set requirements and stops learning, so as to use the trained continuous learning model to perform classification tasks.

2. The small sample continuous learning method based on adaptive adaptation layer selection according to claim 1, characterized in that: Arrange the adapter sensitivity ratios of all linear layers in ascending order, and select the first K layers with the smallest adapter sensitivity ratio values ​​for adapter construction; where K is a positive integer greater than or equal to 1.

3. The small sample continuous learning method based on adaptive adaptation layer selection according to claim 1 or 2, characterized in that: The adapter sensitivity ratio is: in, is the adapter sensitivity ratio size; It is Layer singular value diagonal matrix The last A small non-zero singular value, that is, the singular value corresponding to the first component in the redundant capacity component; correspond The smallest non-zero singular value in .

4. The small sample continuous learning method based on adaptive adaptation layer selection according to claim 1, characterized in that: The expression of the covariance matrix of the activation values ​​is: ， in, represents the covariance matrix; Represents the activation value of the playback sample feature at the current layer; express Transpose; is the number of playback samples; Represents the input dimension of the pre-trained linear weight matrix.

5. The small sample continuous learning method based on adaptive adaptation layer selection according to claim 1, characterized in that: The process of singular value decomposition is: ， in, represents singular value decomposition; and is an orthogonal matrix, is a diagonal matrix containing the singular values ​​arranged from largest to smallest ; represents the covariance matrix; represents the pre-trained linear weight matrix; represents the rank of the pre-trained linear weight matrix; represents the left singular vector; is a matrix No. OK; Represents the transpose of a matrix.

6. The small sample continuous learning method based on adaptive adaptation layer selection according to claim 1, characterized in that: The expression of the updated pre-trained linear weight matrix is: ; in, represents the updated pre-trained linear weight matrix; Represents the pre-trained linear weight matrix corresponding to the knowledge sensitive component; and Represents the adapter update parameter matrix corresponding to the redundant capacity components.

7. The small sample continuous learning method based on adaptive adaptation layer selection according to claim 1, characterized in that: The training sample features of the current incremental adaptive training phase are extracted from the training sample data of the current incremental adaptive training phase; the training sample data of the current incremental adaptive training phase is composed of initial training sample data and playback sample data.

8. A small sample continuous learning system based on adaptive adaptation layer selection, characterized in that: include: The weight decomposition module is used to calculate the covariance matrix of the activation values ​​corresponding to the replay sample data in the linear layer of the continuous learning model based on the replay sample data and the continuous learning model backbone network; multiply the pre-trained linear weight matrix with the covariance matrix of the activation value, and then decompose it into knowledge sensitive components and redundant capacity components through singular value decomposition; The adaptation layer determination module is used to freeze the pre-trained linear weight matrix corresponding to the knowledge sensitive component, multiply it with the training sample feature of the current incremental adaptation training stage to obtain the knowledge sensitive feature; at the same time, the redundant capacity component is used to construct a learnable adapter, and the adaptation layer is adaptively determined according to the pre-calculated adapter sensitivity ratio of each linear layer; the adapter matrix based on the adaptation layer is multiplied with the training sample feature of the current incremental adaptation training stage to obtain the redundant capacity feature; The weight update module is used to superimpose the knowledge-sensitive features and the redundant capacity features to obtain the output features obtained by the linear layer transformation and pass them to the next layer of the continuous learning model to obtain the updated pre-trained linear weight matrix; The adaptation training module is used to reacquire small sample playback data, and perform singular value decomposition, adaptive determination of the adaptation layer and incremental adaptation training operations in sequence based on the updated pre-trained linear weight matrix until the continuous learning model meets the set requirements and stops learning, so as to use the trained continuous learning model to perform classification tasks.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the small sample continuous learning method based on adaptive adaptation layer selection as described in any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the small sample continuous learning method based on adaptive adaptation layer selection are implemented as described in any one of claims 1-7.

Images