Review-free continuous learning method based on prompt grouping

By calculating the prompt gradient and dividing the sensitivity group, combined with the agent correction module, the problem of not being able to utilize similar task knowledge in incremental learning is solved, and incremental learning is realized without review, reducing interference between tasks, promoting knowledge transfer, and Avoid data storage requirements.

CN118966307APending Publication Date: 2024-11-15EAST CHINA NORMAL UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411053932.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-02
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

Existing cues-based incremental learning methods cannot effectively utilize complementary knowledge from similar tasks, resulting in catastrophic forgetting problems, and task-specific or shared update paradigms introduce knowledge of unrelated information that interferes with previous tasks.

Method used

Calculate the prompt gradient of all tasks through backpropagation, divide it into high-sensitivity groups and low-sensitivity groups, select the most similar groups and determine whether to join, introduce a proxy correction module to improve the distinction between feature extractor and classification head, and achieve no Review-based incremental learning.

Benefits of technology

Effectively utilize complementary knowledge of similar tasks, reduce interference from dissimilar tasks, achieve more effective review-free continuous learning, promote active knowledge transfer between tasks, and do not need to save data from previous tasks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118966307A_ABST
    Figure CN118966307A_ABST
Patent Text Reader

Abstract

The invention discloses a no-review incremental learning method based on prompt grouping, and the method is characterized in that the method comprises the steps: sampling the data of a batch of a current task, combining with the prompts of all previous tasks, inputting a frozen visual pre-training model, and obtaining the gradients of all prompts through back propagation; calculating the sensitivity of all prompts; dividing into a high-sensitivity group and a low-sensitivity group, calculating the intersection of the prompt of each group and the two groups, and selecting the prompt group with the maximum intersection as the group most similar to the current task; judging whether the prompts contained in the most similar prompt group in the high-sensitivity group exceed a threshold value or not, if so, classifying the prompts of the current task into the group, and otherwise, classifying the current task into a single group; a classification head correction module is used to improve the distinction degree between tasks. Compared with the prior art, the method has the advantages that the problem that complementary knowledge of similar tasks cannot be utilized is solved, positive knowledge migration between continuous tasks is further explored, and the method has a good application prospect.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of incremental learning, and in particular to a non-review incremental learning method based on prompt grouping. Background Art

[0002] In a continuous learning environment, the model needs to continuously learn new tasks while retaining previously learned knowledge. However, when the model learns new data, it often overwrites or destroys previously learned knowledge, a phenomenon called catastrophic forgetting.

[0003] Among the existing continual learning methods, prompt-based learning shows the potential to reduce forgetting. These methods reduce forgetting by learning a small set of pluggable model embeddings (i.e., prompts) instead of directly modifying the parameters of the visual encoder. However, these methods have the following limitations: Using task-specific prompts cannot fully utilize the complementary knowledge in other prompts belonging to similar tasks. This means that the model cannot benefit from the knowledge learned from previous similar tasks. Suppose there are multiple tasks in the past, including a puppy on the grass, a rabbit on the grass, and a car on the road in the current task. In the task of identifying rabbits, the knowledge about green grass and pointy ears in the puppy task learned in the previous task cannot be utilized. Using shared prompts inevitably introduces irrelevant information. When learning a new task, prompts that are irrelevant to the new task (such as black wheels) may be introduced, which will interfere with the knowledge of the previous task.

[0004] The task-specific or task-shared updating paradigms of the prior art limit the possibility of exploring complementary information from cues from other similar tasks. Summary of the invention

[0005] The purpose of the present invention is to provide a review-free incremental learning method based on prompt grouping to address the deficiencies of the prior art. The gradients on the prompts of all previous tasks are calculated through back propagation to obtain the prompt sensitivity, and then all the prompts are divided into a high-sensitivity group and a low-sensitivity group according to the sensitivity. The most similar group is calculated, and whether to add the current task to the group is determined based on the size of its intersection with the high-sensitivity group. Finally, an agent correction module is introduced to improve the discrimination between the feature extractor and the classification head. The method is simple and effectively solves the problem of the prior art prompt-based continuous learning method that cannot utilize the complementary knowledge of similar tasks, provides a more effective and feasible review-free continuous learning method, and further explores the active knowledge transfer between continuous tasks, which has good application prospects.

[0006] The specific technical scheme for achieving the purpose of the present invention is: a review-free incremental learning method based on prompt grouping, which is characterized in that for the data set of the current task, a batch of data is first sampled and combined with the prompts learned in all previous tasks, and input into the frozen visual pre-training model, and the gradients of all prompts are obtained by back propagation; the sensitivity of all prompts is calculated to indicate the degree of influence of the prompt on the current task; all prompts are divided into two groups according to the prompt sensitivity, namely a high sensitivity group and a low sensitivity group, and the intersection of the prompts of each previous group and the two groups is calculated, and the prompt group with the largest intersection is selected as the group most similar to the current task; it is determined whether the prompts contained in the most similar prompt group account for more than a threshold in the prompts in the high sensitivity group, and if it exceeds a threshold, the prompts of the current task are divided into this group, otherwise the current task is a separate group, and finally the classification head correction module is used to improve the discrimination between tasks. The method specifically includes the following steps:

[0007] Step 1: Task prompt gradient calculation

[0008] 1.1) When learning the tth task, randomly select a data batch And concatenate the prompts of all previous tasks P1, ..., P t-1 , and get P old =Concat(P1, ..., P t-1 ), and input it into a pre-trained cross-modal large model.

[0009] 1.2) Through back propagation Get the gradient of each prompt, where ⊙ represents the Hadamard product, e i is the indicator vector of the i-th prompt (i.e., 1 at position i and 0 at other positions), and 1 represents a vector of all 1s.

[0010] Step 2: Prompt Sensitivity Vector Calculation

[0011] 2.1) According to the gradient of each cue obtained in step 1, define the cue sensitivity s i The normalized value of the gradient is represented by the following formula (a):

[0012]

[0013] in, is the data sampled in the tth task, s i To indicate sensitivity, P old is the hint of all previous tasks spliced ​​together, L p The number of hints for each task.

[0014] 2.2) After calculating the prompt sensitivity, the sensitivity vector s expressed by the following formula (b) is obtained:

[0015]

[0016] Among them, each element represents the importance of each hint in the current task.

[0017] Step 3: Calculate the most similar task group

[0018] 3.1) First, the K-means algorithm is used to divide the instructions into two groups according to their sensitivity scores, namely group P with higher sensitivity high and group P with lower sensitivity low .

[0019] 3.2) Then calculate the existing n task groups and high sensitivity P high The intersection between them gives how many instructions in each group are assigned to P high , and the group with the most instructions is marked as Finally, if P high and The number of instructions in If the ratio of the total number of instructions in exceeds the threshold k, the current task t is added to Otherwise, assign the current task t to a new group

[0020] Step 4: Classification head correction

[0021] 4.1) First, the proxy loss is used to improve the discrimination of the extracted features and reduce the mutual interference between tasks. Specifically, by assigning a proxy to each category in the current task, these proxies will be frozen after the task training is completed and will not participate in the training of subsequent tasks. Then, the anchor sample is used to form a positive sample pair with other samples in the same task, and a negative sample pair is formed with proxies of other categories in the same group or proxies in the previous task. Finally, the loss function is calculated by the following formula (c):

[0022]

[0023] Where X is the sample batch, p + is a set of positive surrogate pairs, p - is the negative proxy set, f θ is the feature extractor.

[0024] 4.2) In addition, the classification head correction is further introduced. After completing the current task training, the mean and variance of each category are saved, and then the feature extractor is frozen. The Gaussian distribution is constructed by all the saved means and variances for sampling, and the classification head is fine-tuned to achieve incremental learning without review. Compared with the prior art, the present invention has the following beneficial technical effects and significant technical progress:

[0025] 1) The present invention introduces prompt sensitivity to complete task grouping, thereby achieving active knowledge transfer between similar tasks and reducing mutual interference between dissimilar tasks.

[0026] 2) The present invention introduces a proxy correction module, which combines the proxy loss and the classification head correction to respectively realize the correction of the feature extractor and the correction of the classification head.

[0027] 3) The present invention does not need to save the data of previous tasks, and realizes a non-review incremental learning method. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flow chart of the present invention;

[0029] Figure 2 Flowchart for prediction query. DETAILED DESCRIPTION

[0030] In order to facilitate the understanding of the present invention, the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0031] Example 1

[0032] See also Figure 1 , follow the steps below to perform prompt-based grouping without review incremental learning:

[0033] S100: For the data of the current task, first sample a batch of data and feed it into the model in combination with the prompts of all previous tasks.

[0034] S110: Back propagation calculation obtains the gradients of all prompts.

[0035] S120: Calculate the prompt sensitivity according to the prompt gradient obtained in S110.

[0036] S130: Using the prompt sensitivity calculated in S120, use the Kmeans algorithm to divide all prompts into a high sensitivity group and a low sensitivity group.

[0037] S140: Calculate the existing n task groups and high sensitivity P high The intersection between them gives how many instructions in each group are assigned to P high , and the group with the most instructions is marked as Finally, if Phigh and The number of instructions in If the ratio of the total number of instructions in exceeds the threshold k, we add the current task t to Otherwise, we assign the current task t to a new group

[0038] S150: After completing the current task training, save the mean and variance of each category, then freeze the feature extractor, construct a Gaussian distribution for sampling through all the saved means and variances, and fine-tune the classification head.

[0039] See also Figure 2 , the prediction query process of the present invention is as follows:

[0040] S200: After the training is completed, a sample is input and a prompt group corresponding to the current sample is queried.

[0041] S210: Complete the reasoning of the current sample in combination with the queried hints.

[0042] The above embodiments are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any changes made by adopting the design principles of the present invention and performing non-creative work on this basis should fall within the protection scope of the present invention.

Claims

1. A review-free incremental learning method based on prompt grouping, characterized in that: The method specifically includes: Step 1: Task prompt gradient calculation 1.1) When learning the tth task, randomly extract a data batch The prompts of all previous tasks are concatenated into P,…,P, obtaining P = Concat(P,…,P), which is input into the pre-trained cross-modal large model, where P is the gradient of each prompt; P is the task prompt gradient; Concat is a concatenation operation, which concatenates all prompts in the 0th dimension. 1.2) Through back propagation Get the gradient of each cue, where ⊙ is the Hadamard product; e is the indicator vector of the i-th cue, that is, 1 at position i and 0 elsewhere; 1 represents a vector of all 1s; Step 2: Prompt Sensitivity Vector Calculation 2.1) Based on the gradient of each cue obtained in step 1, define the cue sensitivity s as the normalized value of the gradient expressed by the following formula (a): in, is the data sampled in the t-th task; s is the cue sensitivity; P is the concatenated cues of all previous tasks; L9 is the number of cues for each task; 2.2) Based on the prompt sensitivity s, the sensitivity vector s expressed by the following formula (b) is obtained: Among them, each element represents the importance of each cue in the current task; Step 3: Calculate the most similar task group 3.1) Use K-means algorithm to divide the instructions into two groups according to the sensitivity scores, namely group P with higher sensitivity <=< and group P with lower sensitivity > ; 3.2) Calculate the existing n task groups and high sensitivity P <=< The intersection between them is used to get how many instructions in each group are assigned to P <=< , and label the group with the most instructions as If P <=< and The number of instructions in If the ratio of the total number of instructions in exceeds the threshold k, the current task t is added to Otherwise, assign the current task t to a new group Step 4: Classification head correction 4.1) Assign a proxy to each category in the current task, and then use the anchor sample to form a positive sample pair with other samples in the same task, and form a negative sample pair with proxies of other categories in the same group or proxies in the previous task, and calculate the loss function using the following formula (c) in, X is the sample batch; p A is the set of positive agent pairs; p is the set of negative agent pairs; f X is the feature extractor; 4.2) After completing the training of the current task, save the mean and variance of each category, then freeze the feature extractor, construct a Gaussian distribution through all the saved means and variances for sampling, and directly fine-tune the classification head to achieve class incremental learning without review.

Citation Information

Patent Citations

  • Sample incremental learning method based on deep neural network

    CN114529752A

  • Life-life learning method based on sample replay and knowledge distillation

    CN116776950A