Method and apparatus for self-attention hint-based replica-free continuous learning with dynamic prototype participation

Abstract

A method and apparatus for hint-based replication-free continuous learning with self-attention of dynamic prototype participation. A machine learning model with a base training set accepts continuous new data inputs and uses them to compute prototype self-attention blocks that update the model for tasks associated with the new data without overly or undesirably overwriting the existing model. Gini penalty is applied to the results of such calculations to prevent overwriting the existing model excessively or undesirably.

Description

Technical Field

[0001] Various aspects of this invention relate to a cue-based, non-repetitive continuous learning method for self-attention utilizing dynamic prototypes. This invention is particularly useful for rapidly and efficiently updating machine learning models as new data continuously becomes available. Background Technology

[0002] Real-world data typically arrives sequentially, requiring associated machine learning models (hereinafter referred to as "MLMs" or "models") to be incrementally trained on new incoming data, rather than simultaneously trained on the entire dataset. Continuous learning aims to solve a sequence of multiple tasks using a single model while avoiding performance degradation on previously learned tasks. Because models can only access the current task data when operating on real-world tasks, most state-of-the-art methods rely on rehearsal buffers to retrain on a subset of past data examples. Recently, cue-based methods have enabled models to learn a pool of cueing as alternative data rehearsals. This cue pool serves as a set of instructions for conditioning the model's frozen backbone based on previous / original data examples to adapt to sequential new tasks.

[0003] However, this raises new problems. For example, paraphrasing-based modeling methods suffer significant performance degradation when the buffer capacity is small. Furthermore, data privacy concerns may prevent the paraphrasing buffer from being retrained / accessed. Additionally, existing cue-based modeling methods lack utilization of the backbone features of previous categories (prototypes), resulting in insufficient resistance to overwriting old knowledge or "forgetting" knowledge associated with previously learned tasks.

[0004] Methods that fully utilize prototypes and develop prototype-involved prompting schemes to better integrate old knowledge and acquire new knowledge will be useful. Apparatus for implementing such methods will also be useful inventions. A prompt-based, non-repetitive continuous learning method utilizing self-attention with dynamic prototype involvement, enabling faster and more efficient updates to machine learning models, will be a useful invention, as will the apparatus for implementing such methods. Various aspects of the present invention address these concerns. Summary of the Invention

[0005] Aspects of the present invention include a method for prompt-based, non-repetitive continuous learning with self-attention utilizing dynamic prototypes and an apparatus for implementing such a method.

[0006] A aspect of this invention is a method for prompting in repetitive learning. It computes recently classified prototypes after the training phase of each task and performs self-attention on prototype-involved self-attention blocks, or "PSAB" modules. Typically, most existing methods require a repetition buffer to store past data for experience replay; however, this may be unavailable in real-world scenarios due to privacy and / or memory / storage constraints. Furthermore, as noted above, existing methods lack utilization of the backbone features of previous categories (prototypes), which results in insufficient resistance to overwriting or "forgetting" prompts associated with previously learned tasks.

[0007] Various aspects of the present invention provide a method for repetitive continuous learning, the method comprising calculating recently categorized prototypes after a training phase for each task and performing self-attention on PSAB modules.

[0008] Further aspects and / or advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practicing the invention. Attached Figure Description

[0009] These and / or other aspects and advantages of the present invention will become clear and more readily apparent from the following description of the embodiments in conjunction with the accompanying drawings.

[0010] Figure 1 This is a conceptual flowchart of the method of the present invention.

[0011] Figure 2 This is a flowchart of the steps related to prototype calculation in the method described.

[0012] Figure 3 This is a flowchart of the steps related to the self-attention block in which the computational prototype participates in the method.

[0013] Figure 4 This is a flowchart of the steps in the method related to calculating the regularization loss and the Gini loss function.

[0014] Figure 5 It is a graph showing a comparison between example similarities between queries and keys using existing methods and example similarities between queries and keys using the method of the present invention.

[0015] Figure 6 This is a conceptual schematic diagram of an apparatus for implementing the method of the present invention. Detailed Implementation

[0016] The present embodiments of the invention will now be discussed in detail, examples of which are illustrated in the accompanying drawings, wherein similar reference numerals always refer to similar elements. The embodiments described below are provided to illustrate the invention with reference to the accompanying drawings.

[0017] For the purposes of this application, the "basic" machine learning module (MLM) will be assumed to have been generated by any known or new processing for training the MLM. The object of this invention is to enable the use of the basic MLM in dynamic environments where new tasks with new inputs are continuously processed before the execution of a new task by continuously updating the MLM with new parameters associated with the new task. The state of the MLM after such processing of new inputs is referred to herein as the "new" MLM. After processing with the method of this invention, the new MLM includes the previous MLM, whether it is the original / basic MLM or an intermediate "new" MLM. Whenever the "basic" MLM is mentioned, it should be understood as either the original / first MLM or the previous "new" MLM: there is no limit to the number of times the method iterates on the "basic" MLM to generate the "new" MLM.

[0018] Reference Figure 1 The basic method according to one aspect of the invention can be understood. The new task data set 10 consists of a query set 11, which queries key-value hint pairs 12, each key-value hint pair 12 consisting of a key 12a and a value 12b. The key-value hint pairs are stored in a learnable hint pool 14. The learnable hint pool 14 includes the query set 11 operating on a pre-trained embedding layer 16, which is "frozen" and includes the result data of the original / basic machine MLM. For clarity, the query set 11 is fed into both the new task data set 10 and the pre-trained embedding layer 16. The learnable hint pool 14 is shared and used across all tasks: "old" hints from previous tasks are frozen, while "new" tasks are optimized and updated. The data used to train the basic MLM does not need to be available for performing the method, saving valuable storage space and reducing transmission, storage, and processing time.

[0019] The selected (weighted) cues 17b and 17a are processed by a pre-trained Visual Translator (ViT) encoder, which feeds the base encoder output and the base prototype 110 into a prototype-involved Self-Attention Block (PSAB) module 19. The PSAB module 19 produces a contextualized output for use by the classifier 111. The selected (weighted) cues are also fed from the learnable cue pool 14 into the loss calculation module 12 to compute the Gini loss (penalized – see below). The Gini penalty, regularization penalty, and cross-entropy loss (penalized) are applied to optimize all learnable parameters during new MLM creation. The PSAB module also feeds parameters into the loss calculation 112 to compute the regularization loss (penalized – see below). The regularization loss is applied to the PASB parameters to prevent or minimize changes to parameters important for output prediction.

[0020] Figure 2 Show Figure 1 The method involves steps related to prototype computation. Processing begins in step 20 with in-class samples at task level t, corresponding to the basic MLM. These samples are then fed into the pre-trained ViT encoder in step 21 (see [link to ViT encoder]). Figure 1 In ), and features are extracted (see Figure 4 After features are extracted, the mean is calculated (see...). Figure 4 At the end of training for each task t (t = 1, 2, ..., T), the mean of the feature representations for each class generated by ViT is calculated as the prototype:

[0021]

[0022] Where {ind c} is the set of indices of samples belonging to category c in the batch; f n c is the nth feature embedding of the corresponding category; D is the feature dimension of the encoder output.

[0023] In step 23, dynamic prototype aggregation is performed. The prototype calculated at the end of task t is represented as p. t :

[0024]

[0025] All prototypes up to task t are represented as P. t :

[0026]

[0027] Where C t This is the total number of categories seen up to task t. This is the prototype group that is updated in step 24.

[0028] After each task, all prototypes are "frozen," creating a new base / existing prototype group for use in step 23. For the next new task, the method aggregates the newly computed prototypes for the new task with the existing prototype groups, resulting in:

[0029] P t+1 =[P t ;p t+1 ]

[0030] Therefore, P t It is dynamically expanded after each learning iteration.

[0031] Figure 3 Show Figure 1 The method involves steps related to the creation of PSAB block modules. In step 31, the batch output of the pre-trained ViT encoder is identified: Where bs is the batch size. In step 32, the basic prototype group is identified (see...). Figure 1 and Figure 2 In step 33, the batch output and the basic prototype group are concatenated along the feature dimension (see...). Figure 2 ).

[0032] In task t, it means So that O i =PSAB(Z) i ).

[0033] To form the PSAB module, the three multilayer perceptual (“MLP”) algorithms used for query, key, and value projection are denoted as W. q W k and W v ,in We then represent the query, key, and value projection as Q, respectively. i K i and V i So that:

[0034] Q i =W q Z i

[0035] K i =W k Z i

[0036] V i =W v Z i

[0037] Using the known Softmax function, the method will then be as follows, determined by Q. iK i and V i The defined vector becomes a probability distribution:

[0038]

[0039] The probability distribution is then multiplied by the projected input of the values ​​in the new task in step 36. In step 37, the result of step 36 is added to the original connection performed in step 33, which leads to the final output in step 38.

[0040]

[0041] The PSAB module effectively integrates prototype information with the original encoder output to produce contextualized output for classification.

[0042] Figure 4 Show Figure 1 The method involves steps related to the calculation of regularization loss and Gini loss. In step 41, the input for the new task is fed into the pre-trained ViT encoder for processing in step 43 (see [link]). Figure 1 The data is then passed to a new MLM for processing in step 42.

[0043] As part of step 42, the eigenvectors are calculated using the eigenvector formula based on the following equation:

[0044] q(x) = f(x)[0, :]

[0045] in It is the output of the pre-trained ViT encoder.

[0046] In step 44, the method computes the gradient of the new MLM output with respect to the PSAB parameters. (Unlike the frozen pre-trained embedding layer and ViT encoder used in step 43, this method includes {W}. q W k W v The PSAB module is updated sequentially task by task. In step 45, the importance of each parameter is assigned in such a way that, in step 46, the difference between each of the basic / original PSAB parameters identified in step 47 and the corresponding new PSAB parameter can be multiplied by the importance assigned to the parameter to generate the final regularization loss in step 48.

[0047] Calculate the regularization weights using the following formula:

[0048]

[0049] Where α is the regularization hyperparameter. It is the old parameter that was recorded, Ωij This highlights the importance of the previously calculated regularization loss. The regularization loss for the PSAB parameters is then used to prevent or reduce errors for parameters such as... Figure 1 The output described in the text predicts changes in important parameters.

[0050] In step 49, the processing for calculating the Gini loss, which aims to encourage diversity in cue selection and reduce cue redundancy, begins. The product of the normalized encoder output and the cue key is calculated to generate a vector. This vector is then processed using the known Softmax algorithm to generate a probability distribution.

[0051] In step 410, the Gini penalty is calculated using the Gini penalty formula based on the following equation:

[0052]

[0053] Specifically, given a batch of B queries, s i，k is the probability between the query and the K keys (from the probability distribution calculated in step 49), and β is the Gini loss weight. By imposing a Gini impurity-based penalty on query-key similarity, the optimization aims to find solutions where query features are more evenly distributed across the suggestion keys. This generates the Gini loss calculated in step 411.

[0054] The final loss to be applied (see Figure 1 Cross-entropy loss (a type of computational loss known in the field of this invention, see [link]). Figure 1 It consists of PSAB regularization loss and Gini loss.

[0055] Figure 5 The diagram illustrates the effect of applying Gini loss penalty to query-key similarity compared to existing methods. Top graph 51 shows that the normalized similarity s between query m and associated key k is highly variable after applying existing continuous learning methods. Bottom graph 52 shows that the query-key similarity is highly consistent after applying Gini loss penalty. This demonstrates that the method of this invention is better suited than existing methods to maintain basic task training while updating the machine learning model with new task inputs.

[0056] Figure 6 Showing the implementation Figure 1 A block diagram of the apparatus for the method. User control 62 and task input source 61 are connected to input system 67, which may include... (Bluetooth) connectivity, USB connectivity, proprietary hardwired or wireless systems, or any other means of connecting them as needed. Task input source 61 may include one or more input sources, such as a camera, mechanical counter, optical or mechanical measuring system, etc. For example, a camera may create visual input data showing assembly line inputs that push various parts along the assembly line for use in the assembly system. Input system 67 (which may also consist of two or more separate systems, one for each input source) sends user control input and task input source data to processor 68. Fixed storage 610 (which may be a hard disk drive, solid-state drive, flash RAM, or any other desired means of persistently storing information) and / or random access memory (RAM) 611 contain a software program or "instructions" having a plurality of executable code elements that implement the methods of the present invention and are executed by processor 68. Note that processor 68 may include a CPU, GPU, proprietary processor, or any reasonable combination thereof. Input data and processing data generated during the various steps of applying the methods of the present invention are also stored in RAM 611 and / or fixed storage 610. After the input has been processed by processor 68 and the final task output has been stored in fixed storage 610 and / or RAM 611 (or offloaded to cloud storage, portable storage, or otherwise stored in a final form for reference), the final task output can be displayed on display 65, printed on printer 66, and / or sent to task output control system 64, which can control one or more mechanisms to implement the task output. For example, the device may determine that certain components are currently used more frequently than others and control the assembly line so that the more frequently used components are input into the assembly line at a higher rate. If these parameters change, for example because a new operator prefers to assemble components in a different order or at a different rate, the continuous learning input will correspondingly change the control inputs controlled by factors such as the relative importance of each component.

[0057] Although several embodiments of the invention have been shown and described, those skilled in the art will recognize that changes may be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the claims and their equivalents.

Claims

1. A method for prompt-based, repetition-free continuous learning with self-attention utilizing dynamic prototypes, comprising the following steps: Use basic training inputs to create basic machine learning models; Define new tasks that apply the basic machine learning model, each new task including new training inputs determined during a new training phase; Calculate at least one new class-based prototype after a new training phase for the new task; Self-attention is performed on the self-attention module involving the prototype using at least one new category-based prototype; Determine one or more key parameters of the self-attention module in which the prototype participates; The penalty is applied to changes in the key parameters of the self-attention module involved in the prototype; The self-attention module involving the prototype, including the penalty, is applied to the basic machine learning model to create a new machine learning model; and The new machine learning model is used to perform the new task.

2. The method of claim 1, wherein calculating the new category-based prototype after the training phase of the new task further comprises: Extract one or more features from one or more samples belonging to the category of the new task from the visual transformer encoder; Sum all extracted features of the aforementioned category; Divide by the count of the samples to calculate the mean feature vector.

3. The method of claim 2, wherein extracting one or more extracted features from the visual transformer encoder further comprises: The basic machine learning model is used as a frozen feature extractor to obtain one or more query features; Cosine similarity is applied to score one or more matches between the query feature and one or more suggestion keys associated with the query feature, and multiple suggestions associated with the suggestion keys are selected; as well as The adjusted embedded features are fed into the basic machine learning model as part of the creation of the new machine learning model.

4. The method of claim 1, wherein performing self-attention on the self-attention module involving the prototype further comprises: Connect all existing prototypes and ViT encoder features to generate contextualized input; The query, key-value projection, and key-value projection are accomplished through three linear layers respectively. Calculate the product of the projected query and the key input, and then calculate the Softmax score; Calculate the product of the Softmax score and the projected input value; Add the result to the original contextualized input; as well as Extract the corresponding batch size position along the first dimension used for classifier prediction.

5. The method of claim 1, wherein applying the penalty to changes in key parameters of the self-attention module involved in the prototype further comprises: Assign a relative importance factor to each of the important parameters; One or more loss values ​​are summed based on the assigned relative importance factors, summing one or more basic parameter values ​​of the base machine learning model and the associated current parameter values ​​of the new machine learning model.

6. The method of claim 5, wherein assigning relative importance factors further comprises: Calculate the gradient of the squared l2 norm of the output of one or more basic machine learning models with respect to each of the importance parameters; The gradients are summed at one or more absolute values ​​on one or more basic machine learning model inputs, and the mean value is taken as the basic relative importance factor. Sum the absolute values ​​of the gradients over one or more new machine learning model inputs, and take the second mean as the new relative importance factor; as well as The final relative importance factor is calculated as a weighted combination of the basic relative importance factor and the new relative importance factor.

7. The method of claim 5, wherein summing one or more loss values ​​between one or more basic parameter values ​​of the base machine learning model and the associated current parameter values ​​of the new machine learning model according to an assigned relative importance factor further comprises: Multiply the change between the updated parameter and the recorded old parameter by the stated importance; as well as The results from all parameters are summed together and multiplied by the regularization weight.

8. The method of claim 1, wherein encouraging cue selection diversity to improve cue learning further comprises: For each new task, initialize the hint set with the associated key; The projected input of the set of hints with associated keys is passed forward to the basic machine learning model to obtain query features; The query features and the associated keys are standardized respectively, and the product is calculated. Apply the Softmax function to the product result to obtain a similar set of probability outputs; Calculate the Gini penalty; as well as The Gini penalty is applied to the set of similar probability outputs.

9. The method of claim 8, wherein forwarding the projection input to the basic machine learning model to obtain query features further comprises extracting a feature vector using a feature vector formula and applying the feature vector to the projection input.

10. The method of claim 8, wherein calculating the Gini penalty for the similar probability output further comprises: Calculate the Gini penalty using the Gini penalty equation; The Gini penalty is applied to the similar probability output.

11. An apparatus comprising a processor coupled to a memory, a fixed storage system, a task input source, and a task output control, wherein the fixed storage is configured to store instructions, and the processor is configured to execute instructions stored in the memory for the following operations: Access the basic machine learning model created using the basic training input; Define new tasks that apply the basic machine learning model, each new task including new training inputs determined during the new training phase and obtained from the task input source; Calculate at least one new class-based prototype after a new training phase for the new task; Self-attention is performed on the self-attention module involving the prototype using at least one new category-based prototype; Determine one or more key parameters of the self-attention module in which the prototype participates; The penalty is applied to changes in the key parameters of the self-attention module involved in the prototype; The self-attention module involving the prototype, including the penalty, is applied to the basic machine learning model to create a new machine learning model; and The new task is performed using the new machine learning model via the task output control.

Images