Continual learning method based on hierarchical adaptive optimization for large model

Abstract

The present application specifically relates to the technical fields of computer vision and pattern recognition. Disclosed is a continual learning method based on hierarchical adaptive optimization for a large model. In the method, a current task representation is obtained on the basis of an average representation of each portion of samples in all training data involved in a task, and a similarity score between the current task representation and each of all stored known task representations is calculated. If the similarity score is greater than a preset threshold, model structure expansion is performed, and the current task representation in a task selector is randomly initialized; otherwise, the model structure and task representation of a known task with the highest similarity are reused. During training, current task data is used to train a current task-specific structure and the current task representation. Finally, by means of hierarchical training of each task, a hierarchical model consisting of a backbone network, task-specific structures and the task selector is obtained. The model can adaptively query a task-specific structure on the basis of an input to complete model inference.

Description

Large model continuous learning method based on layered adaptive optimization

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on March 21, 2024, with application number CN202410328568.X and application name “Large Model Continuous Learning Method Based on Hierarchical Adaptive Optimization”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the technical field of computer vision and pattern recognition, and in particular to a large-model continuous learning method based on hierarchical adaptive optimization. Background Art

[0003] With the rapid development of deep learning technology, the application scope of large-scale distributed models has gradually expanded, and they have demonstrated strong capabilities in handling complex tasks. However, as model size grows and tasks become more diverse, how to control the number of model parameters, improve training efficiency, and reduce computing power requirements while maintaining model performance has become a major challenge.

[0004] Existing large-model learning methods often require retraining or expansion when handling new tasks, which leads to a rapid increase in model parameters and a significant consumption of computing resources. Furthermore, models often lack sufficient flexibility for different types of tasks, making it impossible to adaptively adjust to the specific characteristics of the task. Furthermore, traditional model training methods typically require the use of the entire data set. This not only increases training time when the data volume is large, but also easily leads to a waste of computing resources.

[0005] Application Contents

[0006] This application provides a large model continuous learning method based on hierarchical adaptive optimization, which effectively controls the unlimited expansion of the model when the number of tasks is large and the task relevance varies significantly.

[0007] In a first aspect, a large model continuous learning method based on hierarchical adaptive optimization is provided, the method comprising:

[0008] Obtaining a training set for the current task, building a continuous learning model, and training the continuous learning model, and using the trained continuous learning model to classify visual problem images; wherein the training process specifically includes:

[0009] Using a pre-trained task encoder to extract representations from each part of the training set and store them in the system cache; wherein the training data in the training set includes a text instruction part, an image part, and a text output part;

[0010] Perform similarity evaluation on the corresponding partial representations of the current task and the known tasks, and determine the model structure expansion or model structure reuse based on the relative size of the evaluation score and the predetermined threshold;

[0011] When the model structure expansion is determined, the current task representation in the task selector is randomly initialized, and the structure expansion training is performed using the preset loss function;

[0012] Backpropagation calculates the model gradient, updates the gradient of the current task-specific module and the learnable current task representation, and optimizes the model according to the updated gradient until training is completed.

[0013] Furthermore, after optimizing the model according to the updated gradient until training is completed, the method further includes:

[0014] The model is tested by calculating the similarity score between the test sample input and the known task representation in the task selector. The task-specific module corresponding to the task with the largest similarity score is selected to complete the model reasoning.

[0015] Furthermore, the pre-trained task encoder is used to extract representations of each part of the training set, including:

[0016] The training set is represented and extracted through the image encoder and text encoder.

[0017] Furthermore, the similarity evaluation is performed on the corresponding partial representations of the current task and the known tasks, including:

[0018] Use the pre-trained task encoder to extract the representation of each part of the training data to obtain the text instruction part representation, the image part representation and the text output part representation;

[0019] Calculate the similarity between the current task and each part of the known task representation, and obtain the text instruction part similarity vector, the image part similarity vector, and the text output part similarity vector;

[0020] The three types of similarity vectors are normalized and fused.

[0021] Furthermore, the model structure expansion or model structure reuse is determined based on the relative size of the evaluation score and a predetermined threshold, including:

[0022] When the evaluation score after the three types of similarity vectors are normalized and fused is less than a predetermined threshold, it is determined that the model structure expansion is performed;

[0023] Otherwise, the model structure is reused. When the model structure is reused, the current round of training is performed based on the specific structure and task representation of the most similar task.

[0024] Furthermore, the three types of similarity vectors are normalized and fused, including:

[0025] By formula

[0026] Standardization is performed, where represents the similarity vector, μ and σ represent the normalization parameters respectively;

[0027] And through the formula

[0028] To integrate, Represent three types of normalized vectors respectively.

[0029] Furthermore, a preset loss function is used for structure expansion training, wherein the loss function specifically includes: L pull =(1-γ(e img (x v ,k v )))+(1-γ(e text (x t ,k t )))

[0030] Among them, L pull represents the loss function, γ represents the similarity measure, e img The image encoder representing the task representation, e text Text encoder representing the task representation, x v 、x t Represent the sample input part of the current task, k v 、k t They represent randomly initialized task representations respectively.

[0031] In a second aspect, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the large-model continuous learning method based on hierarchical adaptive optimization as described in any one of the first aspects above.

[0032] In a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the large-model continuous learning method based on hierarchical adaptive optimization as described in any one of the first aspects above is implemented.

[0033] In a fourth aspect, an electronic device is provided, comprising a computer program / instruction, which, when executed by a processor, implements the large-model continuous learning method based on hierarchical adaptive optimization as described in any one of the first aspects above.

[0034] This application has the following advantages:

[0035] (1) The model is expanded and trained in different layers according to different types of tasks. The model has higher flexibility and can be trained without using all the data each time, which reduces the demand for computing power.

[0036] (2) Using task similarity to evaluate model structure expansion and reuse avoids the unlimited expansion of the model for new tasks. During inference, the model selects task-specific modules based on the input portion of the sample, without having to use all parameters for inference, which alleviates the impact of increasing parameters on model inference speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] To more clearly illustrate the embodiments of this application or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are merely illustrative, and those skilled in the art can derive other implementation drawings based on the provided drawings without inventive effort.

[0038] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with this technology. They are not intended to limit the conditions under which this application can be implemented, and therefore have no substantive technical significance. Any structural modifications, changes in proportions, or adjustments in sizes should still fall within the scope of the technical contents disclosed in this application without affecting the efficacy and objectives that can be achieved by this application.

[0039] FIG1 is a schematic diagram of the framework of the model structure expansion method based on task correlation in this application;

[0040] FIG2 is a flow chart of model training provided in an embodiment of the present application;

[0041] FIG3 is a diagram showing the internal structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] The following specific embodiments illustrate the implementation of this application. Those familiar with the art can easily understand the other advantages and functions of this application from the content disclosed in this specification. Obviously, the described embodiments are part of the embodiments of this application, but not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0043] The present application proposes a large-model continuous learning method based on hierarchical adaptive optimization, which can realize hierarchical adaptive training and reasoning of large-scale distributed models. Specifically, the present application proposes that the system adaptively reuses or expands the model structure according to the correlation score between the new task and the known task, while controlling the number of model parameters from increasing without limit, and realizing hierarchical adaptation to different task types. Its structure is shown in Figure 1. Obtain the training set of the current task, build a continuous learning model, and train the continuous learning model, and use the trained continuous learning model to classify visual problem images. Figure 2 shows the model training flow chart in the large-model continuous learning method based on the above structure, specifically:

[0044] S1, uses the pre-trained task encoder to extract representations of each part of the training set and stores them in the system cache.

[0045] The training data in the training set includes a text instruction part, a picture part, and a text output part. In the embodiment of the present application, the symbol x t Represents the text instruction part of the input sample, x v Represents the image part of the input sample, x o represents the text output part of the sample, γ represents the similarity measure, e img The image encoder representing the task representation, e text A text encoder representing a task representation.

[0046] S2, perform similarity evaluation on the corresponding partial representations of the current task and the known tasks, and determine the model structure expansion or model structure reuse based on the relative size of the evaluation score and the predetermined threshold.

[0047] S3, when the model structure expansion is determined, the current task representation in the task selector is randomly initialized.

[0048] Before training begins, use the pre-trained task encoder to extract the representation of each part of the training data. img (x v ),e text (x t ),e text (x o ), the average representations of the entire data set are e v ,e t ,e o . These representations are compared with the parts of the known task representations stored in the system and similarity is calculated. The process is formalized as follows:

[0049] This gives the similarity vector between the current task and all known tasks: Finally, for the three types of similarity vectors Standardize and merge:

[0050] Represent three types of normalized vectors respectively. Similarly, the other two similarity vectors can also be obtained according to formula (4).

[0051] S4, uses the preset loss function to perform structure expansion training.

[0052] In this embodiment, the task representation is stored in the task selector and is used for task query during model inference. That is, when the input of the test sample is known, the similarity between it and the task representation is compared to retrieve the most similar known task, and the model inference is completed using the task-specific module. Therefore, the task representation must be obtained during the model training process and can represent the characteristics of the sample input part of the task. Before the current task training begins, the task representation k is randomly initialized. v ,k t , and then use the sample input part x of the current task v ,x t , trained according to the following loss function: L pull =(1-γ(e img (x v ,k v )))+(1-γ(e text (x t ,k t ))) (6)

[0053] Among them, L pull represents the loss function, γ represents the similarity measure, e img The image encoder representing the task representation, e text Text encoder representing the task representation, x v 、x t Represent the sample input part of the current task, k v 、k t They represent randomly initialized task representations respectively.

[0054] S5, backpropagation calculates the model gradient, updates the gradient of the current task-specific module and the learnable current task representation, and optimizes the model according to the updated gradient until training is completed.

[0055] From the above, we can see that the training process of the system is divided into multiple stages according to different task forms. Among them, the current training is mainly divided into four stages:

[0056] Stage 1 calculates the average representation e of the training set of the current task v ,e t ,e o , and store it in the system cache for similarity evaluation with known tasks and future tasks.

[0057] In stage two, the similarity between the current task and known tasks is evaluated. The comparison of the evaluation score with a predetermined threshold determines whether the model structure should be expanded or reused. If the model is expanded, a new task-specific module and task representation are added, and their parameters are randomly initialized. Otherwise, the task-specific module with the highest similarity is selected, and the task representation is used as a trainable parameter to enter stage three training.

[0058] Phase 3 is the training of the new module. The forward propagation process of the sample in the model is divided into two parts: (1) The sample of the current task (x v ,x t ) The input model backbone is forward propagated with the current task-specific module, and the labeled real output x o Calculate the autoregressive loss. This part of the loss is used to train the new task-specific module. (2) The sample of the current task (x v ,x t ) respectively input the image encoder e of the task representation img With text encoder text , which is used to calculate the task representation loss function of formula (6). This part of the loss value is used to update the parameters of the learnable task representation.

[0059] S6, tests the model, calculates the similarity score between the test sample input part and the known task representation in the task selector, selects the task-specific module corresponding to the task with the largest similarity score, and completes the model reasoning.

[0060] A hierarchical training approach based on different task types ultimately yields a model that performs well across all known tasks. During testing, the model queries the task representation based on the input of the test sample. The task-specific module corresponding to the query result is then used to complete reasoning for that sample.

[0061] In summary, this application enables the continuous transfer and adaptation of large-scale models to downstream tasks. Unlike multi-task joint training, this method hierarchically trains the model's task-specific modules based on different task types. By evaluating the relevance of new tasks to existing ones, this method can adaptively reuse or expand the large model structure, while preventing an unlimited increase in model parameters and achieving hierarchical adaptation to different task types. If the score exceeds a set threshold, the model structure is expanded and the current task representation in the task selector is randomly initialized; otherwise, the model structure and task representation of the known task with the highest similarity are reused. During training, the current task-specific structure and representation are trained using the current task data. Finally, after hierarchical training for each task, a hierarchical model consisting of a backbone network, a task-specific structure, and a task selector is obtained. This model can adaptively perform model inference based on the input query task-specific structure. This application proposes a continuous learning method for large models based on hierarchical adaptive optimization to achieve the continuous transfer and adaptation of large-scale models to downstream tasks. It encourages hierarchical training of the model's task-specific modules based on different task types, effectively improving the model's flexibility and performance across tasks. Secondly, this application effectively controls the unlimited expansion of the model, and has significant advantages, especially when the number of tasks is large and the differences in task relevance are obvious.

[0062] In an optional embodiment of the present application, the method further includes the following steps:

[0063] Step 1: Before training begins, all samples in the current domain pass through the image encoder e represented by the task img With text encoder text , get the average representation e of the training set of the current task v ,e t ,e o and stores it in the system cache.

[0064] Step 2: Calculate the similarity measure between the current task and each known task according to formulas (1)-(5). If max(s i ) is less than the threshold, go to step 3; otherwise, select argmax(s i )’s task-specific module and task representation as the learnable parameters of the current task, and proceed to step 4.

[0065] Step 3: Extend the new current task specific module and randomly initialize the current task representation in the task selector.

[0066] Step 4: During training, the sample of the current task (x v ,x t ) The input model backbone is forward propagated with the current task-specific module, and the labeled real output xo Calculate supervised loss. (2) Samples of the current task (x v ,x t ) and the current task representation according to the task representation loss function of formula (6).

[0067] Step 5: Backpropagate the model gradient and perform gradient updates on the current task-specific module and the learnable current task representation.

[0068] Step 6: During testing, the similarity score between the test sample input and the known task representation in the task selector is calculated, and the task-specific module corresponding to the task with the largest similarity score is selected to complete model inference.

[0069] In one embodiment, an electronic device is provided, which may be a server, and its internal structure diagram may be shown in FIG3 . The electronic device includes a processor, a memory, and a network interface connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities, and the network interface is used to communicate with an external terminal via a network connection. The electronic device implements the aforementioned patent multi-domain knowledge extraction method by loading and running a computer program.

[0070] Those skilled in the art will understand that the structure shown in Figure 3 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0071] In one embodiment, a computer-readable storage medium is further provided, on which a computer program is stored, which involves all or part of the processes in the above-mentioned embodiment method.

[0072] In one embodiment, a computer program product is also provided, including a computer program / instruction, which involves all or part of the process in the above embodiment method.

[0073] Although the present application has been described in detail above using general descriptions and specific embodiments, it will be apparent to those skilled in the art that modifications or improvements may be made based on the present application. Therefore, such modifications or improvements, which do not depart from the spirit of the present application, are within the scope of protection claimed in the present application.

Claims

1. A large model continuous learning method based on hierarchical adaptive optimization, characterized in that: include: Obtaining a training set for the current task, building a continuous learning model, and training the continuous learning model, and using the trained continuous learning model to classify visual problem images; wherein the training process specifically includes: Using a pre-trained task encoder to extract representations from each part of the training set and store them in the system cache; wherein the training data in the training set includes a text instruction part, an image part, and a text output part; Perform similarity evaluation on the corresponding partial representations of the current task and the known tasks, and determine the model structure expansion or model structure reuse based on the relative size of the evaluation score and the predetermined threshold; When the model structure expansion is determined, the current task representation in the task selector is randomly initialized, and the structure expansion training is performed using the preset loss function; Backpropagation calculates the model gradient, updates the gradient of the current task-specific module and the learnable current task representation, and optimizes the model according to the updated gradient until training is completed.

2. The large model continuous learning method based on hierarchical adaptive optimization according to claim 1 is characterized in that: After optimizing the model according to the updated gradient until training is completed, the method further includes: The model is tested by calculating the similarity score between the test sample input and the known task representation in the task selector. The task-specific module corresponding to the task with the largest similarity score is selected to complete the model reasoning.

3. The large model continuous learning method based on hierarchical adaptive optimization according to claim 1 is characterized in that: Use the pre-trained task encoder to extract representations of each part of the training set, including: The training set is represented and extracted through the image encoder and text encoder.

4. The large model continuous learning method based on hierarchical adaptive optimization according to claim 1 is characterized in that: Evaluate the similarity between the current task and the corresponding representations of each part of the known task, including: Use the pre-trained task encoder to extract the representation of each part of the training data to obtain the text instruction part representation, the image part representation and the text output part representation; Calculate the similarity between the current task and each part of the known task representation, and obtain the text instruction part similarity vector, the image part similarity vector, and the text output part similarity vector; The three types of similarity vectors are normalized and fused.

5. The large model continuous learning method based on hierarchical adaptive optimization according to claim 4 is characterized in that: Determine model structure expansion or model structure reuse based on the relative size of the evaluation score and the predetermined threshold, including: When the evaluation score after the three types of similarity vectors are normalized and fused is less than a predetermined threshold, it is determined that the model structure expansion is performed; Otherwise, the model structure is reused. When the model structure is reused, the current round of training is performed based on the specific structure and task representation of the most similar task.

6. The large model continuous learning method based on hierarchical adaptive optimization according to claim 4 is characterized in that: Three types of similarity vectors are normalized and fused, including: By formula Standardization is performed, where represents the similarity vector, μ and σ represent the normalization parameters respectively; And through the formula To integrate, Represent three types of normalized vectors respectively.

7. The large model continuous learning method based on hierarchical adaptive optimization according to claim 1 is characterized in that: In the structure expansion training using the preset loss function, the loss function specifically includes: L pull =(1-γ(e img (x v ,k v )))+(1-γ(e text (x t ,k t ))) Among them, L pull represents the loss function, γ represents the similarity measure, e img The image encoder representing the task representation, e text Text encoder representing the task representation, x v 、x t Represent the sample input part of the current task, k v 、k t They represent randomly initialized task representations respectively.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images