Metalearning inner layer rapid updating strategy algorithm based on gradient information
By using gradient information differences to adjust the inner layer update step size, the problems of large calculation overhead and poor generalization ability in meta-learning rapid update are solved, and a more efficient and stable model adaptation to new tasks is achieved.
Patent Information
- Application Number
- CN202510268227.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-07
- Publication Date
- 2025-08-15
AI Technical Summary
The existing meta-learning methods have large calculation overhead, poor generalization ability, insufficient task adaptability during rapid updates, and are not performing well in cases where task diversity is high or sample size is small.
By calculating the initial gradient information, generating perturbation parameters and recalculating the loss gradient, adjusting the inner layer update step by using gradient differences, and combining MAML, FOMAML or Reptile frameworks to achieve adaptive inner layer rapid update.
It improves the computational efficiency and stability of the meta-learning model, enhances the model's adaptability to new tasks, reduces computational overhead and improves generalization performance.
Smart Images
Figure CN120492918A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of meta-learning, and more specifically relates to a meta-learning inner layer fast update strategy algorithm based on gradient information. Background Art
[0002] In the field of meta-learning, fast update algorithms have become an important research direction in recent years, especially how to quickly adapt to new tasks with a small number of samples. Currently, one of the most representative technologies is the algorithm based on model-agnostic meta-learning (MAML). MAML optimizes model parameters so that it can quickly adapt to different tasks with a small number of gradient updates. Specifically, MAML trains the model through a second-order optimization process during the training phase, allowing the model to quickly adapt to new tasks using very few gradient updates.
[0003] However, although MAML and its derivative algorithms have achieved good results in multiple application scenarios, there are still some problems. First, the MAML algorithm requires relatively complex calculations and a long training time, especially when faced with a large number of tasks, the computational overhead is huge. Second, MAML assumes that the similarity between tasks is high, but in some practical applications, the differences between tasks are large, resulting in its fast update effect being less than expected. More importantly, the parameter update speed and stability of MAML and other similar algorithms are still issues that need to be improved, especially when the tasks are very complex or the number of samples is very small, it may not be possible to achieve sufficiently efficient fast updates. Existing MAML variants such as Sharp-MAML guide the internal update of the MAML model by utilizing the differences in model loss, but this approach only considers the loss difference and does not pay attention to the gradient difference.
[0004] In addition to MAML, there are other fast-update algorithms. For example, methods based on memory-augmented neural networks (MANNs) introduce an external memory mechanism, enabling the model to quickly transfer and adapt to tasks based on a small number of samples. Although this method performs well on some tasks, its model structure is relatively complex and prone to overfitting when processing large amounts of data, resulting in performance degradation.
[0005] In addition, algorithms such as Reptile and ProtoNet have also attempted to improve the effectiveness of fast updates through optimization strategies. These methods primarily focus on optimizing model initialization for faster task adaptation. The Reptile method improves computational efficiency by performing meta-learning through simpler gradient updates during training. The ProtoNet method, on the other hand, achieves good results in few-shot learning tasks by learning prototypes of samples for classification. However, these methods still face similar challenges in scenarios with high task diversity: how to maintain the efficiency and generalization capabilities of fast updates.
[0006] In summary, while existing meta-learning methods, especially those targeting fast updates, have made significant progress in certain tasks, they still suffer from high computational overhead, poor generalization, and poor task adaptability. This paper proposes a new fast update scheme that effectively overcomes these shortcomings by leveraging gradient information differences to rapidly control meta-learning internal updates. This approach improves model stability and adaptability while maintaining high efficiency. Summary of the Invention
[0007] The purpose of this invention is to control the model optimization towards a flatter (better generalization) region by utilizing the difference in gradient information, and at the same time use the gradient difference to control the size of the update step to achieve the purpose of adaptation. This method has shown excellent results in experiments and has achieved excellent results in combination with models such as MAML, FOMAML, and Reptile.
[0008] In order to achieve the above object, the present invention is implemented by adopting the following technical solution: the algorithm includes:
[0009] Calculate the initial gradient of the initial model parameters on the current task;
[0010] Apply controlled perturbations to the parameters along the initial gradient direction to generate perturbation parameters;
[0011] Recalculate the loss based on the perturbed parameters and obtain the updated gradient;
[0012] Calculate the gradient difference to characterize the local curvature of the loss function;
[0013] According to the dynamic adjustment of the inner layer update step size, the step size adjustment formula is:
[0014] The model parameters are updated using the adjusted step size to achieve fast adaptation of the inner layer.
[0015] In one solution, the perturbation direction is limited to the initial gradient direction, and the perturbation amplitude is controlled by the scaling factor η after gradient normalization, with the value range of η being 0.001 and the value range of ε being 1e-81e-6; the normalization operation is implemented by calculating the L2 norm of the initial gradient to ensure that the perturbation amplitude is proportional to the gradient amplitude, thereby avoiding the problem of uncontrollable update direction caused by random perturbations.
[0016] In one embodiment, the step size adjustment further includes a threshold judgment mechanism: when Δg is greater than a preset threshold τ, the step size is reduced to α_new = α·β, where β is the attenuation coefficient and 0<β<1; when Δg is less than or equal to τ, the step size is expanded to α_new = α·(1+β), thereby reducing the step size in steep areas to prevent oscillation and increasing the step size in flat areas to accelerate convergence; the threshold τ is dynamically calculated based on the statistical distribution of Δg in multi-task training, and is 1.5 to 2 times the sliding average of Δg.
[0017] In one embodiment, the calculation of the gradient difference Δg further includes analyzing the directional similarity of the two gradients: by calculating the cosine similarity of g_perturbed and g_initial, if the similarity is lower than a threshold, it is determined to be a high curvature area, and α_new is forced to be set to the minimum value α_min;
[0018] In one embodiment, the algorithm is further combined with the MAML, FOMAML or Reptile framework to replace the fixed step size update with an adaptive step size update in the inner loop; in the MAML framework, the update formula is θ' = θ - α_new·g_initial, and in the Reptile framework, the update result is input into the parameter sliding average calculation in the outer layer.
[0019] In one solution, gradient calculation is implemented through an automatic differentiation tool, and some parameters of the model are frozen when calculating g_perturbed. Only the fully connected layer or convolution kernel weights are perturbed and the gradients are recalculated to reduce computational overhead; the proportion of frozen parameters is 30% to 50%.
[0020] In one embodiment, the algorithm further includes multi-task parallel perturbation verification in the meta-training stage: perturbations are applied simultaneously to multiple tasks in the same batch, Δg of each task is calculated, and a weighted average is taken as the global curvature estimate, where the weight is the inverse of the loss value of each task; the frequency of the multi-task parallel perturbation verification is once every 10 to 20 iterations.
[0021] In one scheme, the algorithm further supports dynamic adjustment of the perturbation direction: random perturbations in orthogonal directions are superimposed on the initial gradient direction, with a perturbation amplitude of 10% to 20% of the initial gradient amplitude, to explore curvature information in non-gradient directions; the orthogonal directions are generated by the Gram-Schmidt orthogonalization method.
[0022] Beneficial effects of the present invention:
[0023] The present invention can improve computational efficiency by taking into account that the meta-learning model fully utilizes the gradient difference information (which has a better effect than the loss difference information) without calculating the second-order derivative (this method is the first order). At the same time, the internal update step size (internal learning rate) is controlled according to the gradient difference information. The parameters can be adaptively adjusted according to the current meta-learning model training situation, thereby making the model training more stable. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 This is a flow chart of the algorithm of the present invention. DETAILED DESCRIPTION
[0025] To facilitate understanding of the present invention, the present invention will be described more fully below with reference to the accompanying drawings. The drawings illustrate exemplary embodiments of the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the present invention.
[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meanings as those understood by those skilled in the art to which the present invention pertains. The terms used in the present specification are for the purpose of describing specific embodiments only and are not intended to limit the present invention. To facilitate understanding of the present invention, a more comprehensive description of the present invention will be provided below with reference to the accompanying drawings. Typical embodiments of the present invention are shown in the drawings. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the present invention.
[0027] like Figure 1 As shown in Figure 5, to solve this problem, a fast direction search method is designed to guide the model to update towards flatter areas during the inner multi-task process.
[0028] If the perturbation is performed using random perturbations (e.g., using random numbers), the effect of the perturbation becomes uncontrollable. This lack of control means that the direction and magnitude of the perturbation may not have clear meaning and may cause the model to update parameters in a direction that is not ideal.
[0029] In order to achieve a meaningful update direction, the way of perturbing the parameters is modified. First, the initial gradient is calculated: After computing the gradient, the parameters θ are perturbed along the gradient direction by adding a scaled perturbation:
[0030]
[0031] where ρ is the scaling factor of the perturbation, is the L2 norm of the initial gradient, and ∈ is a small constant used to prevent division by zero errors. Control the size of the perturbation to ensure that the direction and magnitude of the gradient remain within a reasonable range.
[0032] In the previous step, the perturbed parameters θ' were obtained. Then, these perturbed parameters were used to recalculate the loss L1 = L task (θ'), and calculate the gradient At this point, the initial gradient g0 and the updated gradient g1 are obtained.
[0033] calculate:
[0034] g1-g0=Δg1#(4-14)
[0035] This directly reflects the curvature of the loss function in the local area. If the gradient change Δg is small, it means that the gradient difference before and after the parameter update is small, which usually means that the area where the model is located is relatively flat, because the loss function in the flat area does not change much, and the direction and magnitude of the gradient will not fluctuate dramatically.
[0036] After obtaining the gradient difference Δg1, the parameters are further optimized by utilizing this approximate second-order gradient information. Specifically, the parameters are updated as follows:
[0037]
[0038] Using the new parameter θ'2, a new gradient g2 can be obtained based on θ'2, and the new gradient difference Δg2 between g2 and g0 is calculated. At this time, the magnitudes of Δg1 and Δg2 are compared to determine the degree of change in the model parameters in different directions, thereby helping to evaluate the flatness of the region.
[0039] The inner loop step size α of the model is allowed to be adaptively adjusted according to the gradient difference. When Δg is large, it indicates that the area is steep, so the inner loop step size is reduced to prevent excessive updates in steep areas, causing the model to fall into a local optimal solution. Conversely, when Δg is small, it indicates that the area is flat, so the inner loop step size is increased to pass through the flat area more efficiently. Specifically, the inner loop step size is adaptively controlled using the following rules:
[0040]
[0041] After obtaining the new step size α, the model parameters can be updated accordingly according to different flat areas:
[0042] θ updated =θ-α·(g0+η·g i )#(4-17)
[0043] With this new update strategy, the inner loop of the MAML model is optimized by considering more directional information of task-specific gradients.
[0044] Example:
[0045] In optimization-based meta-learning, the method adopted is MAML (Model-Agnostic Meta-Learning), whose core idea is to quickly adapt to new tasks through a small amount of gradient updates.
[0046] Suppose we have a set of tasks T1, T2, ..., T n and the corresponding loss function Where θ is the parameter of the model. The goal of meta-learning is to find an initial model parameter θ0 so that the model can be quickly updated on new tasks.
[0047] Meta-optimization goals:
[0048]
[0049] T represents the task set. In task T i The loss function on .
[0050] In task T i The gradient of the model parameters θ0. α is the learning rate.
[0051] The goal is to select a good initial model parameter θ0 so that the loss is minimized after one or several gradient updates on a given task.
[0052] Internal update (the process of updating on a task):
[0053]
[0054] Among them, θ' i In task T i The parameters updated by gradient descent are α, and α is the learning rate of the internal update.
[0055] Internal update describes the task T i Through a gradient update from the initial parameter θ0 to θ' iThis process is called fast adaptation, and its purpose is to enable the model to quickly adapt to the task T through a small amount of gradient updates. i specific requirements.
[0056] External update (meta-update process): At the meta-learning level, the goal of external update is to update the initial parameters θ0 by the losses of multiple tasks. This process optimizes:
[0057]
[0058] where β is the learning rate of the external optimization.
[0059] The outer update aims to optimize the initial parameters of the entire model so that after a small number of inner updates, good performance can be achieved on the new task.
[0060] By training on multiple tasks, we find an initial model parameter θ0 that can quickly adapt to new tasks. In other words, the model can quickly adjust its parameters through internal updates to cope with different tasks.
[0061] On each specific task, the model adjusts its parameters by calculating the gradient and performing a gradient descent update. This is a fast adaptation step that the model performs when it "sees" a new task.
[0062] This is the core of meta-learning. The goal is to adjust the initial parameters θ0 through the performance on multiple tasks so that the model can better adapt to new tasks in the future.
[0063] This method can be applied to the internal update process of meta-learning (MAML), that is,
[0064]
[0065] Yes, the parameter update can be completed through a fast internal update strategy.
[0066] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).
[0067] It should be understood that the detailed description of the technical solutions of the present invention using the preferred embodiments above is illustrative and not restrictive. A person skilled in the art, after reading the present specification, may modify the technical solutions described in the embodiments or replace some of the technical features therein with equivalents; such modifications or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A meta-learning inner layer fast update strategy algorithm based on gradient information, characterized by: The algorithm includes: Calculate the initial gradient of the initial model parameters on the current task; Apply controlled perturbations to the parameters along the initial gradient direction to generate perturbation parameters; Recalculate the loss based on the perturbed parameters and obtain the updated gradient; Calculate the gradient difference to characterize the local curvature of the loss function; According to the dynamic adjustment of the inner layer update step size, the step size adjustment formula is: The model parameters are updated using the adjusted step size to achieve fast adaptation of the inner layer.
2. The meta-learning inner layer fast update strategy algorithm based on gradient information according to claim 1 is characterized by: The direction of the perturbation is limited to the direction of the initial gradient, and the perturbation amplitude is controlled by the scaling factor η after gradient normalization, with a value range of η of 0.001 and a value range of ε of 1e−81e-6. The normalization operation is implemented by calculating the L2 norm of the initial gradient to ensure that the perturbation amplitude is proportional to the gradient amplitude, avoiding the problem of uncontrollable update direction caused by random perturbations.
3. The meta-learning inner layer fast update strategy algorithm based on gradient information according to claim 1, characterized in that: The step size adjustment further includes a threshold judgment mechanism: when Δg is greater than a preset threshold τ, the step size is reduced to α_new = α·β, where β is the attenuation coefficient and 0<β<1; when Δg is less than or equal to τ, the step size is expanded to α_new = α·(1+β), thereby reducing the step size in steep areas to prevent oscillation and increasing the step size in flat areas to accelerate convergence; the threshold τ is dynamically calculated based on the statistical distribution of Δg in multi-task training and is 1.5 to 2 times the sliding average of Δg.
4. The meta-learning inner layer fast update strategy algorithm based on gradient information according to claim 1, characterized in that: The calculation of the gradient difference Δg further includes analyzing the directional similarity of the two gradients: by calculating the cosine similarity of g_perturbed and g_initial, if the similarity is lower than a threshold, it is determined to be a high curvature area, and α_new is forced to be set to the minimum value α_min.
5. The meta-learning inner layer fast update strategy algorithm based on gradient information according to claim 1, characterized in that: The algorithm is further combined with the MAML, FOMAML, or Reptile frameworks by replacing the fixed-step update with an adaptive step-size update in the inner loop; in the MAML framework, the update formula is θ' = θ - α_new·g_initial, and in the Reptile framework, the update result is input into the parameter sliding average calculation in the outer layer.
6. The meta-learning inner layer fast update strategy algorithm based on gradient information according to claim 1, characterized in that: Gradient calculation is implemented through automatic differentiation tools, and some model parameters are frozen when calculating g_perturbed. Only the fully connected layers or convolution kernel weights are perturbed and gradients are recalculated to reduce computational overhead. The proportion of frozen parameters is 30%~50%.
7. The meta-learning inner layer fast update strategy algorithm based on gradient information according to claim 1, characterized in that: The algorithm further includes multi-task parallel perturbation verification in the meta-training stage: perturbations are applied simultaneously to multiple tasks in the same batch, Δg is calculated for each task, and a weighted average is taken as the global curvature estimate, where the weight is the inverse of the loss value of each task; the multi-task parallel perturbation verification is performed once every 10 to 20 iterations.
8. The meta-learning inner layer fast update strategy algorithm based on gradient information according to claim 1, characterized in that: The algorithm further supports dynamic adjustment of the perturbation direction: random perturbations in orthogonal directions are superimposed on the initial gradient direction, with a perturbation amplitude of 10% to 20% of the initial gradient amplitude, to explore curvature information in non-gradient directions; the orthogonal directions are generated by the Gram-Schmidt orthogonalization method.