Deep learning-based meta-learning method and apparatus therefor

The proposed meta-learning method uses gradient similarity and L2 loss values to approximate second derivatives, enabling efficient model optimization and parameter updates in large-scale deep learning models with limited data, addressing the inefficiencies of existing methods.

KR103044919B1Active Publication Date: 2026-11-06CHUNG ANG UNIV IND ACADEMIC COOP FOUND
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
KR1020240117775
Authority / Receiving Office
KR · KR
Patent Type
Patents
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2026-11-06
Estimated Expiration
2044-08-30

AI Technical Summary

Technical Problem

Existing meta-learning algorithms, such as MAML, require computationally expensive second derivative calculations, leading to inefficiencies in training large-scale deep learning models, particularly when data is limited.

Method used

A deep learning-based meta-learning method that calculates a hashian approximation using gradient similarity and L2 loss values, allowing updates with only the first derivative, reducing computational complexity and memory usage.

Benefits of technology

Enables efficient model optimization and parameter updates without second derivative calculations, maintaining performance comparable to MAML while reducing computational complexity and memory requirements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 112024095482682-PAT00111_ABST
    Figure 112024095482682-PAT00111_ABST
Patent Text Reader

Abstract

A deep learning-based meta-learning method and apparatus are disclosed. The deep learning-based meta-learning method comprises: (a) calculating a gradient for a loss function for a task at each gradient step during a first inner loop and updating model parameters for the task using the calculated gradient; (b) calculating a similarity value between gradients and an L2 loss value calculated at each gradient step during a second inner loop and calculating a hashian approximation value using the calculated similarity value between gradients and the L2 loss value; and (c) performing an outer loop and calculating a meta-gradient value using the hashian approximation value and updating meta-model parameters using the meta-gradient value.
Need to check novelty before this filing date? Find Prior Art

Description

Technology Field

[0001] The present invention relates to a deep learning-based meta-learning method and an apparatus thereof. Background Technology

[0003] Meta-learning is the process of learning how to learn, and it is a powerful technique that enables rapid learning from limited data. This approach learns through multiple sample tasks, each representing a unique learning problem. The primary objective of meta-learning is to acquire knowledge from these tasks to rapidly learn new tasks with a minimal number of samples. Meta-learning methods can be broadly categorized into distance-based, memory-based, and optimization-based techniques. Among these, optimization-based techniques are widely applicable and are used in various applications.

[0004] Along with this, the paradigm of fine-tuning following pre-training is also gaining attention in the field of neural networks, primarily for transfer learning and semi-supervised learning, and is contributing to improving the generalization ability and adaptability of models in various areas including computer vision, natural language processing, and reinforcement learning.

[0005] Deep learning models have a large number of parameters, which can be advantageous for utilizing vast amounts of training data. However, when training data is limited, the risk of overfitting increases, particularly when the number of meta-parameters is relatively large. In such cases, effective meta-learning algorithms that can efficiently train high-performance models even when sufficient prior training data is unavailable are essential.

[0006] Model Agnostic Meta-Learning (MAML) is an optimization-based approach that estimates optimal parameter initialization to enable rapid adaptation to new tasks. However, MAML requires the calculation of a second derivative due to the calculation of the derivative of the meta-loss with respect to the initial parameters, which has the disadvantage of being computationally expensive in large-scale networks. The problem to be solved

[0008] The present invention is intended to provide a deep learning-based meta-learning method and an apparatus thereof.

[0009] In addition, the present invention aims to provide a deep learning-based meta-learning method and apparatus capable of model optimization using only the calculation of the first derivative without the calculation of the second derivative.

[0010] In addition, the present invention aims to provide a deep learning-based meta-learning method and apparatus that determines a hash cyan approximation based on the similarity between each gradient and enables updating meta-model parameters using only the calculation of the first derivative. means of solving the problem

[0012] According to one aspect of the present invention, a deep learning-based meta-learning method is provided.

[0013] According to one embodiment of the present invention, a deep learning-based meta-learning method may be provided, comprising: (a) calculating a gradient for a loss function for a task at each gradient step during a first inner loop and updating model parameters for the task using the calculated gradient; (b) calculating a similarity value between gradients and an L2 loss value calculated at each gradient step during a second inner loop and calculating a hashian approximation value using the calculated similarity value between gradients and the L2 loss value; and (c) performing an outer loop and calculating a meta-gradient value using the hashian approximation value and updating meta-model parameters using the meta-gradient value.

[0014] The above gradient similarity value is the gradient cosine similarity value calculated at each gradient step.

[0015] The above hashian approximation is calculated using the following mathematical formula, but,

[0016]

[0017] Here, However, k represents the gradient step index of the inner loop, and represents the gradient, However, is, is represented as an interpolation rate that gradually decreases from 1 to 0, and represents the learning rate of the inner loop.

[0018] The similarity value between the gradients above is calculated using the following mathematical formula, but,

[0019]

[0020] Here, L represents the Lipschitz constant, and k represents the gradient step, represents the gradient at step k, and is, represents the auxiliary weight, and represents the coefficient of the L2-norm, and represents the L2-norm function.

[0021] The above metagradient value is calculated using the following mathematical formula, but,

[0022]

[0023] Here, represents the first derivative that calculates the gradient for the task's loss, and I represents the identity matrix.

[0025] According to another aspect of the present invention, an apparatus for performing a deep learning-based meta-learning method is provided.

[0026] According to one embodiment of the present invention, a computing device may be provided comprising: a memory for storing at least one instruction; and a processor for executing the instruction stored in the memory, wherein each instruction executed by the processor comprises: (a) a step of calculating a gradient for a loss function for a task at each gradient step during a first inner loop and updating model parameters for the task using the calculated gradient; (b) a step of calculating a similarity value between gradients and an L2 loss value calculated at each gradient step during a second inner loop and calculating a hashian approximation value using the calculated similarity value between gradients and an L2 loss value; and (c) a step of performing an outer loop, calculating a meta-gradient value using the hashian approximation value, and updating meta-model parameters using the meta-gradient value. Effects of the invention

[0028] By providing a deep learning-based meta-learning method and apparatus according to one embodiment of the present invention, there is an advantage that model optimization is possible with only the calculation of the first derivative without the calculation of the second derivative.

[0029] In addition, the present invention determines a hash cyan approximation based on the similarity between each gradient, and by using this, it is possible to update meta-model parameters with only the calculation of the first derivative, thereby reducing computational complexity and allowing for efficient use of memory. Brief explanation of the drawing

[0031] FIG. 1 is a flowchart illustrating a deep learning-based meta-learning method according to an embodiment of the present invention. FIG. 2 is a diagram illustrating pseudocode for a conventional MAML algorithm. FIG. 3 is a drawing illustrating an optimization process according to an embodiment of the present invention. FIG. 4 is a diagram illustrating pseudocode for a deep learning-based MAML algorithm according to an embodiment of the present invention. FIG. 5 is a diagram comparing the computational complexity and memory usage of a conventional meta-learning method and an embodiment of the present invention. FIG. 6 is a diagram comparing the accuracy of a conventional meta-learning method and an embodiment of the present invention. FIG. 7 is a block diagram schematically illustrating the internal configuration of a computing device that performs a deep learning-based meta-learning method according to an embodiment of the present invention. Specific details for implementing the invention

[0032] As used in this specification, singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "composed" or "comprising" should not be interpreted as necessarily including all of the various components or steps described in the specification, and should be interpreted as meaning that some of the components or steps may be excluded, or that additional components or steps may be included. Furthermore, terms such as "...part," "module," etc., as used in the specification refer to a unit that processes at least one function or operation, which may be implemented in hardware or software, or a combination of hardware and software.

[0033] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0035] FIG. 1 is a flowchart illustrating a deep learning-based meta-learning method according to an embodiment of the present invention, FIG. 2 is a diagram illustrating pseudocode for a conventional MAML algorithm, FIG. 3 is a diagram illustrating an optimization process according to an embodiment of the present invention, FIG. 4 is a diagram illustrating pseudocode for a deep learning-based MAML algorithm according to an embodiment of the present invention, FIG. 5 is a diagram comparing the computational complexity and memory usage of a meta-learning method according to a conventional method and an embodiment of the present invention, and FIG. 6 is a diagram comparing the accuracy of a meta-learning method according to a conventional method and an embodiment of the present invention. The deep learning-based meta-learning method described below is a method that can optimize a model by calculating only the first derivative without calculating the second derivative, based on a Model Agnostic Meta-Learning (MAML) algorithm. Since the MAML algorithm is obvious to those skilled in the art, a detailed description thereof will be omitted, and the following description will focus on the operations necessary to explain the main argument of the present invention.

[0036] In step 110, the computing device (100) calculates the gradient for the loss function for the task at each gradient step during the first inner loop and uses it to update the model parameters for the task.

[0037] The computing device (100) can sample various tasks, calculate a loss function for each task through an inner loop, calculate a gradient for the loss function, and then use this to update model parameters for the task. The first inner loop should be understood as a process of updating model parameters for a task by calculating a gradient through the calculation of a loss function for individual tasks.

[0038] In step 115, the computing device (100) calculates the similarity value between gradients and the L2 loss value calculated during the first inner loop during the second inner loop, respectively, and calculates the hashian approximation using these values.

[0039] In step 120, the computing device (100) performs an outer loop, calculates a meta-gradient value using a hashian approximation, and can update meta-model parameters using this.

[0040] This will be more clearly understood through the following explanation.

[0041] The primary goal of meta-learning is fundamentally to train models on various tasks so that they can rapidly adapt to new tasks with minimal effort.

[0042] The strength of MAML lies in the fact that it can be applied to any model trained using gradient descent, which means it is universally applicable. The core idea of ​​MAML is to find model initializations that are not only suitable for specific tasks but can also be rapidly fine-tuned for new tasks.

[0043] These advantages of MAML make it highly useful in the field of medical image classification / segmentation, where data is scarce but finely tuned initial parameters are important. Generally, rapidly adapted parameters is a randomly sampled task It can be calculated from.

[0044] Each parameter can be updated by a single support set to find the adaptive parameter.

[0045]

[0046] However, in practice, multiple gradient updates are often required. While it is possible to induce adaptation in a single set with a small number of updates, a single update is generally insufficient. Meta loss is It can be expressed as follows. Here, is a task Represents the query set, which represents a set separate from the support set. Model parameters based on the final meta-loss The update can be represented as in mathematical formula 2.

[0047]

[0048] Here, represents task distribution, and represents the learning rates of the inner loop and the outer loop, Is Indicates the meta-batch size of the outer loop for.

[0049] The pseudocode for the conventional MAML algorithm is as shown in Fig. 2.

[0050] Recently, the number of parameters in deep learning models has increased rapidly. In this situation, when applying MAML to quickly adapt to new tasks, the importance of computational efficiency grows, and first-order MAML algorithms can provide computational efficiency and expandability.

[0051] In first-order MAML, First-Order MAML (FOMAML), which avoids second derivatives, is receiving the most attention, and there is also a method using proximity normalization for first-order meta-learning strategies.

[0052] For example, "Nichol, A.; Achiam, J.; Schulman, J. On first-order meta-learning algorithms" uses only the derivative of proximity regularization as the gradient, and can adjust the interpolation between adaptation and meta-parameters as shown in Equation 3.

[0053]

[0054] Equation 3 presents a single gradient update within an inner loop for the sake of notational conciseness and is similar to Equation 1. As can be seen from Equation 3, unlike the MAML method, there is no need to divide the training data into a support set and a query set. This offers significant advantages in terms of data efficiency.

[0055] In addition, "Zhou, P.; Yuan, X.; Xu, H.; Yan, S.; Feng, J. Efficient meta learning via minibatch proximal update." applies proximity regularization to adaptive parameter updates. - After completing the inner loop using approximation, meta-parameters can be updated as adaptive parameters in a manner consistent with "Nichol, A.; Achiam, J.; Schulman, J. On first-order meta-learning algorithms".

[0056] The rapidly adapted parameters are expected to be close to the optimal parameters. Consequently, the length of the inner loop is generally set to a value greater than 1, and go Ensure it matches. Utilize the chain rule for the gradient of the meta-parameter, i.e., in Equation 2. It can be expressed as in mathematical formula 4.

[0057]

[0058] By continuously applying the chain rule, the inner loop calculation can be further extended as shown in Equation 5.

[0059]

[0060] Referring to mathematical formula 1, and The relationship between them is established, and this ultimately The expression can be derived as shown in mathematical formula 6.

[0061]

[0062] Consequently, it can be seen that the second derivative must be calculated at each step while executing the inner loop. The presence of the second derivative helps reduce the loss in MAML training, which evaluates post-adaptation generalization from the parameter set prior to the adaptation phase. However, this feature increases the length of the inner loop and increases MAML computation as the number of model parameters increases. In particular, the computational complexity of the second derivative is As, of the first derivative It is significantly higher than the complexity. This difference in complexity highlights the potential inefficiencies of MAML when dealing with large-scale models or long inner loops.

[0063] However, the FOMAML algorithm bypasses the calculation of the second derivative and utilizes only the gradient. It can be represented as follows.

[0064] In the case of such FOMAML, for MAML operations Using the hashian-vector product to avoid calculations Problems related to complexity can be effectively avoided. Hessian-vector product The result can be expressed as in mathematical formula 7.

[0065]

[0066] scalar value After calculating the gradient for , the second derivative operation on the vector can be performed as shown in Equation 6. Consequently, computational complexity increases at a constant rate, approximately five times that of the first derivative, while memory usage increases by up to two times per inner loop. Although the impact may vary depending on the model or data structure, compared to FOMAL, MAML's memory usage increases by an additional 50% with each inner loop iteration. This means that if MAML performs two inner loop iterations, it uses approximately twice the memory of FOMAML. Since FOMAML does not need to save the graph for each inner loop iteration, it can maintain constant memory usage regardless of the number of iterations.

[0067] Additionally, FOMAML can set the second term (i.e., the term that derives the gradient of the loss with respect to the loss) to zero (0) when updating the metamodel. That is, the update of the metamodel parameters can be performed using only the first derivative of the loss after the inner loop update for each task.

[0068] That is, since the second term is ignored, the second term can be treated as zero (0).

[0069]

[0070] The result can be simply reduced to 1. This approach is derived by mathematical equation 9, and represents the definition of an object.

[0071]

[0072] In mathematical equation 9, the negative sign in the denominator of the middle formula is because the update in gradient descent proceeds in the negative direction of the gradient. FOMAML is and Approximating the ratio between them to 1, It ignores the fact that the reciprocal of is infinite. While this can demonstrate excellent performance in simple data structures where the inner loop length is appropriate, it faces potential problems as the loop length increases. The connection between model parameters and meta-parameters determined at the end of the inner loop may weaken, which can lead to significantly degraded performance compared to MAML.

[0073] Furthermore, even if the length of the inner loop is short, meta-gradients may not contain sufficient information to effectively mitigate overfitting during the testing phase. When adopting this approach, it is essential to consider these nuances in complex situations where the integrity of the model's adaptability is critical. Conventionally, the efficacy of optimization-based meta-learning algorithms has been mathematically analyzed in comparison with MAML, and as a result, various methods to effectively replace second derivative calculations have been studied.

[0074] In one embodiment of the present invention, instead of setting the second term to zero (0), an approximate hashian value is calculated using the cosine similarity loss between the gradients for each step calculated during the inner loop, and an update to the meta-model parameters can be performed using this.

[0075] I will explain this in more detail.

[0076] 2nd paragraph When the second derivative is negative, the weight of the corresponding gradient component increases, and when the second derivative is positive, the weight of the corresponding gradient component decreases.

[0077] In deep learning, it is generally assumed that the loss exhibits both non-convex and convex characteristics (see Fig. 3). If the second derivative has a large positive value, it can be estimated that the corresponding gradient component has already approached the optimal point. Conversely, if the second derivative is negative, it may indicate that the component is at a significant distance from the optimal point. Meta-learning aims to adapt rapidly to a given task while simultaneously avoiding overfitting to the optimal parameters.

[0078] thus, It can be inferred that it fine-tunes the gradients derived from the query set to be more efficient for generalization. Based on this analysis, a heuristic function for approximating the second derivative can be proposed.

[0079] According to one embodiment of the present invention, instead of improving generalization performance by excluding or replacing the second derivative, it is possible to obtain performance similar to MAML through the second derivative approximation.

[0080] Mathematical Equation 10 uses the hashian approximation function (referred to as AHE) to approximate the metagradient This shows a method for calculating the final gradient. According to one embodiment of the present invention, a weight map for fine-tuning the final gradient can be generated to enable a more effective update.

[0081] According to one embodiment of the present invention, the expected effect of the second derivative derived from the support set can be generated using AHE. AHE can be estimated using Equation 9.

[0082]

[0083] This is different from FOMAL fixing the hashian to 0. It finds an appropriate hashian value. AHE is the input set. Receive It is a variable that prints. Appropriate The method for updating and determining will be explained in more detail below. This is a function go -Exhibiting strong convexity, all Hashian condition for We assume that it satisfies the condition. Here, L represents the Lifshitz constant and I represents the identity matrix.

[0084] In the context of meta-learning where tasks are performed with limited data, the risk of overfitting increases when inner-loop learning is conducted. Therefore, It is common for both and L to have larger values. This is the hashian All eigenvalues ​​of This means that the lower bound is set by and the upper bound is set by L. This characteristic is expected to be advantageous as it provides a balanced approach that promotes efficient inner-loop learning while addressing overfitting in meta-learning settings.

[0085] Using two loss functions for gradient similarity loss this It can be made to approximate. This is very important because parameters are highly likely to overfit to the support set in most meta-learning inner loops. Consequently, gradients can have very small values, and in this case, floating-point errors may occur when the gradient is in the denominator. This second It hinders accurate approximation of the ratio. Therefore, interpolation is used It can be expressed as in mathematical formula 11.

[0086]

[0087]

[0088] Here, represents the interpolation ratio and can gradually decrease from 1 to 0 during the meta-learning phase.

[0089] In the case of general cosine similarity loss, vectors are normalized to unit vectors. For the sake of ease of understanding and explanation, we will briefly explain cosine similarity loss.

[0090] The cosine similarity loss function is defined as negative cosine similarity, which is a formula chosen because cosine similarity has a maximum value when similarity is high, and can be expressed as Equation 13.

[0091]

[0092] The strength of cosine similarity lies in its ability to capture the angular distance between vectors, which is particularly suitable for normalized feature vectors.

[0093] In these conventional cosine similarity loss functions, the magnitude of the loss is independent of the vector scale, but it still depends on the gradient.

[0094] Accordingly, in one embodiment of the present invention, the cosine similarity loss function is redefined as in Equation 14.

[0095]

[0096] Gradient similarity loss It can be calculated as in Equation 15 through an inner loop.

[0097]

[0098] represents the coefficients of the L2-norm, and L represents the number of layers associated with the gradients and parameters. All inputs to the loss function can be vectorized.

[0099] Pseudocode for a MAML algorithm using cosine similarity between gradients and approximate hashian values ​​according to one embodiment of the present invention is as shown in FIG. 4. In one embodiment of the present invention, For training, similarity values ​​between gradients for k iterations in the support set The gradients obtained from are averaged and used for training. This is similar to the update in Equation 3. Parameters It is used to adjust the difference between gradients through interpolation, and can enable a more stable approximation by effectively controlling the gradient magnitude of AHE. The scheduling of may vary depending on the characteristics of the meta-learning problem. As the possibility of overfitting in the inner loop increases The value can be reduced, and as shown in Equation 12, the larger Values ​​can be allowed. In addition, a cosine scheduling method that converges to 0 before the last outer loop finishes may be applied, and an appropriate You can also set the value directly.

[0100] As described above, in one embodiment of the present invention, meta-learning can be enabled by not calculating the hashian-vector product and using only the calculation of the first derivative. Correlation between gradients can be integrated in the inner loop, and gradients can be adjusted in the outer loop.

[0101] Figure 5 illustrates the results of comparing computational complexity and memory usage according to meta-learning according to a conventional method and an embodiment of the present invention. As shown in Figure 5, it can be seen that compared to FOMAML, computational complexity increases by twofold, but memory usage shows almost no difference.

[0102] Figure 6 shows the results of comparing the accuracy according to a conventional meta-learning method and an embodiment of the present invention. As shown in Figure 6, it can be seen that the meta-learning method according to an embodiment of the present invention has superior performance compared to FOMAML and exhibits performance similar to MAML.

[0103] Referring to Figures 5 and 6, it can be seen that the computational complexity increases by about twofold compared to FOMAML, but the computational complexity is significantly lower compared to MAML while showing similar performance to MAML.

[0105] FIG. 7 is a block diagram schematically illustrating the internal configuration of a computing device that performs a deep learning-based meta-learning method according to one embodiment of the present invention.

[0106] Referring to FIG. 7, a computing device (100) according to one embodiment of the present invention is configured to include a memory (710) and a processor (720).

[0107] The memory (710) stores instructions (program code) for performing a deep learning-based meta-learning method according to one embodiment of the present invention.

[0108] The processor (720) is a means for controlling internal components of a computing device that performs a deep learning-based meta-learning method according to one embodiment of the present invention. Additionally, the processor (720) can execute instructions stored in memory (710).

[0109] The instruction executed by the processor (720) can perform a series of processes including calculating a gradient for a loss function for a task at each gradient step during a first inner loop, updating model parameters for the task using the calculated gradient, calculating a similarity value between gradients and an L2 loss value calculated at each gradient step during a second inner loop, calculating a hashian approximation value using the calculated similarity value between gradients and an L2 loss value, performing an outer loop, calculating a meta-gradient value using the hashian approximation value, and updating meta-model parameters using the meta-gradient value.

[0110] As this is the same as explained with reference to Figure 1, a redundant explanation will be omitted.

[0112] The apparatus and method according to an embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the computer-readable medium may be those specifically designed and configured for the present invention, or they may be those known and available to a person skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0113] The hardware device described above may be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa.

[0114] The present invention has been described above with reference to its embodiments. Those skilled in the art will understand that the present invention may be implemented in modified forms without departing from the essential characteristics of the invention. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the invention is defined by the claims, not by the foregoing description, and all variations within the scope of the claims should be interpreted as being included in the invention.

Claims

Claim 1 A deep learning-based meta-learning method performed on a computing device, comprising: (a) calculating a gradient for a loss function for a task at each gradient step during a first inner loop and updating model parameters for the task using the calculated gradient; (b) calculating a similarity value between gradients and an L2 loss value calculated at each gradient step during a second inner loop and calculating a hashian approximation value using the calculated similarity value between gradients and the L2 loss value; and (c) performing an outer loop and calculating a meta-gradient value using the hashian approximation value and updating meta-model parameters using the meta-gradient value. Claim 2 A deep learning-based meta-learning method according to claim 1, characterized in that the similarity value between gradients is a cosine similarity value between gradients calculated at each gradient step. Claim 3 A deep learning-based meta-learning method according to claim 1, characterized in that the hashian approximation is calculated using the following mathematical formula. Here, However, k represents the gradient step index of the inner loop, and represents the gradient, However, is, is represented as an interpolation rate that gradually decreases from 1 to 0, and represents the learning rate of the inner loop. Claim 4 A deep learning-based meta-learning method according to claim 3, characterized in that the similarity value between the gradients is calculated using the following mathematical formula. Here, L represents the Lipschitz constant, and k represents the gradient step, represents the gradient at step k, and is, represents the auxiliary weight, and represents the coefficient of the L2-norm, and represents the L2-norm function. Claim 5 A deep learning-based meta-learning method according to claim 4, characterized in that the meta-gradient value is calculated using the following mathematical formula. Here, represents the first derivative that calculates the gradient for the task's loss, and I represents the identity matrix. Claim 6 A computer-readable recording medium having recorded program code for performing a method according to any one of claims 1 to 5. Claim 7 A computing device comprising: a memory for storing at least one instruction; and a processor for executing the instruction stored in the memory, wherein each instruction executed by the processor comprises: (a) a step of calculating a gradient for a loss function for a task at each gradient step during a first inner loop and updating model parameters for the task using the calculated gradient; (b) a step of calculating a similarity value between gradients and an L2 loss value calculated at each gradient step during a second inner loop and calculating a hashian approximation value using the calculated similarity value between gradients and an L2 loss value; and (c) a step of performing an outer loop, calculating a meta-gradient value using the hashian approximation value, and updating meta-model parameters using the meta-gradient value. Claim 8 A computing device according to claim 7, characterized in that the gradient similarity value is a gradient cosine similarity value calculated at each gradient step. Claim 9 A computing device according to claim 7, characterized in that the hashian approximation is calculated using the following mathematical formula. Here, However, k represents the gradient step index of the inner loop, and represents the gradient, and However, is, is represented as an interpolation rate that gradually decreases from 1 to 0, and represents the learning rate of the inner loop.