Method and device for testing reliability of upper limit of loss of large model

Abstract

The embodiments of this specification provide a method and device for testing the reliability of the upper bound of the loss of a large model. When both the model parameters and the hyperparameters are updated during the training process of the large model, the reliability of the estimated upper bound of the loss of the large model is tested. Specifically, a special data set is selected to find an evaluation data set with the largest possible loss on the large model, and the evaluation loss on the evaluation data set is determined to test whether the evaluation loss is close enough to the upper bound of the loss, thereby testing whether the upper bound of the loss is reasonable and stable. Among them, the obtained evaluation data set satisfies the divergence that the distance between the update results obtained by updating the same sample under different update rules is greater than the divergence of the first value, and the distance between the update results obtained by updating the same update rule on different samples is greater than the growth of the second multiple of the distance between the parameters of the corresponding samples, so that the evaluation loss is as large as possible. The above technical scheme can test the robustness of the upper bound of the loss of the large model.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of computer technology, and more particularly, to a method and apparatus for verifying the reliability of an upper bound of a large model loss. Background Art

[0002] Large Language Model (LLM), also known as large-scale language model, or large model, usually has a large number of parameters, such as billions. The large language model is a natural language processing model based on deep learning. It can learn the grammar and semantics of natural language, so as to generate human-readable text. Due to its huge corpus, the large language model can be used as a pre-trained model for various language processing scenarios, such as question-and-answer scenarios, push scenarios, generation scenarios, etc. These scenarios involve various application fields, such as question-and-answer scenarios can be in the medical field, academic field, etc.

[0003] In the process of training large models, some hyperparameters are usually involved, such as the learning rate in gradient updates. The selection of hyperparameters affects the efficiency and effect of training models. With the continuous expansion of model scale and the improvement of data quality requirements, the selection of effective hyperparameters may also be related to the generalization ability of the model and the accuracy of capturing complex language phenomena. In order to obtain better hyperparameters, conventional techniques can use grid search, random search, Bayesian optimization, evolutionary algorithms and other optimization algorithm hyperparameters for optimization. Recently, the gradient-based double-layer optimization method for hyperparameters has also made breakthroughs, especially in scenarios such as neural network architecture search. In order to clarify the generalization performance of this algorithm, existing research works use algorithm stability as an analysis tool to establish the upper bound of the algorithm's stability and give the upper bound of its expected generalization error. However, whether the given upper bound of hyperparameters is reliable (or robust) is still an important technical issue that affects the optimization of hyperparameters and thus affects the accuracy of the trained large model in processing text data. Summary of the invention

[0004] One or more embodiments of the present specification describe a method and apparatus for verifying the reliability of the upper bound of large model loss, so as to solve one or more problems mentioned in the background technology.

[0005] According to a first aspect, a method for checking the reliability of an upper bound of a large model loss is provided, wherein the upper bound of the loss is an expected generalization error upper bound, and is used to measure the model performance of the large model; the method comprises: obtaining a selected evaluation data set, wherein the samples in the evaluation data set include texts and satisfy the following properties: the distance between update results obtained by updating the same sample under different update rules is greater than the divergence of a first value, and the distance between update results obtained by updating the same update rule on different samples is greater than the growth of a second multiple of the distance between parameters of the corresponding samples; determining the evaluation loss of the large model to be evaluated under the evaluation data set; comparing the evaluation loss with the upper bound of the loss, so as to determine the reliability of the upper bound of the loss in the large model performance evaluation according to the comparison result.

[0006] In one embodiment, the update rule is a change rule for a single hyperparameter from one state to another state.

[0007] In one embodiment, the upper bound of the expected loss is the maximum value of the expected loss caused by the hyperparameters under the two-layer optimization. The two-layer optimization includes: the inner layer optimization of the large model parameters; the outer layer optimization of the hyperparameters in the multi-model training process, and a single outer layer optimization cycle corresponds to several inner layer optimization cycles.

[0008] In one embodiment, the comparing the measured loss with the upper loss bound comprises: determining a ratio of the measured loss to the upper loss bound as a comparison result.

[0009] In a further embodiment, determining the reliability of the upper loss bound in the large model performance evaluation based on the comparison result includes: when the difference between the ratio and 1 is less than a predetermined threshold, determining that the upper loss bound is reliable in the large model performance evaluation; when the difference between the ratio and 1 is greater than a predetermined threshold, determining that the upper loss bound is not reliable in the large model performance evaluation.

[0010] In one embodiment, the evaluation loss is calculated by raising the first reference term to the Kth power, where K is the number of update cycles of the model parameters. The first reference term is 1 plus the product of the update step size η of the model parameter θ and the expansion coefficient γ′. While satisfying the divergence and the growth, the expansion coefficient γ′ satisfies: the inverse of the product of the difference between the hyperparameters of two different samples is not less than the difference in the hyperparameter gradient values ​​of the two different samples.

[0011] According to a second aspect, a reliability test device for a large model loss upper bound is provided, wherein the loss upper bound is an expected generalization error upper bound, and is used to measure the model performance of the large model; the device comprises:

[0012] An acquisition unit is configured to acquire a selected evaluation data set, wherein the samples in the evaluation data set include texts and satisfy the following properties: the distance between update results obtained by updating the same sample under different update rules is greater than the divergence of a first value, and the distance between update results obtained by updating the same update rule on different samples is greater than the growth of a second multiple of the distance between parameters of the corresponding samples;

[0013] A determination unit, configured to determine the evaluation loss of the large model to be evaluated under the evaluation data set;

[0014] A verification unit is configured to compare the evaluation loss with the loss upper bound to determine the reliability of the loss upper bound in the large model performance evaluation based on the comparison result.

[0015] In one embodiment, the evaluation loss is calculated by the Kth power of the first reference term, where K is the number of update cycles of the model parameter, and the first reference term is 1 plus the product of the update step length η of the model parameter θ and the expansion coefficient γ′. Under the condition of satisfying the divergence and the growth, the expansion coefficient γ′ satisfies: the inverse of the product of the difference between the hyperparameters of two different samples is not less than the difference between the hyperparameter gradient values ​​of the two different samples. .

[0016] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed in a computer, the computer is caused to execute the method of the first aspect.

[0017] According to a fourth aspect, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method of the first aspect is implemented.

[0018] Through the method and device provided in the embodiments of this specification, when both the model parameters and the hyperparameters are updated during the training process of the large model, the upper bound of the loss of the estimated final large model is tested for reliability. The technical concept included therein is: select a special data set to find an evaluation data set with as large a loss as possible after being processed by the large model, and determine the evaluation loss on the evaluation data set, and test whether the evaluation loss is close enough to the upper bound of the loss, thereby testing whether the upper bound of the loss is reasonable and stable. Among them, the obtained evaluation data set satisfies the divergence that the distance between the update results obtained by updating the same sample under different update rules is greater than the divergence of the first value, and the distance between the update results obtained by updating the same update rule on different samples is greater than the growth of the second multiple of the distance between the parameters of the corresponding samples, so that the evaluation loss is as large as possible.

[0019] The above technical scheme can evaluate the upper bound of loss determined for double-layer optimization in conventional technology, and provide an effective evaluation scheme for the reliability (robustness) of the upper bound of loss of large models, which is conducive to better control of the performance of large models and improve the effectiveness of large models in text processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0021] Figure 1 A schematic diagram of a specific implementation architecture under the technical concept of this specification;

[0022] Figure 2 A schematic diagram showing the effect of the update rule on the parameter state;

[0023] Figure 3 A schematic diagram of a process for checking the reliability of the upper bound of the loss of a large model according to an embodiment of this specification is shown;

[0024] Figure 4 A structural block diagram of an apparatus for verifying the reliability of the upper bound of large model loss according to an embodiment of the present specification is shown. DETAILED DESCRIPTION

[0025] The solution provided in this specification is described below in conjunction with the accompanying drawings.

[0026] Generally, a hyperparameter is a parameter in the context of machine learning whose value is set before starting the learning process. The values ​​of these parameters are not learned from the training samples but are predefined. The definition of hyperparameters involves higher-level concepts of the model, such as complexity or learning capacity. The goal of hyperparameter optimization is to select a set of better hyperparameters to improve the performance of the learning algorithm on an independent dataset. Examples of hyperparameters are: the number of neural network layers, the number or depth of trees, the learning rate, the number of clusters in k-means clustering, etc.

[0027] The hyperparameter optimization (HO) algorithm aims to find the best hyperparameters on the validation set (usually associated with the optimized model parameters). Classic HO methods include grid search, random search, Bayesian optimization, and evolutionary algorithms. The two-layer optimization algorithm for hyperparameters can be a model training method that includes two layers of optimization: model parameter update and hyperparameter update.

[0028] An example of two-layer optimization is given below.

[0029] Assume that the model parameter set is denoted as θ, the hyperparameter set is denoted as λ, and the training sample set is denoted as S tr , the validation sample set is denoted as S val , the update step length of the model parameter θ is η, the update step length of the hyperparameter is α, the number of training cycles is T, and the two-layer optimization algorithm can include the optimization of a layer of model parameter set θ and a layer of hyperparameter set λ. Among them, the training sample set S tr Used to optimize the model parameter set θ and verify the sample set S val Used to optimize the hyperparameter set λ. The model parameter set θ and the hyperparameter set λ can be cross-optimized.

[0030] Taking the gradient descent method to update parameters as an example, in a single update cycle of optimizing the model parameter set θ, the training loss is obtained by processing the training sample set by the model, and then the model parameter set θ is updated using the gradient descent method. For example, the update process of the kth cycle is recorded as: Among them, θ k-1 represents the parameters before the current update cycle k, θ k represents the parameters updated in the current update cycle, λ t-1 represents the value of the hyperparameter set λ before the current update cycle t, represents the training set of the current batch, represents the model loss on the training set, Represents the gradient of each model parameter in the parameter set θ, and ηk represents the current step size hyperparameter.

[0031] In gradient-based methods, HO is formulated as a two-level programming problem. The inner layer searches for the best model parameters for the training set given the current hyperparameters. In the outer layer, the hyperparameters are optimized by gradient descent. However, the gradient is difficult to compute because it requires differentiating the optimized model parameters and hyperparameters. Currently, the Jacobian matrix of the inner layer update can be obtained by two mainstream strategies: unrolling differentiation (UD) or approximate application of the implicit function theorem (IFT). Thus the hyperparameter gradient (hyper gradient) is determined:

[0032] Considering the small number of hyperparameters, their update cycle can be inconsistent with the update cycle of the model parameter set θ. For example, the hyperparameter set λ is updated every K model parameter update cycles. Assume that the update cycle of the current optimized hyperparameter is t, and the test set samples of the current batch Determine the gradient of the hyperparameter set λ g for: Further, update the hyperparameter set λ to: t←λ t-1 -α t g.

[0033] For the above two-level optimization algorithm, conventional techniques, such as the paper Stability and generalization of bilevel programming in hyperparameter optimization[J] published by Bao F, Wu G, Li C et al. in Advances in Neural Information Processing Systems, 2021, 34: 4529-4541, evaluate the optimization results of optimizing hyperparameters and model parameters through the upper-level optimization algorithm based on the test set. That is, the performance of the model including the optimized hyperparameters and model parameters is evaluated according to the upper bound of the loss on the test set. Thus, it is proved that the verification loss satisfies the continuity based on the Lipschitz constant, and the smooth parameter γ for all samples in the training set and the test set, the training set loss l tr (λ,·; z) is γ-smooth for all λ∈∧, z∈Z. Let t represent the sample in the test set, T is the total number of samples, then for all 1≤t≤T, in the case of In the case of , there is an expected upper bound on the model loss for a single hyperparameter update cycle Therefore, the consistent stability upper bound of the gradient-based hyperparameter two-level optimization algorithm can give the expected generalization error upper bound (or expected loss upper bound) of the hyperparameters and model parameters optimized in this algorithm on the test set, which is:

[0034]

[0035] The above scheme analyzes the stability error upper bound (hereinafter also referred to as the loss upper bound) of the two-layer optimization algorithm, which is used to measure the performance of the model after optimization by the upper-layer optimization algorithm. Whether the stability error upper bound is reliable (robust) determines whether the evaluation of the model is scientific and reasonable, that is, whether the trained model's processing of actual business data (such as text data processed by large models, etc.) meets business needs. In the case where the loss upper bound is significantly different from the true error upper bound, the stability usability of the loss upper bound for evaluating model performance is relatively poor. Here, the true error upper bound can be understood as the maximum value of the cumulative loss of the trained model on various data sets for a single data set (also called the maximum cumulative loss). For example, the model's accumulated losses on various data sets are 2, 9, 17, 20, 16, etc., and the maximum value is 20. The maximum accumulated loss is 20, and the upper limit of the loss is 21, which is close to the true upper limit of the loss. The reliability of the upper limit of the loss for evaluating model performance is good. On the contrary, if the upper limit of the loss is 100, which is far beyond the true upper limit of the loss, the reliability of the upper limit of the loss for evaluating model performance is poor. In other words, when the upper limit of the loss is 100, if there is a true loss of 98 (or other values ​​close to 100), the upper limit of the loss is stable and reliable. If the maximum true loss is 20 (or other values ​​far away from 100), the upper limit of the loss is not stable and reliable enough.

[0036] In order to test whether the upper limit of loss is reliable, the upper limit of loss can be determined A strict lower limit of the consistency (radius) stability is determined, and the gap between the upper limit of the loss and the lower limit is detected. When the gap does not exceed a predetermined range, the upper limit of the loss can be considered to be tight and reliable, and suitable for evaluating model performance. The strict lower limit here can be understood as the maximum cumulative loss of the model in various data sets. However, in practice, it is difficult to obtain a true strict lower limit. To this end, this specification provides a technical solution for reliability testing of the upper limit of the loss of a large model, which is used to test whether the upper limit of the loss of the model is reliable, that is, whether it is robust. Under the technical concept of this specification, it is intended to find a data set that can make the error between the predicted results and the true results of the model to be evaluated accumulate on each verification sample, and the cumulative loss obtained is as large as possible, to prove that there is a data set, through the cumulative loss of the model to be evaluated (the cumulative loss is less than the upper limit), and the gap with the upper limit of the loss described by formula (1) is within a predetermined range (such as not exceeding a predetermined value).

[0037] It can be understood that the above strict lower limit can be understood as the upper bound of the loss (such as divergence, etc.) of the model to be evaluated in various data sets, or as the lower bound of the upper bound of the loss described above. In the iterative algorithm, the model loss will be recursively accumulated throughout the optimization process. Specifically, this specification can introduce the lower bound extension property, select a data set with as large a loss as possible processed by the model to be evaluated as the evaluation data set, and determine the evaluation loss accumulated by the model to be evaluated on the evaluation data set, and then compare the gap between the evaluation loss and the upper bound of the loss, so as to determine the reliability of the upper bound of the loss in the performance evaluation of large models based on the comparison results.

[0038] Here, the lower bound expansion property is used to define the loss expansion law of the model to be evaluated in the process of processing the evaluation data set. The lower bound expansion property may include but is not limited to one of the following: the divergence of the update rule, the growth of the loss, which is used to characterize the update law that guarantees divergence in each iteration. Then, based on the evaluation data set, sampling is performed in various update rules, and the accumulated loss is calculated when the update rules and sample points are changed, so as to obtain the estimated loss upper bound of the model to be evaluated in various data sets.

[0039] The divergence of the update rule and the growth of the loss are introduced below.

[0040] Assume that ω is a parameter generalization symbol of a model parameter or hyperparameter, defined in space Ω. When ω is a model parameter, Ω is the parameter space of the model. When ω is a hyperparameter, Ω is the hyperparameter space of the model. The update rule is described by a function G. ω is updated from the current state ω t To the next state ω t+1 The update algorithm is denoted as an update rule G t A model parameter update process can include a series of update rules, such as In another update state, some update rules are recorded as Therefore, the output set of parameter ω under the two update rules can be recorded as and

[0041] Essentially, ω t and ω t ′ The difference between them develops dynamically throughout the model parameter update process and is driven by two factors: the difference in the current update rule and the difference in the current parameters caused by the previous update. In order to systematically analyze how the difference between two update sequences leads to huge differences in output, the characteristics of the update rule that leads to increased divergence are characterized by the divergence of the update rule and the growth of the loss.

[0042] The divergence of the update rule is: for any ω∈Ω, from the same starting point, the state update distance of the two update rules G and G′ along the difference direction ν between the two is greater than a distance σ divergence. For example, it can be expressed as:

[0043]

[0044] Among them, ω describes a starting point, that is, the parameter state obtained after processing a certain sample. For the next training sample, the difference between the two update rules G(ω) and G′(ω) exists and is in the direction ν. Then there must be a positive constant σ, so that the modulus of G(ω)-G′(ω) is greater than or equal to σ. The update rule G(ω) can be understood as an update rule starting from the current state ω, such as Figure 3 As shown in , in the case of gradient descent update, the direction pointed by the white arrow indicates the update of the parameter ω under one update rule G, the direction pointed by the black arrow indicates the update of the parameter ω under another update rule G′, and the gray arrow indicates the parameter difference G(ω)-G′(ω) after the two update rules are updated. The length of the gray arrow is the modulus of G(ω)-G′(ω). In the case that there is a direction ν in G(ω)-G′(ω), it is inevitable to find a positive number σ such that the modulus of G(ω)-G′(ω) is greater than or equal to σ.

[0045] The update rule divergence describes the divergence of different update rules under the same parameter state that is greater than or equal to σ, and the loss growth can describe the expansion characteristics of different parameter states under the same update rule. In general, for any two different parameters ω, ω′∈Ω, the growth of the update rule G along the direction ν′ (parallel to ν) satisfies:

[0046]

[0047] That is, under the same update rule G, from ω and ω ′ For updates performed at different starting points, when the update direction G(ω)-G(ω′) exists and is consistent with the direction of the difference between ω and ω′, there must be a positive ratio ρ such that the distance between the update results ||G(ω)-G(ω′)|| is greater than or equal to ρ times the parameter distance ||ω-ω′||.

[0048] It can be seen that the divergence of the update rule describes the divergent properties of the same starting point (corresponding to the same parameter state) under different update rules, and the growth of the loss describes the expansion properties of different starting points under the same update rule. The combination of the two can determine the accumulated model loss under different starting points and different update rules.

[0049] Assume S val and is a twin validation set that differs only in the i-th validation sample entry (e.g., replaced by a random sample), and the twin validation set S val and can be used as an evaluation data set. val and The update rule sequence on is expressed as and According to the technical concept of this specification, the result of determining the lower bound of the reliability of the consistent argument by the hyperparameter distance after T steps depends on the divergence of different update rules on the same sample and the growth of the loss under different samples of the same update rule.

[0050] Assume there exists a non-zero vector v, along the regular sequence and It has the divergence of the update rule (such as the divergence greater than or equal to σ described above) and the loss growth, then for the verification set S val For all samples z in , the model validation loss can have the γ′ expansion property. For example, the evaluation loss (i.e., the model validation loss accumulated on each sample) is expressed as:

[0051]

[0052] The evaluation loss in formula (2) is equivalent to a strict lower bound determination method for the expected loss upper bound of the corresponding data set. In order to be consistent with the upper bound of the loss for easy comparison, σ is written as 2α t Here, γ′>0, which means that for any two parameter values ​​w and w′ parallel to the direction v, the corresponding gradient value difference is greater than or equal to the opposite of γ′ times the difference between w and w′, as expressed as

[0053] Combined with formula (1), for the expected loss ∈ arg You can have:

[0054]

[0055] By comparing the evaluation loss with the loss upper bound in formula (1), the reliability of the loss upper bound in model performance evaluation can be determined based on the comparison result. In practice, the specific values ​​of the evaluation loss and the loss upper bound can be directly calculated and compared.

[0056] Considering the complexity of calculating the loss, in an optional embodiment, the difference between the measured loss and the upper loss bound can be compared. It can be observed that the difference between the lower loss bound (2) and the upper loss bound (1) mentioned above is only in the smoothing or expansion coefficients γ and γ', and the continuity or divergence coefficients L and L'. Therefore, the contrast between the two bounds (i.e., their closeness) depends on the values ​​of these coefficients.

[0057] Therefore, the next step to establish the reliability lower bound is to analyze the expansion coefficient γ′ and the divergence coefficient L′ of the outer optimization during the super gradient update process. From the previous formula, we can see that the inner Jacobian matrix It is the key bridge between the inner and outer levels that affect the super gradient. K (λ) Lischitz coefficient for the outer hyperparameter λ satisfy: In order to provide a guarantee for the super gradient effect, it can be based on The Lipschitz continuity described by , instead of the divergence coefficient L ′, detects the lower bound of its maximum capacity loss ∈ ar3 .

[0058] Given hyperparameters λ and λ′, the corresponding internal update rule is represented as G λ , η and G λ′ , η , K times in a row, giving two inner update rule sequences. For any two hyperparameters λ and λ′, assume that for all training set samples S tr , and μ tr >0, in G λ，η and G λ′，η σ that satisfies ||λ-λ′|| tr Divergence, and the training loss l tr (λ,·;z) satisfies μ tr Expansion, existence by G λ，η and G λ′，η Determine the non-zero vector v, then the Lischitz coefficient of the inner parameter with respect to the outer parameter for:

[0059]

[0060] Omitting only depends on η, σ tr and μ tr The constant of the expansion coefficient γ′ mentioned above is expressed as:

[0061] In one embodiment, in order to induce the instability of the loss, the model loss can be defined by a matrix A. Wherein A can be a symmetric matrix, and the smallest eigenvalue is a negative number, such as denoted as γ 1 , the modulus of the smallest eigenvalue is greater than the modulus of the largest eigenvalue. Then both the validation loss and the training loss can be written as follows: θ represents the model parameters, λ represents the hyperparameter set, x represents the sample feature data, and y represents the model output. represents the matrix transpose. Assume v 1 Yes1 The corresponding unit eigenvector is the twin dataset S that differs only in the i-th sample val and The i-th sample z i Respectively expressed as: i =(x i ,y i )=(v 1 ,1),

[0062] Here, the matrix A determines the convexity of the problem. In this specification, A is indefinite and symmetric. By setting A, the most common non-convex situation can be obtained. The eigenvector corresponding to the smallest eigenvalue (i.e., v1) represents the least convex direction, thus providing the greatest loss scalability, and both the inner and outer optimizations achieve the highest level of divergence and scalability in this direction. Therefore, S val and The different samples in are set to be reverse aligned with v1 to make the HO algorithm unstable.

[0063] The form of the above loss function follows the principle of maximum simplicity. Specifically, the quadratic form Can induce non-convexity, the second bilinear cross term The simplest scenario representing the crossover between hyperparameters and parameters ensures that the inner Jacobian is non-zero, and the last term A connection is provided between the parameters and the input data. For simplicity, the validation loss l val and the training loss l tr can be set to be the same, because the result does not depend on their consistency. In this way, the loss calculation method constructed by A can maximize the verification loss.

[0064] Since the technical concept of this specification is to find a lower bound that is close enough to the upper bound of the expected loss in formula (1), for the twin dataset S obtained in the above manner val and According to one embodiment, in the case of further calculating the corresponding loss, some particularities may be introduced to obtain a loss no less than that described in formula (2).

[0065] As a specific example, assume that a solution based on UD with a constant inner step size η is adopted, where the inner step size η satisfies And the outer parameter step size is α t In the case of , the update rule for the outer hyperparameter update is and is to satisfy the v 1 Direction 2α tThe divergence characteristic in the L′ direction. Moreover, for all validation set samples z, the validation loss satisfies the γ′ expansion property. Considering that both the loss and the expansion coefficient γ′ are exponential, when the exponent K is large enough, both can only consider the highest order, such as: Therefore, the determination of the boundary is transformed into a relative relationship determined by T, K and m.

[0066] Continue to introduce special features. Let the outer layer update step size α t In each step t, it is a small positive constant c, and the expected loss ∈ ar3 The value determined by T is: However, the step size decreases linearly, and when the initial value is c, α t ≤c / t, which requires an additional deformation step. Assume that the inner step size η satisfies 1-ηλ d ≥0, c is a positive constant with variable stability, then based on the expected loss ∈ in formulas (1) and (2) ar3 The upper bound and special case loss upper bound are transformed into:

[0067]

[0068] Since the expected loss ∈ ar3 The upper bound and the lower bound have the same form. When comparing the difference between the evaluation loss and the upper bound of the loss, the values ​​at both ends of equation (3) can be directly compared. According to a specific example, the ratio r can be used as a comparison coefficient to measure the difference between the evaluation loss and the upper bound of the special case loss, then: It can be seen that the closer r is to 1, the closer the evaluation loss is to the upper limit of the loss. Conversely, the closer r is to 0, the farther the evaluation loss is from the upper limit of the loss.

[0069] Based on the above principles, Figure 3 The flowchart of verifying the reliability of the upper limit of the loss of a large model according to an embodiment of this specification is shown. The execution subject of this flowchart can be any computer, device, or server with certain computing power. Figure 3 As shown, the process of checking the reliability of the upper bound of the loss of a large model may include: step 301, obtaining a selected evaluation data set, wherein the samples in the evaluation data set include text and satisfy the following properties: the distance between the update results obtained by updating the same sample under different update rules is greater than the divergence of the first value, and the distance between the update results obtained by updating the same update rule on different samples is greater than the growth of the second multiple of the distance between the parameters of the corresponding samples; step 302, determining the evaluation loss of the large model to be evaluated under the evaluation data set; step 303, comparing the evaluation loss with the upper bound of the loss to determine the reliability of the upper bound of the loss in the performance evaluation of the large model based on the comparison result.

[0070] First, in step 301, a selected evaluation data set is obtained.

[0071] It can be understood that large models are used for natural language processing. For large models, the training set and validation set can be various text data, and the evaluation data set can include various text samples.

[0072] According to the principle in the previous article, the deviation (i.e., loss) between the processing results of the samples in the evaluation data set processed by the large model and the actual results should be as large as possible. In order to achieve this goal, when selecting the evaluation data set, the samples in the evaluation data set can be made to meet the following properties: the distance between the update results obtained by updating the same sample under different update rules is greater than the divergence of the first value σ (also called σ divergence), and the distance between the update results obtained by updating different samples under the same update rule is greater than the growth of the second multiple ρ of the distance between the parameters of the corresponding samples (also called ρ growth).

[0073] According to a specific example, the selected evaluation data set is, for example, the twin validation set S described in the previous principle. val and Among them, the validation set It can be that only in the validation set S val A dataset with different entries in the ith validation sample (e.g., replaced by a random sample). Validation set S" val It can be any dataset.

[0074] Next, through step 302, the evaluation loss of the large model to be evaluated under the evaluation data set is determined.

[0075] The evaluation loss can be determined based on the gap between the processing result (such as predicted text) obtained by the large model to be evaluated processing the text in the evaluation dataset and the corresponding label, and accumulated in each sample.

[0076] According to one embodiment, the evaluation loss can be obtained by processing samples in the evaluation data set through a trained large model and calculating the cumulative loss.

[0077] According to another embodiment, when the update rule of the evaluation data set satisfies the above σ divergence and the verification loss satisfies the above ρ growth, the evaluation loss may be no less than Therefore, it can be As the cumulative evaluation loss. Among them, μ tr is the expansion coefficient (also known as γ′) on the evaluation data set (i.e., validation set), σ tr is the first value of the σ divergence satisfied by the verification data set, η is the update step size of the model parameters, and K is the number of hyperparameter update cycles. trDenoted as the first reference term, considering that the first reference term is a positive number greater than 1, when K is large enough, the constant term can be ignored, such as retaining the Kth power of the first reference term. The expansion coefficient γ′ satisfies: the inverse of the product of the difference between the hyperparameters of two different samples is not less than the difference between the hyperparameter gradient values ​​of the two different samples, that is,

[0078] In other embodiments, the assessment loss may also be determined by other reasonable methods, which are not limited here.

[0079] Then, in step 303, the evaluation loss is compared with the upper loss bound to determine the reliability of the upper loss bound in the large model performance evaluation based on the comparison result.

[0080] The evaluation loss is a model loss determined by using a data set with the largest possible loss as the evaluation data set found under the technical concept of this specification. It can be compared with the upper limit of the loss described in formula (1) as a special case to prove whether there is a model loss that is close enough to the upper limit of the loss.

[0081] In one embodiment, the difference between the evaluation loss and the above loss upper bound (the sign is removed and the value is obtained) can be calculated as a comparison result. At this time, the smaller the difference, the closer the evaluation loss is to the above loss upper bound, and the larger the difference, the further away the evaluation loss is from the above loss upper bound. Optionally, when the difference is less than a predetermined value (such as 1), it is determined that the evaluation loss is close enough to the above loss upper bound, and the above loss upper bound is stable and can be used to measure the performance of the large model. Otherwise, when the difference is greater than a predetermined value, it is determined that the difference between the evaluation loss and the above loss upper bound is large, and the above loss upper bound is unstable for measuring the performance of the large model. The case where the difference is equal to the predetermined value can be set to determine that the evaluation loss is close enough to the above loss upper bound according to actual business needs, or it can be set to determine that the difference between the evaluation loss and the above loss upper bound is large, and there is no limitation here.

[0082] In another embodiment, the ratio of the evaluation loss to the above loss upper bound can be calculated as a comparison result. It can be understood that when the evaluation loss is used as the numerator, the ratio is less than 1, and when the loss upper bound is used as the numerator, the ratio is greater than 1. When the ratio is close enough to 1, such as when the difference between the ratio and 1 is less than a predetermined threshold (such as 0.1), it can be determined that the above loss upper bound is stable and can be used to measure the performance of the large model. Among them, the ratio being close enough to 1 can also be described in other forms, such as when the evaluation loss is used as the numerator, the ratio is not less than 0.9, when the loss upper bound is used as the numerator, the ratio is not greater than 1.1, etc. On the other hand, when the difference between the ratio and 1 is greater than the predetermined threshold, it is determined that the difference between the evaluation loss and the above loss upper bound is large, and the above loss upper bound is used to measure the performance of the large model. The predetermined threshold is a critical value, which can be divided into one of the two situations according to actual needs.

[0083] In an optional embodiment, the above loss upper bound and evaluation loss can be unified in form, and the different parts can be compared to obtain the comparison result, so there is no need to actually calculate the specific loss value. For example, in the form of formula (3), further simplified, using the comparison coefficient As a comparison result of measuring the expected loss and the upper limit of the special case loss. The difference between the comparison result and 1 is used to determine the reliability of the upper limit of the loss. If the difference between the comparison coefficient and 1 is less than a predetermined threshold (such as 0.1), it can be determined that the above upper limit of the loss is stable and can be used to measure the performance of the large model. Otherwise, it is determined that the above upper limit of the loss is unstable.

[0084] Reviewing the above process, the reliability test method of the upper bound of the loss of a large model provided under the technical concept of this specification performs reliability test on the upper bound of the loss of the estimated final large model when both the model parameters and the hyperparameters are updated during the training process of the large model. The technical concept included therein is: select a special data set to find an evaluation data set with the largest possible loss on the large model, and determine the evaluation loss on the evaluation data set, and test whether the evaluation loss is close enough to the upper bound of the loss, so as to test whether the upper bound of the loss is reasonable, reliable, and robust. Among them, the obtained evaluation data set satisfies the divergence that the distance between the update results obtained by updating the same sample under different update rules is greater than the first value, and the distance between the update results obtained by updating the same update rule on different samples is greater than the growth of the second multiple of the distance between the parameters of the corresponding samples, so that the evaluation loss is as large as possible.

[0085] The above technical solution can evaluate the upper bound of loss determined by double-layer optimization in conventional technology, and provide an effective evaluation scheme for the reliability of the upper bound of loss of large models, which is conducive to better control of the performance of large models and improve the effectiveness of large models in text processing. It is worth noting that the above technical solution can also be extended to other model performance evaluation scenarios, such as non-double-layer optimization scenarios, which will not be repeated here.

[0086] According to another embodiment, a device for determining the reliability of the upper bound of large model loss is also provided. The device can be provided on a computer, terminal, or server with a certain computing capability. More specifically, Figure 1 The computing platform shown in the figure. The upper bound of the loss is the upper bound of the expected generalization error, which is used to measure the model performance of large models. Figure 4 An apparatus 400 for checking the reliability of a large model loss upper bound according to one embodiment is shown.

[0087] like Figure 4 As shown, the apparatus 400 may include:

[0088] The acquisition unit 401 is configured to acquire a selected evaluation data set, wherein the samples in the evaluation data set include texts and satisfy the following properties: the distance between the update results obtained by updating the same sample under different update rules is greater than the divergence of the first value, and the distance between the update results obtained by updating the same update rule on different samples is greater than the growth of the second multiple of the distance between the parameters of the corresponding samples;

[0089] A determination unit 402 is configured to determine the evaluation loss of the large model to be evaluated under the evaluation data set;

[0090] The verification unit 403 is configured to compare the evaluation loss with the upper loss bound to determine the reliability of the upper loss bound in the large model performance evaluation based on the comparison result.

[0091] In one embodiment, the verification unit 403 may be further configured to: determine a ratio of the evaluation loss to the upper limit of the loss as a comparison result.

[0092] In a further embodiment, the inspection unit 403 is further configured to: when the difference between the ratio and 1 is less than a predetermined threshold, determine that the upper limit of the loss is reliable in the large model performance evaluation; when the difference between the ratio and 1 is greater than the predetermined threshold, determine that the upper limit of the loss is not reliable in the large model performance evaluation.

[0093] In one embodiment, the evaluation loss is calculated by raising the first reference term to the Kth power, where K is the number of update cycles of the model parameters. The first reference term is 1 plus the product of the update step size η of the model parameter θ and the expansion coefficient γ′. While satisfying the above divergence and the above growth, the expansion coefficient γ′ satisfies: the inverse of the product of the difference between the hyperparameters of two different samples is not less than the difference in the hyperparameter gradient values ​​of the two different samples.

[0094] It is worth mentioning that Figure 4 The device 400 shown is Figure 3 The method described corresponds to Figure 3 The corresponding description in the illustrated method embodiment is also applicable to the device 400 and will not be repeated here.

[0095] According to another embodiment, there is also provided a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute a combination of Figure 3 The method described by et al.

[0096] According to another embodiment of the present invention, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the Figure 3 The method described in the above examples should be appreciated by those skilled in the art that the functions described in the embodiments of this specification can be implemented by hardware, software, firmware, or any combination thereof in one or more of the above examples. When implemented by software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0097] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the technical concept of this specification. It should be understood that the above description is only the specific implementation method of the technical concept of this specification, and is not used to limit the protection scope of the technical concept of this specification. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the embodiments of this specification should be included in the protection scope of the technical concept of this specification.

Claims

1. A method for testing the reliability of a large model loss upper bound, wherein the loss upper bound is an expected generalization error upper bound, and is used to measure the model performance of the large model; the method comprises: Acquire a selected evaluation data set, wherein the samples in the evaluation data set include text and satisfy the following properties: the distance between update results obtained by updating the same sample under different update rules is greater than the divergence of a first value, and the distance between update results obtained by updating the same update rule on different samples is greater than the growth of a second multiple of the distance between parameters of the corresponding samples; Determine the evaluation loss of the large model to be evaluated under the evaluation data set; The evaluation loss is compared with the loss upper bound to determine the reliability of the loss upper bound in the large model performance evaluation based on the comparison result.

2. The method of claim 1, wherein: The update rule is a rule for changing a single hyperparameter from one state to another.

3. The method of claim 1, wherein: The upper bound of the expected loss is the maximum value of the expected loss caused by the hyperparameters under the double-layer optimization. The double-layer optimization includes: the inner layer optimization of the large model parameters; the outer layer optimization of the hyperparameters in the multi-model training process, and a single outer layer optimization cycle corresponds to several inner layer optimization cycles.

4. The method of claim 1, wherein: The comparing the evaluation loss with the loss upper bound comprises: A ratio of the evaluation loss to the upper limit of the loss is determined as a comparison result.

5. The method of claim 4, wherein: Determining the reliability of the upper loss bound in the large model performance evaluation based on the comparison result includes: When the difference between the ratio and 1 is less than a predetermined threshold, it is determined that the upper loss bound is reliable in the large model performance evaluation; When the difference between the ratio and 1 is greater than a predetermined threshold, it is determined that the upper loss bound is not reliable in the large model performance evaluation.

6. The method of claim 1, wherein: The evaluation loss is calculated by raising the first reference term to the Kth power, where K is the number of update cycles of the model parameters. The first reference term is 1 plus the product of the update step length η of the model parameter θ and the expansion coefficient γ′. While satisfying the divergence and the growth, the expansion coefficient γ′ satisfies: the inverse of the product of the difference between the hyperparameters of two different samples is not less than the difference between the hyperparameter gradient values ​​of the two different samples.

7. A device for testing the reliability of the upper bound of the loss of a large model, wherein the upper bound of the loss is the upper bound of the expected generalization error, and is used to measure the model performance of the large model; the device comprises: An acquisition unit is configured to acquire a selected evaluation data set, wherein the samples in the evaluation data set include texts and satisfy the following properties: the distance between update results obtained by updating the same sample under different update rules is greater than the divergence of a first value, and the distance between update results obtained by updating the same update rule on different samples is greater than the growth of a second multiple of the distance between parameters of the corresponding samples; A determination unit, configured to determine the evaluation loss of the large model to be evaluated under the evaluation data set; A verification unit is configured to compare the evaluation loss with the loss upper bound to determine the reliability of the loss upper bound in the large model performance evaluation based on the comparison result.

8. The device according to claim 7, wherein: The evaluation loss is calculated by raising the first reference term to the Kth power, where K is the number of update cycles of the model parameters. The first reference term is 1 plus the product of the update step length η of the model parameter θ and the expansion coefficient γ′. While satisfying the divergence and the growth, the expansion coefficient γ′ satisfies: the inverse of the product of the difference between the hyperparameters of two different samples is not less than the difference between the hyperparameter gradient values ​​of the two different samples.

9. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 6.

10. A computing device comprising a memory and a processor, characterized in that: The memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 6 is implemented.

Images