A method, system, device, and medium for evaluating large model precision and optimization
By generating prompt word templates using a high-precision first-class large model, comparing the reasoning results of the second-class large model token by token, and optimizing the low-precision model by combining similarity and knowledge distillation methods, the problems of high cost and reduced accuracy of traditional evaluation methods are solved, and the service quality of large models is efficiently evaluated and optimized.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GENERATION TIMES TECHNOLOGY (SHANGHAI) CO LTD
- Filing Date
- 2026-03-27
- Publication Date
- 2026-07-31
AI Technical Summary
Traditional large model evaluation methods are costly and time-consuming, and existing optimization methods lead to reduced model accuracy, making it impossible to efficiently evaluate and optimize the service quality of large models.
We use a high-precision first-class large model to generate prompt word templates, compare the reasoning results of the second-class large model with each token, calculate the accuracy, and optimize the low-precision model through similarity methods and knowledge distillation to improve the model accuracy.
The accuracy of large models can be evaluated and improved at low cost. The accuracy of the optimized model is close to 95% of the original model, the inference cost is reduced by three times, and the inference efficiency is improved.
Smart Images
Figure CN122491468A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of model optimization and evaluation technology, specifically relating to a method, system, device, and medium for evaluating the accuracy and optimization of large models. Background Technology
[0002] Model API services encapsulate machine learning models or large language models into application programming interfaces (APIs), allowing external applications to request and access the model's prediction or generation capabilities via network. These services are widely used in fields such as natural language processing and image recognition. For example, large language model APIs provide developers with capabilities such as text generation and document analysis, covering scenarios from prototype development to enterprise-level applications. Specifically, this involves deploying open-source LLM and hybrid multimodal models such as Qwen2-7B, deepseek, and GLM on GPU servers like the 4090 / H20 / L20 using open-source frameworks such as VLLM / Sglang, providing large model inference services to upper layers.
[0003] Traditional large-scale models typically employ autoregressive methods to evaluate service quality. Since the smallest unit of output from these models is a token, each token must be compared against the dataset during evaluation. Therefore, the inference cost equals the sum of all tokens, resulting in significant cost waste and being time-consuming and labor-intensive. Furthermore, based on current inference optimization practices, existing large-scale models often employ model compression methods such as quantization and parameter pruning to reduce computational and storage overhead and improve inference efficiency. However, these methods inevitably lead to a decrease in model accuracy. Summary of the Invention
[0004] To address the above problems, this invention proposes a method, system, device, and medium for evaluating the accuracy and optimization of large models. The technical solution adopted by this invention to solve the above technical problems is as follows:
[0005] A method for evaluating the accuracy and optimization of large models includes the following steps:
[0006] S1, use the first type of large model to generate a prompt word template and the corresponding first inference result for evaluating the service quality of the large model, and input the prompt word template into the second type of large model to be evaluated to obtain the second inference result;
[0007] The inference accuracy of the first type of large model is greater than that of the second type of large model;
[0008] S2, compare the first reasoning result obtained in step S1 with the second reasoning result token by token, obtain the total number of reasoning error results and the set of error tokens corresponding to the prompt word template, and calculate the reasoning accuracy of the second type of large model based on the total number of reasoning error results;
[0009] S3, Determine If yes, proceed to step S4; otherwise, proceed to step S5. This represents the preset model accuracy threshold. This represents the inference accuracy of the second type of large model;
[0010] S4. Input the set of error tokens obtained in step S2 into the first type of large model and the second type of large model respectively to obtain the layer output set based on the network layer. Optimize the second type of large model using similarity method and knowledge distillation method.
[0011] S5 uses the optimized second-class large model to evaluate the service quality of the model.
[0012] Step S2 includes the following steps:
[0013] S2.1, Initialize the number of iterations , Token indicator Empty error token set An empty set of correct tokens ,in, Indicates the first The number of errors in the second inference result corresponding to each prompt word template;
[0014] S2.2, will the first The first inference result corresponding to the prompt word template The token and the corresponding second reasoning result Compare the tokens; if they match, then... Add to the correct token set Execute step S2.5. If there is a discrepancy, change the token. Add to error token set Then, proceed to step S2.3. Indicates the first The second inference result corresponding to the prompt word template. One Token;
[0015] S2.3, will the first Prompt template and correct token set The tokens in the code are combined to form a prompt word. This prompt word is then input into the second type of large model to obtain the first... The third inference result corresponding to the prompt word template;
[0016] S2.4, the third reasoning result obtained in step S2.3 is... The token and its corresponding first inference result. The tokens are compared; if they match... ,Will Add to the correct token set Execute step S2.5, and simultaneously, based on the third inference result obtained in step S2.3, perform the following steps: Update the second inference result corresponding to the prompt word template. If they are inconsistent, return to step S2.3.
[0017] S2.5, Execution ,judge If yes, return to step S2.2; otherwise, proceed. ,judge If so, initialize the Token indicator. Then, return to step S2.2; otherwise, proceed to step S2.6. This represents the total number of tokens for the second reasoning result corresponding to each prompt word template;
[0018] S2.6 Calculate the total number of errors in the second inference results corresponding to all prompt word templates in the prompt word template set, and calculate the inference accuracy of the second type of large model based on the total number of errors and the length of the second inference results.
[0019] The formula for calculating the inference accuracy of the second type of large model is as follows:
[0020] ;
[0021] In the formula, This represents the total number of errors in the second reasoning result. This indicates the length of the second inference result corresponding to all prompt word templates in step S1.
[0022] Step S4 includes the following steps:
[0023] S4.1 Input the error token set obtained in step S2 into the first type of large model and the second type of large model respectively to obtain the corresponding layer output set based on the network layer;
[0024] S4.2, use the similarity method to calculate the similarity between corresponding elements in the layer output set of the first type of large model and the layer output set of the second type of large model obtained in step S4.1 respectively;
[0025] S4.3, use MATLAB to simulate the similarity change function of each network layer obtained in step S4.2;
[0026] S4.4, Based on the similarity change function obtained in step S4.3, obtain the slope corresponding to each network layer, and sort all the slopes in descending order to obtain the slope set;
[0027] S4.5, Concentrate the slopes obtained from step S4.4 of the second type of large model. The network layer parameters corresponding to each element are frozen, and the second type of large model is optimized using knowledge distillation. ,in, Indicates rounding up;
[0028] S4.6 Input the prompt word template obtained in step S1 into the optimized second type of large model to obtain the optimized inference result, return to step S2, and update the second inference result based on the optimized inference result.
[0029] A system for evaluating the accuracy and optimization of large models, comprising:
[0030] First model storage module: used to store the first type of large model, and to generate prompt word templates and corresponding first inference results for evaluating the service quality of the large model using the first type of large model;
[0031] Second model storage module: Used to store the second type of large model, and generate the second inference result based on the prompt word template generated by the first model storage module and the second type of large model;
[0032] Model evaluation module: It is used to compare the first inference result obtained by the first model storage module with the second inference result obtained by the second model storage module token by token, obtain the total number of errors and the set of error tokens corresponding to the prompt word template, and calculate the inference accuracy of the second type of large model based on the total number of errors;
[0033] Model optimization module: used for judgment If so, based on the error token set obtained from the model evaluation module, the first type of large model from the first model storage module, and the second type of large model from the second model storage module, the layer output set based on each network layer is obtained. Then, the second type of large model from the second model storage module is optimized using similarity and knowledge distillation methods. Otherwise, no optimization is needed. This represents the inference accuracy of the second type of large model obtained by the model evaluation module. This represents the preset model accuracy threshold.
[0034] Service Quality Assessment Module: The optimized second-class large model obtained from the model optimization module is used to assess the service quality of the model.
[0035] An electronic device includes a processor and a memory, the memory storing a computer program that, when executed by the processor, performs the steps of the method for evaluating the accuracy and optimization of a large model as described above.
[0036] A computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the method for evaluating the accuracy and optimization of a large model as described above.
[0037] The beneficial effects of this invention are:
[0038] 1. It can evaluate the accuracy of large models at very low cost, and proposes optimization methods to improve the accuracy of large models when the accuracy is low (e.g., when the original model has been optimized by quantization, compression or pruning, which affects the accuracy).
[0039] 2. By comparing and optimizing Qwen2-7B with its low-precision quantized version Qwen2-7B-int4, the accuracy of Qwen2-7B-int4 was brought close to 95% of that of Qwen2-7B. Furthermore, the inference cost (throughput) of Qwen2-7B-int4 was reduced by 3 times compared to Qwen2-7B, effectively reducing the model's inference cost and optimizing inference efficiency. Attached Figure Description
[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0041] Figure 1 This is a schematic diagram of the process of the present invention. Detailed Implementation
[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0043] A method for evaluating the accuracy and optimization of large models, such as Figure 1 As shown, it includes the following steps:
[0044] S1, use the first type of large model to generate a prompt word template and the corresponding first inference result for evaluating the service quality of the large model, and input the prompt word template into the second type of large model to be evaluated to obtain the second inference result;
[0045] The prompt word template can be generated using a high-quality (100%) first-class large model service. The first-class large model can be an LLM large model released by the official open-source platforms such as Qwen2-7B, Qwen3-32B, or Deepseek. Most of these models use Bfloat16 / Float16 / Float32 floating-point types during runtime, and the model accuracy is completely consistent with the official API parameters (determined random seed). Preferably, before generating the prompt word template, the inference decoding strategy parameter TOPK=1 and the randomness control parameter of the first-class large model are set to a fixed random seed to avoid inconsistencies caused by randomness.
[0046] The prompt word template is adopted It is indicated that, to more accurately evaluate the second type of large models, a set of prompt word templates consisting of multiple prompt word templates can also be used for evaluation. This second type of large model can be a model that supports quantization methods such as FP8 / INT8 / AWQ / GPTQ, such as Qwen2-7B-int4 or Qwen3-32B-AWQ-4Bit. The prompt word template set adopts... This means that inputting the set of prompt word templates into the first type of large model yields the corresponding first inference result, and the corresponding expression is:
[0047] (1)
[0048] (2)
[0049] In the formula, This represents a set of prompt word templates. The corresponding first reasoning result, Indicates the first Tip template, This represents a set of prompt word templates. The total number of prompt word templates in the database. Indicates the first The first inference result corresponding to the prompt word template, Indicates the first The first inference result corresponding to the prompt word template, and , Indicates the first The first inference result corresponding to the prompt word template Each Token This represents the total number of tokens for the first inference result corresponding to a prompt word template.
[0050] Set of prompt word templates Inputting the second type of large model yields the second inference result, the corresponding expression of which is:
[0051] (3)
[0052] In the formula, This represents a set of prompt word templates. The corresponding second reasoning result, Indicates the first The second inference result corresponding to the prompt word template, Indicates the first The length of the second inference result corresponding to each prompt word template is [length missing]. , It is a length counting function, and , Indicates the first The second inference result corresponding to the prompt word template. Each Token This represents the total number of tokens for the second inference result corresponding to a prompt word template. In this embodiment, the total number of tokens for the first inference result corresponding to each prompt word template is the same as the total number of tokens for the second inference result corresponding to it, and the total number of tokens for the first inference result corresponding to different prompt word templates and the total number of tokens for the second inference result corresponding to different prompt word templates are also the same.
[0053] S2, compare the first inference result obtained in step S1 with the second inference result token by token, obtain the total number of inference result errors and the set of error tokens corresponding to the prompt word template, and calculate the inference accuracy of the second type of large model based on the total number of inference result errors, including the following steps:
[0054] S2.1, Initialize the number of iterations , Token indicator Empty error token set An empty set of correct tokens ,in, Indicates the first The number of errors in the second inference result corresponding to each prompt word template;
[0055] When comparing the first inference result with the second inference result token by token, a counter can be used to count the number of errors in the inference results of the second type of large model.
[0056] S2.2, will the first The first inference result corresponding to the prompt word template The token and the corresponding second reasoning result Compare the tokens; if they match, then... Add to the correct token set Execute step S2.5. If there is a discrepancy, change the token. Add to error token set Then proceed to step S2.3;
[0057] When the first reasoning result obtained in step S1 is consistent with the second reasoning result, that is... At that time, it represents the token of the second type of large model. The reasoning is accurate, Add to the correct token set The counter increments to 0 when the first reasoning result obtained in step S1 is inconsistent with the second reasoning result. At that time, it represents the token of the second type of large model. Reasoning error, counter incremented by 1. Indicates the first The first inference result corresponding to the prompt word template Each Token Indicates the first The second inference result corresponding to the prompt word template. Each Token Indicates the first The set of correct tokens for the second inference result corresponding to the prompt word template.
[0058] S2.3, will the first Prompt template and correct token set The tokens in the code are combined to form a prompt word. This prompt word is then input into the second type of large model to obtain the first... The third inference result corresponding to the prompt word template;
[0059] S2.4, the third reasoning result obtained in step S2.3 is... The token and its corresponding first inference result. The tokens are compared; if they match... ,Will Add to the correct token set Execute step S2.5, and simultaneously, based on the third inference result obtained in step S2.3, perform the following steps: Update the second inference result corresponding to the prompt word template. If they are inconsistent, return to step S2.3.
[0060] S2.5, Execution ,judge If yes, return to step S2.2; otherwise, proceed. ,judge If so, initialize the Token indicator. Then, return to step S2.2; otherwise, proceed to step S2.6.
[0061] S2.6 Calculate the total number of errors in the second inference results corresponding to all prompt word templates in the prompt word template set, and calculate the inference accuracy of the second type of large model based on the total number of errors and the length of the second inference results;
[0062] The total number of errors in the second reasoning result is adopted. The corresponding calculation formula is:
[0063] ;
[0064] The inference accuracy of the second type of large model is adopted. The corresponding calculation formula is:
[0065] ;
[0066] S3, Determine If yes, proceed to step S4; otherwise, proceed to step S5. This represents the preset model accuracy threshold.
[0067] The model accuracy threshold can be set above 90%, and this application sets it at 95%. When the inference accuracy of the second type of large model falls below the preset model accuracy threshold... At that time, it was determined that the model's accuracy was poor and needed optimization.
[0068] S4, input the error token set obtained in step S2 into the first type of large model and the second type of large model respectively to obtain the layer output set based on the network layers. Optimize the second type of large model using similarity method and knowledge distillation method, including the following steps:
[0069] S4.1 Input the error token set obtained in step S2 into the first type of large model and the second type of large model respectively to obtain the corresponding layer output set based on the network layer;
[0070] The layer output set of the first type of large model adopts This indicates that the corresponding expression is:
[0071] ;
[0072] In the formula, Represents the first type of large model. The output of the layer, This indicates the number of network layers in the first type of large model.
[0073] The layer output set of the second type of large model adopts This indicates that the corresponding expression is:
[0074] ;
[0075] In the formula, Indicates the second type of large model. The output of the layer, This indicates the number of network layers in the second type of large model.
[0076] S4.2, use the similarity method to calculate the similarity of the outputs of the network layers corresponding to the first type of large model and the second type of large model respectively;
[0077] The similarity methods include cosine similarity or mean squared error (MSE), etc. This application calculates the similarity sequentially based on the cosine similarity method. Each corresponding Token and The similarity between each corresponding token, Each corresponding Token and The similarity between each corresponding token, ..., finally yields Step S4.3 involves analyzing the similarity trend of each network layer using the corresponding method.
[0078] S4.3, use MATLAB to simulate the similarity change function of each network layer obtained in step S4.2;
[0079] Since the data flow of the network is from the input layer to the output layer, the accuracy loss also accumulates gradually. Therefore, the similarity between the output of the first layer of the first type of large model and the output of the first layer of the second type of large model is closest to 1. Thus, by simulating the similarity change trend of each layer using tools such as MATLAB, a decreasing function can be obtained.
[0080] S4.4, Based on the similarity change function obtained in step S4.3, obtain the slope corresponding to each network layer, and sort all the slopes in descending order to obtain the slope set;
[0081] The slope describes the degree of precision loss. The larger the slope, the more severe the precision loss in that layer. By filtering by slope, we can get feedback on the severity of precision loss in each layer of the large model, so as to optimize the layers with severe precision loss in a targeted manner, while freezing the parameters of the layers with less severe precision loss.
[0082] S4.5, Concentrate the slopes obtained from step S4.4 of the second type of large model. The network layer parameters corresponding to each element are frozen, and the second type of large model is optimized using knowledge distillation. ,in, Indicates rounding up;
[0083] The knowledge distillation method described in this application is prior art, and will not be described in detail in the embodiments of this application.
[0084] S4.6 Input the prompt word template obtained in step S1 into the optimized second type of large model to obtain the optimized inference result, return to step S2, and update the second inference result based on the optimized inference result;
[0085] S5 uses the optimized second-class large model to evaluate the service quality of the model.
[0086] Through the above optimizations, the accuracy of the Qwen2-7B-int4 model is equivalent to 95.1% of the accuracy of the Qwen2-7B model, reaching the threshold. Therefore, optimization is stopped, and the optimized large model weights are output for API service.
[0087] This application also provides a system for evaluating the accuracy and optimization of large models, including:
[0088] First model storage module: used to store the first type of large model, and to generate prompt word templates and corresponding first inference results for evaluating the service quality of the large model using the first type of large model;
[0089] Second model storage module: Used to store the second type of large model, and generate the second inference result based on the prompt word template generated by the first model storage module and the second type of large model;
[0090] Model evaluation module: It is used to compare the first inference result obtained by the first model storage module with the second inference result obtained by the second model storage module token by token, obtain the total number of errors and the set of error tokens corresponding to the prompt word template, and calculate the inference accuracy of the second type of large model based on the total number of errors;
[0091] Model optimization module: used for judgment If so, based on the error token set obtained from the model evaluation module, the first type of large model from the first model storage module, and the second type of large model from the second model storage module, the layer output set based on each network layer is obtained. The second type of large model from the second model storage module is then optimized using similarity and knowledge distillation methods. Otherwise, the service quality evaluation module is activated. This represents the inference accuracy of the second type of large model obtained by the model evaluation module. This represents the preset model accuracy threshold.
[0092] Service Quality Assessment Module: The optimized second-class large model obtained from the model optimization module is used to assess the service quality of the model.
[0093] This application also provides an electronic device, which includes a processor and a memory. The memory stores a computer program, and when the computer program is executed by the processor, it performs the steps of the above-described method for evaluating the accuracy and optimization of large models.
[0094] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program performs the steps of the method for evaluating the accuracy and optimization of large models described above. Specifically, the storage medium can be a general-purpose storage medium, such as a removable disk or hard disk. When the computer program on the storage medium is run, it can execute embodiments of the method for evaluating the accuracy and optimization of large models described above.
[0095] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for evaluating the accuracy and optimization of large models, characterized in that, Includes the following steps: S1, use the first type of large model to generate a prompt word template and the corresponding first inference result for evaluating the service quality of the large model, and input the prompt word template into the second type of large model to be evaluated to obtain the second inference result; The inference accuracy of the first type of large model is greater than that of the second type of large model; S2, compare the first reasoning result obtained in step S1 with the second reasoning result token by token, obtain the total number of reasoning error results and the set of error tokens corresponding to the prompt word template, and calculate the reasoning accuracy of the second type of large model based on the total number of reasoning error results; S3, Determine If yes, proceed to step S4; otherwise, proceed to step S5. This represents the preset model accuracy threshold. This represents the inference accuracy of the second type of large model; S4. Input the set of error tokens obtained in step S2 into the first type of large model and the second type of large model respectively to obtain the layer output set based on the network layer. Optimize the second type of large model using similarity method and knowledge distillation method. S5 uses the optimized second-class large model to evaluate the service quality of the model.
2. The method for evaluating the accuracy and optimization of large models according to claim 1, characterized in that, Step S2 includes the following steps: S2.1, Initialize the number of iterations , Token indicator Empty error token set An empty set of correct tokens ,in, Indicates the first The number of errors in the second inference result corresponding to each prompt word template; S2.2, will the first The first inference result corresponding to the prompt word template The token and the corresponding second reasoning result Compare the tokens; if they match, then... Add to the correct token set Execute step S2.
5. If there is a discrepancy, change the token. Add to error token set Then, proceed to step S2.
3. Indicates the first The second inference result corresponding to the prompt word template. One Token; S2.3, will the first Prompt template and correct token set The tokens in the code are combined to form a prompt word. This prompt word is then input into the second type of large model to obtain the first... The third inference result corresponding to the prompt word template; S2.4, the third reasoning result obtained in step S2.3 is... The token and its corresponding first inference result. The tokens are compared; if they match... ,Will Add to the correct token set Execute step S2.5, and simultaneously, based on the third inference result obtained in step S2.3, perform the following steps: Update the second inference result corresponding to the prompt word template. If they are inconsistent, return to step S2.
3. S2.5, Execution ,judge If yes, return to step S2.2; otherwise, proceed. ,judge If so, initialize the Token indicator. Then, return to step S2.2; otherwise, proceed to step S2.
6. This represents the total number of tokens for the second reasoning result corresponding to each prompt word template; S2.6 Calculate the total number of errors in the second inference results corresponding to all prompt word templates in the prompt word template set, and calculate the inference accuracy of the second type of large model based on the total number of errors and the length of the second inference results.
3. The method for evaluating the accuracy and optimization of large models according to claim 2, characterized in that, The formula for calculating the inference accuracy of the second type of large model is as follows: ； In the formula, This represents the total number of errors in the second reasoning result. This indicates the length of the second inference result corresponding to all prompt word templates in step S1.
4. The method for evaluating the accuracy and optimization of large models according to claim 1, characterized in that, Step S4 includes the following steps: S4.1 Input the error token set obtained in step S2 into the first type of large model and the second type of large model respectively to obtain the corresponding layer output set based on the network layer; S4.2, use the similarity method to calculate the similarity between corresponding elements in the layer output set of the first type of large model and the layer output set of the second type of large model obtained in step S4.1 respectively; S4.3, use MATLAB to simulate the similarity change function of each network layer obtained in step S4.2; S4.4, Based on the similarity change function obtained in step S4.3, obtain the slope corresponding to each network layer, and sort all the slopes in descending order to obtain the slope set; S4.5, Concentrate the slopes obtained from step S4.4 of the second type of large model. The network layer parameters corresponding to each element are frozen, and the second type of large model is optimized using knowledge distillation. ,in, Indicates rounding up; S4.6 Input the prompt word template obtained in step S1 into the optimized second type of large model to obtain the optimized inference result, return to step S2, and update the second inference result based on the optimized inference result.
5. A system for evaluating the accuracy and optimization of large models, characterized in that, include: First model storage module: used to store the first type of large model, and to generate prompt word templates and corresponding first inference results for evaluating the service quality of the large model using the first type of large model; Second model storage module: Used to store the second type of large model, and generate the second inference result based on the prompt word template generated by the first model storage module and the second type of large model; Model evaluation module: It is used to compare the first inference result obtained by the first model storage module with the second inference result obtained by the second model storage module token by token, obtain the total number of errors and the set of error tokens corresponding to the prompt word template, and calculate the inference accuracy of the second type of large model based on the total number of errors; Model optimization module: used for judgment If so, based on the error token set obtained from the model evaluation module, the first type of large model from the first model storage module, and the second type of large model from the second model storage module, the layer output set based on each network layer is obtained. The second type of large model from the second model storage module is then optimized using similarity and knowledge distillation methods. Otherwise, the service quality evaluation module is activated. This represents the inference accuracy of the second type of large model obtained by the model evaluation module. This represents the preset model accuracy threshold. Service Quality Assessment Module: The optimized second-class large model obtained from the model optimization module is used to assess the service quality of the model.
6. An electronic device comprising a processor and a memory, the memory storing a computer program, characterized in that, When the computer program is executed by a processor, it performs the steps of the method for evaluating the accuracy and optimization of large models as described in any one of claims 1-5.
7. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by a processor, it performs the steps of the method for evaluating the accuracy and optimization of large models as described in any one of claims 1-5.