Large language model generation stability evaluation method, computer equipment and storage medium

Abstract

The invention provides a big language model generation stability evaluation method, computer equipment and a storage medium. According to the method, original questions are expanded through retouching, a more comprehensive test set is obtained, answers are generated by setting different temperature and top-k super parameters, and a comprehensive evaluated answer is obtained. Then, the semantic generalization of the evaluation is ensured in a manner of calculating the aggregation degree by using vectorization, the stability result of the evaluation is obtained through an aggregation degree formula, and the confidence of the stability result is obtained through a confidence index; compared with the prior art, the generation stability of the large language model can be really improved and evaluated in a wider range.

Description

Technical Field

[0001] The present invention belongs to the technical field of large language models, and in particular relates to a large language model generation stability evaluation method, a computer device and a storage medium. Background Art

[0002] With the rapid development of deep learning technology, large language models (LLM) have become one of the core technologies in the field of natural language processing. Such models, such as GPT-4, BERT, and T5, have mastered rich language knowledge and contextual understanding capabilities through pre-training on large-scale datasets, enabling them to demonstrate excellent performance in tasks such as generating dialogues, answering questions, and text summarization. However, despite the powerful ability of large language models in generating content, their generation stability issues are still of great concern.

[0003] Generative stability refers to the consistency and reliability of the content generated by the model under different inputs, similar inputs, or the same input conditions. In some tasks with deterministic answers, such as answering specific knowledge questions and executing instructions, the results generated by current large language models may show great instability due to the randomness of the model, parameter configuration, or slight differences in input. For example, for the same question, the model may sometimes generate multiple different versions of answers, or provide inaccurate or ambiguous information, resulting in users being unable to obtain consistent and reliable outputs. This problem of generative stability is particularly prominent in scenarios that require accurate answers, such as knowledge question answering, technical support, and medical care.

[0004] At present, the methods for evaluating the generation stability of large language models are mainly based on repeatability testing and consistency index evaluation. A common practice is to use a specific test set input model for multiple generation, record the content consistency between different generation times, and calculate the similarity of the repeated generated content. In addition, the adjustment of generation parameters such as temperature, Top-k and Top-p is also an important part of stability evaluation. By observing the stability performance of the generated content under different parameters, the adaptability of the model in different tasks and scenarios is comprehensively evaluated, but the stability evaluation of these parameters does not have a quantitative value to measure the generation stability. In addition, these methods can reflect the generation consistency of the model to a certain extent, but they cannot fully cover the requirements of multiple tasks, and thus are not adaptable enough to comprehensively evaluate the generation stability under similar content input. Moreover, these methods use the same set of single indicators for evaluation for different task types with different generation stability requirements, which will be biased for some tasks with diverse requirements. Therefore, how to improve and evaluate the generation stability of large language models is still a problem that needs to be solved urgently. Summary of the invention

[0005] In view of the above problems, the present invention provides a large language model generation stability evaluation method, a computer device and a storage medium.

[0006] The method for evaluating the stability of large language model generation provided by the present invention comprises the following steps: D1, collect an evaluation dataset S1 for the objectivity task, namely objective question answering; D2, polishing the sample A in the evaluation data set S1 to obtain a polished data set S2; D3, based on different values ​​of temperature and top-k input parameters of the large model, the large model to be evaluated is used to generate corresponding answers in the polished data set S2 multiple times to form a data set S3; D4. Encode each answer in the data set S3 using a sentence vector to obtain a corresponding vector. Then, for each original question in the polished data set S2, i.e., the sample A, the vectors corresponding to all answers generated by the large model to be evaluated for a (temperature, top-k) parameter pair are taken as a subset Ans_op_emb. D5, traverse each of the subsets Ans_op_emb, calculate the vector aggregation vector_agg for it, and then obtain the generated answer stability evaluation value set S3_r and the confidence set S3_rc of the generated answer stability evaluation value corresponding to the evaluation data set S1; D6. Calculate the mean of the generated answer stability evaluation value set S3_r and the confidence set S3_rc as the stability evaluation value of the large model to be evaluated on the evaluation data set S1, and the confidence of the evaluation value.

[0007] further, In the step D1, the evaluation data set S1 is constructed based on the open source WebQA data set, questions in the WebQA data set are sampled, and an expected number of question data are extracted as the evaluation data set S1 of the objectivity task.

[0008] further, In the step D2, the sample A in the evaluation data set S1 is polished, and the sample B obtained after polishing, i.e., the polished sample, is placed in the same sample C as the sample A before polishing. After polishing, the evaluation data set S1 is converted into the polished data set S2 composed of the sample C, and the polished sample is the polishing problem.

[0009] further, In the step D2, after polishing one of the sample A, n polished samples are output, where n is the expected number of polished samples and is an integer greater than 1.

[0010] further, In the step D3, the values ​​of the parameter temperature of m1 candidates and the values ​​of the parameter top-k of m2 candidates are set, and the corresponding answers to each question in the polished dataset S2 are generated multiple times, and the answers constitute the dataset S3, where m1 and m2 are integers greater than 1. Among them, the Cartesian product combination of the m1 candidate parameter temperature values ​​and the m2 candidate parameter top-k values ​​is performed to obtain m1×m2 (temperature, top-k) parameter pairs, recorded as temperature_topk_pairs. Then, each group of temperature_topk_pairs is respectively configured into the input parameter of the large model to be evaluated, and the large model to be evaluated is called m times to generate answers for each original question and the corresponding polished question. m is the expected number of generated answers, which is an integer greater than 1. Finally, for each original question and the corresponding polished question, each group of temperature_topk_pairs can obtain m×(n+1) corresponding generated answers. Combining each group of temperature_topk_pairs, for each original question and the corresponding polished question, a total of m1×m2×m×(n+1) pieces of data, i.e., answers, are generated.

[0011] further, In the step D4, the specific output form of the subset Ans_op_emb is a vector matrix. After the Cartesian product combination, each original question and its corresponding polished question generate a total of m1×m2 subsets Ans_op_emb.

[0012] further, In the step D5, each subset Ans_op_emb is traversed, and the vector aggregation vector_agg is calculated for it, so as to obtain the set Ans_op_aggs of vector aggregation of the generated answers of the original question corresponding to the subset Ans_op_emb and the corresponding polished question, and then the mean Ans_op_agg_mean and standard deviation Ans_op_agg_std of the set Ans_op_aggs are calculated, and then the standard deviation Ans_op_agg_std is converted into the confidence Ans_op_agg_cfid of the mean Ans_op_agg_mean, wherein the mean Ans_op_agg_mean represents the stability evaluation value of the generated answer to the original question, which is an element in the generated answer stability evaluation value set S3_r, and the confidence Ans_op_agg_cfid is an element in the confidence set S3_rc.

[0013] further, The vector aggregation degree vector_agg satisfies: vector_agg=log(1 / sum(abs(mean(vector)-vector))), In the above formula, vector is the vector matrix to be calculated, that is, the subset Ans_op_emb; mean is the mean calculated by column of the vector matrix, abs is the absolute value, sum is the sum of all elements of the matrix, and log is the logarithm with base 10; The confidence level Ans_op_agg_cfid satisfies: Ans_op_agg_cfid=min(log(1 / Ans_op_agg_std),10) / 10, In the above formula, min(log(x), 10) means that the maximum value of the logarithm log(x) with base 10 is truncated, and the maximum numerator of the logarithm is 10.

[0014] The present invention also provides a computer device, which includes a memory, a first processor, and a first computer program stored in the memory and executable on the first processor. When the first computer program is executed by the first processor, the above-mentioned large language model generation stability evaluation method is implemented.

[0015] The present invention also provides a computer-readable storage medium, which is used to store a second computer program. The second computer program can be executed by at least one second processor so that the at least one second processor executes the above-mentioned large language model generation stability assessment method.

[0016] The present invention expands the original question by polishing it to obtain a more comprehensive test set, and then generates answers by setting different temperature and top-k hyperparameters to obtain a comprehensive evaluation answer. Then, by using vectorized calculation of aggregation, the semantic generalization of the evaluation is guaranteed, the stability result of the evaluation is obtained through the aggregation formula, and the confidence of the stability result is obtained through the confidence index. The generation stability of large language models can be truly improved and evaluated in a wider range than the existing technology.

[0017] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A flow chart of a method for evaluating the stability of large language model generation according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" in the specification and claims of the present application and the above-mentioned drawings and any variations thereof are intended to cover non-exclusive inclusions. The terms "first", "second", "third", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order or a primary and secondary relationship. "Multiple" appearing in this application refers to more than two (including two).

[0022] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0023] Figure 1 This is a flow chart of the method for evaluating the stability of a large language model provided by the present invention. Figure 1 , the large language model generation stability evaluation method includes the following steps.

[0024] D1. Collect an evaluation dataset S1 for the objective task, namely objective question answering. Specifically, this step includes: The evaluation dataset S1 is constructed based on the open source WebQA dataset. The WebQA dataset is a Chinese question-answering dataset open sourced by Baidu. It is essentially a closed-book QA dataset, in which each piece of data, i.e., a sample, contains a question. The WebQA dataset contains 32,223 questions. The questions in the WebQA dataset are sampled, and the expected number of question data, such as 1,000, is extracted as the evaluation dataset S1 for the objective task, i.e., objective question-answering.

[0025] The samples in the evaluation dataset S1, i.e. the original problem examples, are as follows: What is the object that can absorb all seven colors of the sun? Who was the last king of the Shang Dynasty in China? Which sea does the Danube flow into? Which animal was the first sphygmomanometer used to measure? In chronically inflamed tissues, which type of cells are most common: How many players are there on each ice hockey team? In which city of our country is the world's shortest river "Rehe" located? D2. Polish sample A in the evaluation dataset S1 to obtain the polished dataset S2.

[0026] Specifically, sample A in the evaluation data set S1 is polished, and sample B obtained after polishing, i.e., the polished sample, is placed in the same sample C as sample A before polishing. After polishing, the evaluation data set S1 is converted into a polished data set S2 consisting of sample C.

[0027] Polishing is to rewrite a sample according to the goal of maintaining semantic consistency but inconsistent expression. The rewritten sample is recorded as the polished sample, i.e., the polishing problem. Polishing ensures that the semantics of the polished sample is consistent with that of the sample before polishing, but the expression is richer. Among them, after polishing, a sample A outputs n polished samples, where n is the expected number of polished samples, which is an integer greater than 1, such as 10. For example, for the sample "the object that can absorb all seven colors of the sun is", take n=10, and after polishing, 10 polished samples are obtained: "What is the object that can completely absorb the seven colors of sunlight?" "Which object can absorb all seven colors of sunlight?" "Which object can absorb all seven colors of sunlight?", "What is the object that can absorb all seven colors of sunlight?" "What is the object that can completely absorb all colors of sunlight?" "What is the name of an object that can absorb the seven colors of light from the sun?" "What kind of objects can absorb all seven colors of sunlight?", "What is the object that absorbs all seven colors of light from the sun?" "What is the object that can absorb all seven colors of light from the sun?" "What is the object that can absorb the seven colors of sunlight?".

[0028] This step expands the samples or text representation diversity of the original test set, reduces the evaluation bias caused by sample leakage during the training process of the large model, and makes the evaluation results more reliable.

[0029] D3. Based on the values ​​of temperature and top-k of the different large model input parameters, the large model to be evaluated is used multiple times (greater than 1) to generate the corresponding answers in the polished dataset S2 to form dataset S3.

[0030] Among them, based on the polished dataset S2, using the large model to be evaluated, setting the values ​​of the parameter temperature of m1 candidates and the parameter top-k of m2 candidates, repeatedly generating the corresponding answers for each question in the polished dataset S2, these answers constitute the dataset S3, m1 and m2 are integers greater than 1. Among them, temperature is a large model input parameter used to control the diversity of generated samples. The larger the temperature, the higher the diversity of the generated sample text. The value range of temperature is (0, 1). Top-k is a large model input parameter used to control the number of candidate tokens (tags, i.e., the basic unit of input of the large model) generated each time an answer is generated. The larger the value of this parameter, the more diverse the generated samples are. Its value range is [1, total_token_num], where total_token_num is the total number of tokenizers of the large model.

[0031] Specifically, the values ​​of the m1 candidate temperature parameters and the m2 candidate top-k parameters are combined by Cartesian product to obtain m1×m2 (temperature, top-k) parameter pairs, denoted as temperature_topk_pairs. For example, if m1=m2=5, the values ​​of the temperature parameter can be 0.1, 0.3, 0.5, 0.7, 1.0, and the values ​​of the top-k parameter can be 1, 3, 5, 7, 10, respectively. After Cartesian product combination, we get m1×m2=25 temperature pairs. e_topk_pairs is: (0.1, 1), (0.1, 3), (0.1, 5), (0.1, 7), (0.1, 10), (0.3, 1), (0.3, 3), (0.3, 5), (0.3, 7), (0.3, 10), (0.5, 1), (0.5, 3), (0.5, 5), (0.5, 7), (0.5, 10), (0.7, 1), (0.7, 3), (0.7, 5), (0.7, 7), (0.7, 10), (1.0, 1), (1.0, 3), (1.0, 5), (1.0, 7), (1.0, 10). Then, each group of temperature_topk_pairs is configured to the input parameter of the large model to be evaluated, and for each original question, i.e., sample A, and polished question, i.e., polished sample, the large model to be evaluated is called to generate answers m times, where m is the expected number of generated answers, which is an integer greater than 1, such as 5. Finally, for each original question and the corresponding polished question, each temperature_topk_pairs can obtain m×(n+1) corresponding generated answers. Combining each group of temperature_topk_pairs, for each original question and the corresponding polished question, a total of m1×m2×m×(n+1) pieces of data, i.e., answers, are generated. For example, according to step D2 and the example above this step, for the sample "the object that can absorb all the seven colors of the sun is" and the corresponding polished sample, 5×5×5×(10+1)=1375 answers can be obtained. In actual work, m1, m2, m, and n are set by technicians according to working conditions.

[0032] This step generates answers based on different temperature_topk_pairs, which can more comprehensively evaluate the generation stability of the model, that is, taking parameter sensitivity into account. In theory, an objective task should be able to output consistent answers relatively stably regardless of how much the set parameters change.

[0033] D4. Encode each answer in the data set S3 using the sentence vector to obtain the corresponding vector. Then, for each original question in the polished data set S2, the vectors corresponding to all the answers generated by the large model to be evaluated for the corresponding polished question for one temperature_topk_pairs are taken as a subset Ans_op_emb. The specific output form of the subset Ans_op_emb is a vector matrix. After Cartesian product combination, each original question and the polished question corresponding to the original question generate a total of m1×m2 Ans_op_embs.

[0034] D5. Traverse each Ans_op_emb, calculate the vector aggregation vector_agg for it, and then obtain the generated answer stability evaluation value set S3_r corresponding to the evaluation data set S1, and the confidence set S3_rc of the generated answer stability evaluation value.

[0035] Among them, based on step D4, each Ans_op_emb is traversed, and the vector aggregation vector_agg is calculated for it, so as to obtain the set of vector aggregation of the original question corresponding to the Ans_op_emb and the generated answers to the corresponding polished questions, recorded as Ans_op_aggs, and then the mean and standard deviation of the set are calculated, recorded as Ans_op_agg_mean and Ans_op_agg_std respectively, and Ans_op_agg_std is converted into the confidence level Ans_op_agg_cfid of Ans_op_agg_mean, wherein Ans_op_agg_mean represents the stability evaluation value of the generated answer to the original question. The larger the value, the more stable the generated answer, which is stored as an element in the generated answer stability evaluation value set S3_r, or the generated answer stability evaluation value set S3_r is a set composed of the stability evaluation values ​​Ans_op_agg_mean of the generated answers to each original question. The larger the value of Ans_op_agg_cfid is, the higher the confidence of the Ans_op_agg_mean value is. Ans_op_agg_cfid is stored as an element in the confidence set S3_rc, or the confidence set S3_rc is a set consisting of the confidence Ans_op_agg_cfid of the stability evaluation value Ans_op_agg_mean of the generated answer to each original question.

[0036] in, vector_agg=log(1 / sum(abs(mean(vector)-vector))), In the above formula, vector is the vector matrix to be calculated, that is, the Ans_op_emb; mean is the mean calculated by column of the vector matrix (that is, the vector in the vector matrix), abs is the absolute value, sum is the sum of all elements of the matrix, and log is the logarithm with base 10. The larger the value of vector_agg, the higher the aggregation of the calculated vector matrix.

[0037] Ans_op_agg_cfid=min(log(1 / Ans_op_agg_std),10) / 10, In the above formula, min(log(x), 10) means that the maximum value of the logarithm log(x) with base 10 is truncated, and the maximum numerator of the logarithm is 10. Ans_op_agg_cfid is a value between 0 and 1.

[0038] D6. Calculate the mean of the generated answer stability evaluation value set S3_r and the generated answer stability evaluation value confidence set S3_rc as the stability evaluation value of the large model to be evaluated on the evaluation data set S1 of the objectivity task and the confidence of the evaluation value.

[0039] The present invention also provides a computer device, which includes a memory, a first processor, and a first computer program stored in the memory and executable on the first processor. When the first computer program is executed by the first processor, the above-mentioned large language model generation stability evaluation method is implemented.

[0040] The present invention also provides a computer-readable storage medium, which is used to store a second computer program. The second computer program can be executed by at least one second processor so that the at least one second processor executes the above-mentioned large language model generation stability assessment method.

[0041] The present invention expands the original question by polishing it to obtain a more comprehensive test set, and then generates answers by setting different temperature and top-k hyperparameters to obtain a comprehensive evaluation answer. Then, by using vectorized calculation of aggregation, the semantic generalization of the evaluation is guaranteed, the stability result of the evaluation is obtained through the aggregation formula, and the confidence of the stability result is obtained through the confidence index. The generation stability of large language models can be truly improved and evaluated in a wider range than the existing technology.

[0042] Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent substitutions for some of the technical features therein; and these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for evaluating the stability of large language model generation, characterized in that: Includes steps: D1, collect an evaluation dataset S1 for the objectivity task, namely objective question answering; D2, polishing the sample A in the evaluation data set S1 to obtain a polished data set S2; D3, based on different values ​​of temperature and top-k input parameters of the large model, the large model to be evaluated is used to generate corresponding answers in the polished data set S2 multiple times to form a data set S3; D4. Encode each answer in the data set S3 using a sentence vector to obtain a corresponding vector. Then, for each original question in the polished data set S2, i.e., the sample A, the vectors corresponding to all answers generated by the large model to be evaluated for a (temperature, top-k) parameter pair are taken as a subset Ans_op_emb. D5, traverse each of the subsets Ans_op_emb, calculate the vector aggregation vector_agg for it, and then obtain the generated answer stability evaluation value set S3_r and the confidence set S3_rc of the generated answer stability evaluation value corresponding to the evaluation data set S1; D6. Calculate the mean of the generated answer stability evaluation value set S3_r and the confidence set S3_rc as the stability evaluation value of the large model to be evaluated on the evaluation data set S1, and the confidence of the evaluation value.

2. The method for evaluating the stability of large language model generation according to claim 1, characterized in that: In the step D1, the evaluation data set S1 is constructed based on the open source WebQA data set, questions in the WebQA data set are sampled, and an expected number of question data are extracted as the evaluation data set S1 of the objectivity task.

3. The method for evaluating the stability of large language model generation according to claim 1, characterized in that: In the step D2, the sample A in the evaluation data set S1 is polished, and the sample B obtained after polishing, i.e., the polished sample, is placed in the same sample C as the sample A before polishing. After polishing, the evaluation data set S1 is converted into the polished data set S2 composed of the sample C, and the polished sample is the polishing problem.

4. The method for evaluating the stability of large language model generation according to claim 3, characterized in that: In the step D2, after polishing one of the sample A, n polished samples are output, where n is the expected number of polished samples and is an integer greater than 1.

5. The method for evaluating the stability of large language model generation according to claim 4, characterized in that: In the step D3, the values ​​of the parameter temperature of m1 candidates and the values ​​of the parameter top-k of m2 candidates are set, and the corresponding answers to each question in the polished dataset S2 are generated multiple times, and the answers constitute the dataset S3, where m1 and m2 are integers greater than 1. Among them, the Cartesian product combination of the m1 candidate parameter temperature values ​​and the m2 candidate parameter top-k values ​​is performed to obtain m1×m2 (temperature, top-k) parameter pairs, recorded as temperature_topk_pairs. Then, each group of temperature_topk_pairs is respectively configured into the input parameter of the large model to be evaluated, and the large model to be evaluated is called m times to generate answers for each original question and the corresponding polished question. m is the expected number of generated answers, which is an integer greater than 1. Finally, for each original question and the corresponding polished question, each group of temperature_topk_pairs can obtain m×(n+1) corresponding generated answers. Combining each group of temperature_topk_pairs, for each original question and the corresponding polished question, a total of m1×m2×m×(n+1) pieces of data, i.e., answers, are generated.

6. The method for evaluating the stability of large language model generation according to claim 5, characterized in that: In the step D4, the specific output form of the subset Ans_op_emb is a vector matrix. After the Cartesian product combination, each original question and its corresponding polished question generate a total of m1×m2 subsets Ans_op_emb.

7. The method for evaluating the generation stability of a large language model according to claim 1, characterized in that: In the step D5, each subset Ans_op_emb is traversed, and the vector aggregation vector_agg is calculated for it, so as to obtain the set Ans_op_aggs of vector aggregation of the generated answers of the original question corresponding to the subset Ans_op_emb and the corresponding polished question, and then the mean Ans_op_agg_mean and standard deviation Ans_op_agg_std of the set Ans_op_aggs are calculated, and then the standard deviation Ans_op_agg_std is converted into the confidence Ans_op_agg_cfid of the mean Ans_op_agg_mean, wherein the mean Ans_op_agg_mean represents the stability evaluation value of the generated answer to the original question, which is an element in the generated answer stability evaluation value set S3_r, and the confidence Ans_op_agg_cfid is an element in the confidence set S3_rc.

8. The method for evaluating the generation stability of a large language model according to claim 7, characterized in that: The vector aggregation degree vector_agg satisfies: vector_agg=log(1 / sum(abs(mean(vector)-vector))), In the above formula, vector is the vector matrix to be calculated, that is, the subset Ans_op_emb; mean is the mean calculated by column of the vector matrix, abs is the absolute value, sum is the sum of all elements of the matrix, and log is the logarithm with base 10; The confidence level Ans_op_agg_cfid satisfies: Ans_op_agg_cfid=min(log(1 / Ans_op_agg_std),10) / 10, In the above formula, min(log(x), 10) means that the maximum value of the logarithm log(x) with base 10 is truncated, and the maximum numerator of the logarithm is 10.

9. Computer device, characterized in that The method comprises a memory, a first processor, and a first computer program stored in the memory and executable on the first processor, wherein the first computer program, when executed by the first processor, implements the large language model generation stability evaluation method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a second computer program, and the second computer program can be executed by at least one second processor so that the at least one second processor executes the large language model generation stability evaluation method according to any one of claims 1 to 8.

Images