Large language model self-evaluation method and device, electronic equipment and storage medium

Abstract

The invention provides a large language model self-evaluation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining target input which comprises a to-be-answered question and a corresponding answer result; based on a pre-trained large language model, according to the target input, predicting to obtain a self-evaluation result; wherein the large language model is obtained by training and optimizing a training sample set formed by positive samples and negative samples based on a discriminator and a generator, and both the positive samples and the negative samples comprise questions, answers and quality labels. According to the method, supervised training is performed on the large language model according to the positive sample and the negative sample based on the discriminator and the generator, so that the self-evaluation accuracy of the large language model can be effectively improved, the self-evolution of the large language model is realized by fully utilizing the existing data, and the reasoning ability and generalization ability of the large language model are improved.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a large language model self-evaluation method, device, electronic device and storage medium. Background Art

[0002] In recent years, large language models have gradually come into the public eye. They usually refer to deep learning models with billions or more parameters. They perform well in a wide range of tasks rather than a single task and are very powerful.

[0003] However, existing large language models usually rely on preset parameters and fixed structures, lacking the ability to learn and improve themselves. Although model performance can be improved by continuously collecting new data and adjusting training parameters, this method is time-consuming and labor-intensive, and it is difficult to fully improve the generalization ability of the model.

[0004] To solve the above problems, researchers have tried to give large language models the ability to self-evaluate. One method is to imitate traditional conversations and allow the large language model to judge the accuracy of its reasoning results or answers and make self-corrections based on its evaluation. This method is believed to help large language models learn more efficiently and improve their performance.

[0005] However, the self-evaluation of a large language model is not absolutely accurate. Due to its own limitations, a large language model may make incorrect judgments. If it is completely dependent on its self-evaluation for optimization, it may bring unpredictable risks. For example, if the self-evaluation and self-correction of a large language model form a closed feedback loop without external supervision or correction, then this loop may cause the large language model to gradually deviate from the optimal solution and enter a local optimum or even an invalid solution area.

[0006] Therefore, it is necessary to solve the problem that the existing large language models have low self-evaluation accuracy, which leads to poor model performance. Summary of the invention

[0007] The present invention provides a large language model self-evaluation method, device, electronic device and storage medium, which are used to overcome the defects of low accuracy of existing large language model self-evaluation, resulting in poor model performance, effectively improve the self-evaluation accuracy of the large language model, and further promote the self-improvement of the large language model.

[0008] On the one hand, the present invention provides a large language model self-evaluation method, comprising: obtaining a target input, wherein the target input includes a question to be answered and its corresponding answer result; based on a pre-trained large language model, predicting a self-evaluation result according to the target input; wherein the large language model is obtained by training and optimizing a training sample set composed of positive samples and negative samples based on a discriminator and a generator, and the positive samples and negative samples both include questions, answers and quality labels.

[0009] Furthermore, training and optimizing the large language model includes: training and optimizing the discriminator according to the positive samples and negative samples in the training sample set; training and optimizing the generator according to the collected new samples based on the discriminator whose training converges; wherein the generator whose training converges is the pre-trained large language model.

[0010] Furthermore, the discriminator is trained and optimized based on the positive samples and negative samples in the training sample set, including: during each training, the questions and answers in the positive samples or negative samples are input into the discriminator, and the quality evaluation of the positive samples or negative samples is obtained accordingly; the difference between the quality evaluation of the positive samples or negative samples and the corresponding quality labels is used as the training loss, and the discriminator is iteratively optimized to obtain a discriminator with converged training.

[0011] Furthermore, the discriminator based on training convergence trains and optimizes the generator according to the collected new samples, including: inputting the questions and answers in the new samples into the discriminator with training convergence to obtain the confidence probability of the new samples; determining the sample category of the new samples according to the confidence probability of the new samples, and the sample category includes positive samples and negative samples; retaining new samples whose sample category is positive samples, and using the questions in the retained new samples as model inputs, the predicted answers as model outputs, and the difference between the predicted answers and the answers in the retained new samples as training loss, and iteratively optimizing the generator.

[0012] Furthermore, the generator is trained and optimized, which then includes: obtaining test questions and test answers generated by the generator; using the test questions and their corresponding test answers as negative samples, and fine-tuning the discriminator in combination with positive samples not used in the training sample set; based on the fine-tuned discriminator, fine-tuning the generator according to new samples to obtain a final generator; wherein the final generator is the pre-trained large language model.

[0013] Furthermore, the question and answer in the new sample is input into the discriminator trained to convergence to obtain the confidence probability of the new sample, including: extracting the logits value corresponding to the set vocabulary, and applying the Softmax function to obtain the confidence probability corresponding to the set vocabulary; wherein the confidence probability of the set vocabulary is the confidence probability of the new sample; correspondingly, according to the confidence probability of the new sample, the sample category of the new sample is determined, including: when the confidence probability of the set vocabulary is greater than or equal to the set probability, the new sample is determined as a positive sample; when the confidence probability of the set vocabulary is less than the set probability, the new sample is determined as a negative sample.

[0014] Furthermore, the positive samples are obtained by manual annotation by experts, and the negative samples are generated by the large language model itself.

[0015] In a second aspect, the present invention also provides a large language model self-evaluation device, comprising: a target input acquisition module, used to obtain a target input, wherein the target input includes a question to be answered and its corresponding answer result; a model self-evaluation module, used to predict a self-evaluation result based on the target input based on a pre-trained large language model; wherein the large language model is obtained by training and optimizing a training sample set composed of positive samples and negative samples based on a discriminator and a generator, and the positive samples and negative samples both include questions, answers and quality labels.

[0016] In a third aspect, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the large language model self-evaluation method as described above is implemented.

[0017] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the large language model self-evaluation methods described above.

[0018] The large language model self-evaluation method provided by the present invention obtains a target input, which includes a question to be answered and its corresponding answer result, and predicts a self-evaluation result based on the target input based on a pre-trained large language model; wherein the large language model is obtained by training and optimizing a training sample set composed of positive samples and negative samples based on a discriminator and a generator, and both the positive samples and the negative samples include questions, answers and quality labels. The method can effectively improve the self-evaluation accuracy of the large language model by conducting supervised training on the large language model based on positive samples and negative samples based on the discriminator and the generator, thereby making full use of existing data to realize the self-evolution of the large language model and improve the reasoning ability and generalization ability of the large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a flowchart of a large language model self-evaluation method provided by an embodiment of the present invention.

[0021] Figure 2 It is a schematic diagram of training optimization of a large language model provided by an embodiment of the present invention.

[0022] Figure 3 Schematic diagram of training optimization of the discriminator provided by an embodiment of the present invention.

[0023] Figure 4 It is a schematic diagram of training optimization of the generator provided by an embodiment of the present invention.

[0024] Figure 5 It is a structural diagram of a large language model self-evaluation device provided by an embodiment of the present invention.

[0025] Figure 6 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] It should be noted that the existing technology proposes to let the large language model judge the accuracy of its reasoning results or answers by itself, and make self-corrections based on its evaluation. This method is believed to help the large language model learn more efficiently and improve its performance.

[0028] However, the self-evaluation of the large language model is not absolutely accurate. Due to its own limitations, the large language model may make wrong judgments. If you rely entirely on its self-evaluation for optimization, it may bring unpredictable risks. For example, the large language model is "optimized" worse and worse.

[0029] In view of this, the present invention proposes a large language model self-evaluation method, specifically, Figure 1 A schematic flow chart of a large language model self-evaluation method provided in an embodiment of the present invention is shown.

[0030] like Figure 1 As shown, the method includes steps S110-S120, and steps S110-S120 and related steps will be described in detail below.

[0031] S110, obtaining a target input, wherein the target input includes a question to be answered and a corresponding answer result.

[0032] It is understandable that in order to obtain the self-evaluation of the large language model, the target input must be obtained first. Specifically, a specific question to be answered is first clearly defined, such as "Which region of my country is Beijing located in?" This question is clear and answerable; then, the answer result of the question to be answered is obtained by the large language model itself, such as "Beijing is located in the North China region of my country." In this way, the question to be answered and its corresponding answer result are combined to obtain the target input.

[0033] The questions to be answered may be any questions that the user is interested in, and are not specifically limited to humanities or geography.

[0034] Since the present embodiment provides a large language model self-evaluation method, the answer result in this step is obtained by answering the question to be answered by the large language model that performs self-evaluation. That is, the question to be answered is input into the pre-trained large language model in a dialogue manner, and the answer result of the large language model to the question to be answered can be obtained.

[0035] In addition to the questions to be answered and their corresponding answers, the target input may also include some prompt words that allow the big model to self-evaluate the answers to the questions to be answered. The prompt words can be set according to the needs and are not specifically limited here. For example, in a specific embodiment, the prompt words are "Is this answer to the question to be answered good?"

[0036] Based on obtaining the target input in step S110, step S120 is further executed.

[0037] S120, based on the pre-trained large language model, predict the self-evaluation result according to the target input; wherein the large language model is trained and optimized based on the discriminator and the generator through a training sample set consisting of positive samples and negative samples, and the positive samples and negative samples both include questions, answers and quality labels.

[0038] It can be understood that by inputting the target input obtained by combining the question to be answered and its corresponding answer result into a pre-trained large language model, the self-evaluation result of the large language model can be predicted.

[0039] The self-evaluation result of the large language model may be a quality score, such as 93%, or a category label regarding the quality of the answer, such as high, medium, or low, which is not specifically limited here.

[0040] It is worth mentioning that the large language model is trained and optimized based on a pre-built training sample set, which contains positive samples, negative samples, and quality labels. Positive samples refer to those question-answer pairs that are considered to be of high quality, while negative samples refer to those question-answer pairs that are considered to be of low quality, which helps the large language model learn to distinguish between good answers and bad answers. The quality label refers to a category label for the quality of the answer for positive and negative samples, such as high, medium, and low.

[0041] In the process of training and optimizing large language models, supervised learning is used to learn how to correctly evaluate the quality of answers from labeled training sample sets.

[0042] It should be noted that the network structure of the discriminator and the generator mentioned in this embodiment is the same as the large language model, and the discriminator and the generator can be the same large language model as the large language model to be self-evaluated. Of course, the discriminator and the generator can also be two other large language models independent of the large language model to be self-evaluated, and no specific limitation is made here.

[0043] In a specific embodiment, the discriminator and the generator are the same large language model as the large language model to be self-evaluated, and the naming is only for better understanding different training optimization stages. In this setting, there are the following advantages.

[0044] First, since the generator and discriminator share the same model architecture and parameter space, the interaction between them is more consistent. The generator can improve the quality of the answers it generates based on the feedback from the discriminator, and the discriminator can also learn more detailed evaluation criteria from the output of the generator, thus avoiding evaluation bias caused by differences between models.

[0045] Secondly, using the same model as both the generator and the discriminator simplifies model management and maintenance. There is no need to train and deploy multiple models separately, which reduces resource consumption. Since only one model needs to be trained and maintained, there will be significant savings in both computing resources and time costs. This is especially important for large language models, which are very expensive to train.

[0046] Furthermore, training under a unified framework can better control various hyperparameters and training strategies during the training process, making the entire system more stable and efficient. At the same time, since the generator and discriminator are part of the same model, the model already takes the discrimination criteria into account when generating answers, which helps improve the model's generalization ability when facing new data.

[0047] Finally, when the generator and discriminator are the same model, the model can more flexibly adjust its generation strategy to adapt to changing evaluation criteria. This adaptability can help the model better cope with changing data distribution and task requirements.

[0048] In this embodiment, by obtaining the target input, the target input includes the question to be answered and its corresponding answer result, and based on the pre-trained large language model, the self-evaluation result is predicted according to the target input; wherein the large language model is obtained by training and optimizing the training sample set composed of positive samples and negative samples based on the discriminator and the generator, and the positive samples and the negative samples both include questions, answers and quality labels. This method can effectively improve the self-evaluation accuracy of the large language model by conducting supervised training on the large language model based on the discriminator and the generator according to the positive samples and the negative samples, and then make full use of the existing data to realize the self-evolution of the large language model, and improve the reasoning ability and generalization ability of the large language model.

[0049] On the basis of the above embodiments, the supervised training process of a large language model will be described in detail below.

[0050] Training and optimizing a large language model includes: training and optimizing a discriminator based on positive samples and negative samples in a training sample set; training and optimizing a generator based on a discriminator that has converged in training and new samples that have been collected; wherein the generator that has converged in training is a pre-trained large language model.

[0051] It is understandable that the large language model in this embodiment involves alternating training of the discriminator and the generator. Before training optimization, a training sample set needs to be prepared first, and the training sample set includes positive samples and negative samples.

[0052] Among them, positive samples refer to high-quality question-answer pairs, where the answers to the questions are correct, which can be obtained by manual annotation by experts. Negative samples refer to low-quality question-answer pairs, where the answers to the questions may be wrong, irrelevant or incomplete, which can be generated by existing large language models.

[0053] During training optimization, Figure 2 A schematic diagram of training optimization of a large language model provided by an embodiment of the present invention is shown.

[0054] like Figure 2As shown, first, the model is initialized. The initialized large language model can be a pre-trained model or a randomly initialized model, which is not specifically limited here.

[0055] Then, using the positive and negative samples in the training sample set, the discriminator learns to distinguish between high-quality and low-quality answers.

[0056] Next, new samples are collected and screened using the trained discriminator, retaining only high-quality new samples to train and optimize the generator, thereby obtaining a trained generator.

[0057] Finally, the discriminator and generator are fine-tuned a second time to obtain the final generator, that is, the pre-trained large language model.

[0058] It should be noted here that, since the discriminator and the generator in this embodiment are the same large language model, the large language model to be evaluated is first trained and optimized as the discriminator to allow it to learn to distinguish between high-quality and low-quality answers. Then, the large language model to be evaluated is trained and optimized as the generator to allow it to learn to self-evaluate the answers it generates.

[0059] That is to say, the large language model to be evaluated is used to learn different sub-tasks one after the other: one is the task of distinguishing between high-quality answers and low-quality answers, and the other is the task of self-evaluating the generated answers.

[0060] In a specific embodiment, a detailed description is given for the training process of the discriminator.

[0061] Figure 3 A schematic diagram of training optimization of a discriminator provided by an embodiment of the present invention is shown.

[0062] like Figure 3 As shown, according to the positive samples and negative samples in the training sample set, the discriminator is trained and optimized, including: in each training, the questions and answers in the positive samples or negative samples are input into the discriminator, and the quality evaluation of the positive samples or negative samples is obtained accordingly; the difference between the quality evaluation of the positive samples or negative samples and the corresponding quality labels is used as the training loss, and the discriminator is iteratively optimized to obtain a discriminator with training convergence.

[0063] It is easy to understand that during training, each input of the discriminator is either a positive sample or a negative sample. The questions and answers in the positive samples are input into the initialized discriminator to obtain the quality evaluation of the output positive samples, and the questions and answers in the negative samples are input into the initialized discriminator to obtain the quality evaluation of the output negative samples.

[0064] Then, the difference between the quality evaluation of the positive sample and the quality label corresponding to the positive sample, and the difference between the quality evaluation of the negative sample and the quality label corresponding to the negative sample are calculated.

[0065] Next, the discriminator is iteratively optimized and a loss function is defined, such as the mean square error loss function or the binary cross entropy loss function, to measure the difference between the quality evaluation of the positive or negative samples predicted by the discriminator and the corresponding quality label. The back propagation algorithm is used to calculate the loss function relative to the model parameters and a suitable optimizer is selected to update the parameters of the discriminator to minimize the loss function. Then, the training is stopped after repeated iterations until the discriminator reaches a certain convergence standard.

[0066] After the initial training of the discriminator is completed, the generator training begins based on it.

[0067] In another specific embodiment, the training process of the generator is described in detail.

[0068] Figure 4 A schematic diagram of training optimization of the generator provided by an embodiment of the present invention is shown.

[0069] like Figure 4 As shown, based on the discriminator that has converged in training, the generator is trained and optimized according to the collected new samples, including: inputting the questions and answers in the new samples into the discriminator that has converged in training to obtain the confidence probability of the new samples; determining the sample category of the new samples according to the confidence probability of the new samples, and the sample categories include positive samples and negative samples; retaining new samples whose sample categories are positive samples, and using the questions in the retained new samples as model inputs, the predicted answers as model outputs, and the difference between the predicted answers and the answers in the retained new samples as training losses, and iteratively optimizing the generator.

[0070] It is easy to understand that after the discriminator is trained to convergence, a batch of new question-answer pairs are collected as new samples. These new samples can be generated by the generator or obtained from other sources.

[0071] Then, the question and answer in the new sample are input into the discriminator that has been trained to converge, and the confidence probability corresponding to the new sample is determined, that is, the quality score of the answer in the new sample evaluated by the discriminator.

[0072] During the discriminator's judgment process, the logits value corresponding to the set vocabulary is extracted, and the Softmax function is applied to obtain the confidence probability corresponding to the set vocabulary; among them, the confidence probability of the set vocabulary is the confidence probability of the new sample.

[0073] The setting vocabulary here can be set according to actual needs and is not specifically limited here. For example, in a specific embodiment, the setting vocabulary is "high". In another specific embodiment, the setting vocabulary is "low".

[0074] Next, the sample category of the new sample is determined according to the confidence probability of the new sample. Specifically, when the confidence probability of the set vocabulary is greater than or equal to the set probability, the new sample is determined as a positive sample; when the confidence probability of the set vocabulary is less than the set probability, the new sample is determined as a negative sample.

[0075] The threshold value may be set according to actual conditions and is not specifically limited herein. For example, in a specific embodiment, the threshold value is set to 95%.

[0076] After determining the sample category of the new sample, filter out the new samples whose sample category is negative samples, and only retain the positive samples whose sample category is positive samples. These new samples are considered to be of high quality and can be used to further train the generator (that is, the discriminator that has converged).

[0077] Then, the questions in the retained positive samples are used as model input, the predicted answers generated by the generator are used as model output, and the difference between the predicted answers generated by the generator and the answers in the retained new samples is calculated as the training loss. The loss function used in this process can be mean square error or cross entropy loss.

[0078] Finally, the optimizer is used to update the parameters of the generator according to the calculated loss to minimize the loss function. The above steps are repeated until the performance of the generator on new samples reaches the expected level or converges.

[0079] After the initial training of the discriminator and generator, we start to fine-tune the discriminator and generator for the second time.

[0080] In yet another specific embodiment, the secondary fine-tuning process of the discriminator and the generator is described in detail.

[0081] The training and optimization of the generator includes: obtaining the test questions and test answers generated by the generator; using the test questions and their corresponding test answers as negative samples, and fine-tuning the discriminator in combination with the positive samples not used in the training sample set; based on the fine-tuned discriminator, fine-tuning the generator according to the new samples to obtain the final generator; wherein the final generator is a pre-trained large language model.

[0082] It is easy to understand that a batch of new question and answer pairs are generated using the trained generator as test questions and test answers, and these test questions and test answers are used as negative samples. Negative samples are considered to be of lower quality because they are generated results that have not been manually reviewed.

[0083] Then, the discriminator is fine-tuned using the newly generated negative samples from the generator and the positive samples that have not been used in the training sample set to further optimize its ability to distinguish between positive and negative samples. That is, the discriminator is fine-tuned using the positive or negative samples as the model input, the quality evaluation of the positive or negative samples as the model output, and the difference between the quality evaluation of the positive or negative samples and the corresponding quality label as the training loss.

[0084] Finally, based on the fine-tuned discriminator, the generator is fine-tuned according to the new samples to obtain the final generator, which is the pre-trained large language model.

[0085] It should be noted that the configuration of the discriminator and the generator is fine-tuned in this embodiment, which is the same as the process of training and optimizing the discriminator and the generator as described above, and will not be repeated here.

[0086] In this embodiment, the discriminator is trained and optimized according to the positive and negative samples in the training sample set, and the generator is trained and optimized according to the collected new samples based on the discriminator that has converged in training; wherein the generator that has converged in training is a pre-trained large language model, and then based on the pre-trained large language model, the self-evaluation result is predicted according to the target input; wherein the large language model is obtained by training and optimizing the training sample set composed of positive and negative samples based on the discriminator and the generator, and the positive and negative samples both include questions, answers and quality labels. This method continuously improves the generator's ability to generate high-quality answers by alternately training / fine-tuning the discriminator and the generator, and ensures that the discriminator can accurately distinguish between the generated high-quality and low-quality answers. The final generator will be a fully optimized large language model, which can effectively improve the self-evaluation accuracy of the large language model, and then make full use of the existing data to realize the self-evolution of the large language model, and improve the reasoning ability and generalization ability of the large language model.

[0087] In some other embodiments, the pre-trained large language model is applied to various downstream tasks, including but not limited to autonomous driving, robotic operation, etc.

[0088] Corresponding to the large language model self-evaluation method provided in the above embodiments, the present invention also provides a large language model self-evaluation device. Figure 5 A schematic diagram of the structure of a large language model self-evaluation device provided in an embodiment of the present invention is shown.

[0089] like Figure 5As shown, the device includes: a target input acquisition module 510, used to obtain a target input, wherein the target input includes a question to be answered and its corresponding answer result; a model self-evaluation module 520, used to predict a self-evaluation result based on a pre-trained large language model according to the target input; wherein the large language model is obtained by training and optimizing a training sample set composed of positive samples and negative samples based on a discriminator and a generator, and the positive samples and negative samples both include questions, answers and quality labels.

[0090] In this embodiment, the target input is obtained by the target input acquisition module 510, and the target input includes the question to be answered and its corresponding answer result. The model self-evaluation module 520 predicts the self-evaluation result based on the pre-trained large language model according to the target input; wherein the large language model is obtained by training and optimizing the training sample set composed of positive samples and negative samples based on the discriminator and the generator, and the positive samples and the negative samples both include questions, answers and quality labels. The device can effectively improve the self-evaluation accuracy of the large language model by conducting supervised training on the large language model based on the discriminator and the generator according to the positive samples and the negative samples, and then make full use of the existing data to realize the self-evolution of the large language model, and improve the reasoning ability and generalization ability of the large language model.

[0091] It should be noted that the large language model self-evaluation device provided in the embodiment of the present invention and the large language model self-evaluation method described in the above embodiments can be referred to each other, and will not be described in detail here.

[0092] Figure 6 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the large language model self-evaluation method, which includes: obtaining a target input, wherein the target input includes a question to be answered and its corresponding answer result; based on the pre-trained large language model, predicting a self-evaluation result according to the target input; wherein the large language model is obtained by training and optimizing a training sample set composed of positive samples and negative samples based on a discriminator and a generator, wherein the positive samples and negative samples both include questions, answers and quality labels.

[0093] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0094] On the other hand, the present invention also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the large language model self-evaluation method provided by the above methods, and the method includes: obtaining a target input, and the target input includes a question to be answered and its corresponding answer result; based on a pre-trained large language model, predicting a self-evaluation result according to the target input; wherein the large language model is obtained by training and optimizing a training sample set composed of positive samples and negative samples based on a discriminator and a generator, and the positive samples and negative samples both include questions, answers and quality labels.

[0095] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the large language model self-evaluation method provided by the above-mentioned methods, the method comprising: obtaining a target input, the target input comprising a question to be answered and its corresponding answer result; based on a pre-trained large language model, predicting a self-evaluation result according to the target input; wherein the large language model is obtained by training and optimizing a training sample set composed of positive samples and negative samples based on a discriminator and a generator, and the positive samples and negative samples both comprise questions, answers and quality labels.

[0096] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0097] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A large language model self-evaluation method, characterized in that: include: Obtaining a target input, wherein the target input includes a question to be answered and a corresponding answer result; Based on the pre-trained large language model, a self-evaluation result is predicted according to the target input; The large language model is obtained by training and optimizing a training sample set consisting of positive samples and negative samples based on a discriminator and a generator, and both the positive samples and the negative samples include questions, answers and quality labels.

2. The large language model self-evaluation method according to claim 1, characterized in that: Training and optimizing the large language model includes: Training and optimizing the discriminator according to the positive samples and negative samples in the training sample set; Based on the discriminator that has converged in training, the generator is trained and optimized according to the collected new samples; The generator whose training converges is the pre-trained large language model.

3. The large language model self-evaluation method according to claim 2, characterized in that: According to the positive samples and negative samples in the training sample set, training and optimizing the discriminator includes: During each training, the questions and answers in the positive or negative samples are input into the discriminator, and the quality evaluation of the positive or negative samples is obtained accordingly; The difference between the quality evaluation of positive or negative samples and the corresponding quality label is used as the training loss, and the discriminator is iteratively optimized to obtain a discriminator with training convergence.

4. The large language model self-evaluation method according to claim 2, characterized in that: The discriminator based on training convergence, according to the collected new samples, trains and optimizes the generator, including: Input the questions and answers in the new sample into the discriminator that has been trained to converge, and obtain the confidence probability of the new sample; Determine the sample category of the new sample according to the confidence probability of the new sample, where the sample category includes positive samples and negative samples; New samples whose sample categories are positive samples are retained, and the questions in the retained new samples are used as model input, the predicted answers are used as model output, and the difference between the predicted answers and the answers in the retained new samples is used as the training loss, and the generator is iteratively optimized.

5. The large language model self-evaluation method according to claim 2, characterized in that: Training and optimizing the generator then includes: Obtaining test questions and test answers generated by the generator; Using the test questions and their corresponding test answers as negative samples, combined with the positive samples not used in the training sample set, to fine-tune the discriminator; Based on the fine-tuned discriminator, fine-tune the generator according to the new sample to obtain the final generator; The final generator is the pre-trained large language model.

6. The large language model self-evaluation method according to claim 4, characterized in that: The step of inputting the question and answer in the new sample into the discriminator trained to convergence to obtain the confidence probability of the new sample includes: Extract the logits value corresponding to the set vocabulary, and apply the Softmax function to obtain the confidence probability corresponding to the set vocabulary; wherein the confidence probability of the set vocabulary is the confidence probability of the new sample; Correspondingly, determining the sample category of the new sample according to the confidence probability of the new sample includes: When the confidence probability of the set vocabulary is greater than or equal to the set probability, determining the new sample as a positive sample; When the confidence probability of the set vocabulary is less than the set probability, the new sample is determined as a negative sample.

7. The large language model self-evaluation method according to any one of claims 1 to 6, characterized in that: The positive samples are obtained by manual annotation by experts, and the negative samples are generated by the large language model itself.

8. A large language model self-evaluation device, characterized in that: include: A target input acquisition module, used to acquire a target input, wherein the target input includes a question to be answered and a corresponding answer result; A model self-evaluation module, used to predict a self-evaluation result based on the target input based on a pre-trained large language model; The large language model is obtained by training and optimizing a training sample set consisting of positive samples and negative samples based on a discriminator and a generator, and both the positive samples and the negative samples include questions, answers and quality labels.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the large language model self-evaluation method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the large language model self-evaluation method according to any one of claims 1 to 7 is implemented.

Images