Replay message generation method and device, storage medium and electronic equipment

Abstract

The invention discloses a reply information generation method and device, a storage medium and electronic equipment, and the method comprises the steps: obtaining a to-be-replied question of a user, inputting a large language model, and obtaining current reply information; and determining to-be-compared reply information based on the historical reply information. And determining a reply difference obtained by comparing the current reply information with the to-be-compared reply information, inputting the reply difference and the to-be-replied question into the large language model, re-determining the current reply information until iteration is stopped, and displaying the current reply information. The reply difference and the question to be replied are input into the large language model again, the large language model is prompted, and the reply difference is concerned, so that the self-confidence condition is reduced, and the accuracy of the reply information output by the large language model is improved. The reply information to be compared belongs to historical reply information, so that the reply information to be compared serves as input of next iteration, stability of the reply information output by the large language model can be improved, overlarge difference of the reply information output by each iteration is avoided, and robustness of the large language model is improved.

Description

Technical Field

[0001] The present invention relates to the field of computers, and more particularly to a method, device, storage medium and electronic device for generating reply information. Background Art

[0002] The large language model obtained through artificial intelligence technology can determine the response information based on various questions raised by users and display it to users. The questions raised by users may include private information.

[0003] The accuracy of the response information determined by the current large language model is low, there is also the problem of overconfidence, and the large language model itself has the problem of poor robustness.

[0004] Based on this, this specification provides a method for generating reply information. Summary of the invention

[0005] The present specification provides a reply information generation method, device, storage medium and electronic device to at least partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions: This specification provides a method for generating a reply message, including: Get the questions to be answered entered by the user; Input the question to be answered into a large language model to obtain current answer information output by the large language model; and obtain a number of historical answer information output by the large language model based on the information to be answered; Determine the response information to be compared among a number of historical response information; Determine the difference result of the current reply according to the current reply information and the reply information to be compared; and determine the current reply information as the historical reply information; Input the current reply difference result and the question to be answered into the large language model, and re-determine the current reply information until a preset stop iteration condition is met; Get the current reply information when the iteration stops and display it.

[0007] Optionally, inputting the question to be answered into a large language model to obtain current answer information output by the large language model specifically includes: Get the preset first prompt information; Based on the preset first prompt information and the question to be answered, resampling is performed to obtain a first resampling result; The first resampling result is input into a large language model to obtain current reply information output by the large language model.

[0008] Optionally, before determining the current reply difference result according to the current reply information and the reply information to be compared, the method further includes: The current reply information is resampled to obtain the resampled current reply information.

[0009] Optionally, determining a current reply difference result according to the current reply information and the reply information to be compared specifically includes: Obtain several historical response difference results to obtain a historical difference set; Determining second prompt information based on the historical difference set; According to the second prompt information, the current reply information is compared with the reply information to be compared to determine the current reply difference result.

[0010] Optionally, according to the second prompt information, comparing the current reply information with the reply information to be compared to determine a current reply difference result specifically includes: According to the second prompt information, the current reply information is compared with the reply information to be compared to determine a feedback result; A current reply difference result is determined according to the feedback result and the historical difference set.

[0011] Optionally, the current reply difference result and the question to be replied are input into the large language model to re-determine the current reply information, specifically including: Determining third prompt information according to the current response difference result; Based on the third prompt information and the question to be answered, resampling is performed to obtain a second resampling result; The second resampling result and the current reply difference result are input into the large language model to re-determine the current reply information.

[0012] Optionally, redefine the current reply information, including: When there is no difference between the current reply information and the reply to be compared, resampling is performed based on the preset first prompt information and the question to be answered to obtain a first resampling result; The first resampling result is input into the large language model to re-determine the current reply information.

[0013] Optionally, the method further comprises: Acquire all reply information generated during the iteration process to obtain a reply information set, wherein the reply information set includes historical reply information, current reply information, and reply information to be compared; Determining a response distribution result of the response information set; According to the reply distribution result, the confidence of the current reply information when stopping iteration is determined and displayed.

[0014] This specification provides a reply information generating device, the device comprising: The acquisition module is used to obtain the questions to be answered input by the user; A reply information determination module is used to input the question to be replied to into a large language model to obtain current reply information output by the large language model; and obtain a number of historical reply information output by the large language model based on the information to be replied to; A comparison information determination module, used to determine the reply information to be compared from a number of historical reply information; A difference determination module, used to determine the difference result of the current reply according to the current reply information and the reply information to be compared; and determine the current reply information as the historical reply information; An iteration module, used for inputting the current reply difference result and the question to be answered into the large language model, and re-determining the current reply information until a preset stop iteration condition is met; The display module is used to obtain and display the current response information when the iteration is stopped.

[0015] Optionally, the reply information determination module is specifically used to obtain a preset first prompt information; based on the preset first prompt information and the question to be answered, resampling is performed to obtain a first resampling result; the first resampling result is input into a large language model to obtain a current reply information output by the large language model.

[0016] Optionally, the device further comprises: The resampling module is used to resample the current reply information to obtain the resampled current reply information.

[0017] Optionally, the difference determination module is specifically used to obtain several historical reply difference results to obtain a historical difference set; based on the historical difference set, determine a second prompt information; according to the second prompt information, compare the current reply information with the reply information to be compared to determine the current reply difference result.

[0018] Optionally, the difference determination module is specifically used to compare the current reply information with the reply information to be compared according to the second prompt information to determine a feedback result; and determine a current reply difference result according to the feedback result and the historical difference set.

[0019] Optionally, the iteration module is specifically used to determine a third prompt information based on the current reply difference result; resample based on the third prompt information and the question to be answered to obtain a second resampling result; input the second resampling result and the current reply difference result into the large language model to re-determine the current reply information.

[0020] Optionally, the iteration module is specifically used to resample based on the preset first prompt information and the question to be answered to obtain a first resampling result when there is no difference between the current reply information and the reply to be compared; input the first resampling result into the large language model to re-determine the current reply information.

[0021] Optionally, the device further comprises: The confidence determination module is used to obtain all the reply information generated during the iteration process to obtain a reply information set, which includes historical reply information, current reply information, and reply information to be compared; determine the reply distribution result of the reply information set; based on the reply distribution result, determine the confidence of the current reply information when stopping the iteration, and display it.

[0022] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned reply information generation method is implemented.

[0023] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned reply information generating method when executing the program.

[0024] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects: In the reply information generation method provided in the present specification, a question to be replied input by a user is obtained, and the question to be replied is input into a large language model to obtain the current reply information output by the large language model. And a number of historical reply information output by the large language model based on the information to be replied is obtained, and the reply information to be compared is determined from the several historical reply information. According to the current reply information and the reply information to be compared, the current reply difference result is determined, and the current reply information is determined as the historical reply information. The current reply difference result and the question to be replied are input into the large language model, and the current reply information is re-determined until the preset stop iteration condition is met, and the current reply information when the iteration is stopped is obtained and displayed.

[0025] It can be seen from the above method that by comparing the historical reply information with the current reply information, the difference in the reply is determined, and the difference and the question to be replied are input into the large language model again to prompt the large language model to pay attention to the difference in the reply, reduce the overconfidence, and improve the accuracy of the reply information output by the large language model. In addition, since the reply information to be compared belongs to the historical reply information, it can also improve the stability of the reply information output by the large language model as the input of the next iteration, avoid the reply information output from each iteration from being too different, and improve the robustness of the large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings: Figure 1 A flowchart of a method for generating a reply message provided in this specification; Figure 2 A schematic diagram of the self-consistent decoding strategy flow provided for this manual; Figure 3 A flowchart of another method for generating reply information provided in this manual; Figure 4 A schematic diagram of the second prompt information provided in this manual; Figure 5 A schematic diagram of the third prompt information provided in this manual; Figure 6 A schematic diagram of a reply information generating device provided in this specification; Figure 7 The corresponding Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0029] Figure 1 A flowchart of a method for generating a reply message provided in this specification specifically includes the following steps: S100: Obtaining the question to be answered input by the user.

[0030] The large language model can generate reply information based on the questions input by the user, using its powerful reasoning ability. The parameter magnitude of the large language model is at least in the hundreds of millions. For example, in the knowledge question and answer scenario, the user inputs several questions and the large language model outputs the answers. For another example, in the mathematical logic reasoning scenario, the user inputs the mathematical formula to be reasoned, and the large language model outputs the reasoning result. However, there are still some problems with the current large language model, including the low accuracy of the output reply information, the problem of overconfidence, and its own poor robustness. It is understandable that overconfidence not only gives wrong reply information, but also has a high confidence in wrong predictions. Therefore, this specification provides a method for generating reply information. The execution subject of this specification can be a computing device that deploys a large language model to generate reply information, such as a server, or a device such as a desktop computer, a laptop computer, etc., and this specification does not limit this. For the sake of ease of description, the following only takes the server as the execution subject to illustrate the reply information generation method provided in this specification.

[0031] The server needs to obtain the user's unanswered question so as to determine the reply information according to the unanswered question. Specifically, in response to the user's input operation, the unanswered question input by the user is obtained.

[0032] S102: Input the question to be answered into a large language model to obtain current answer information output by the large language model; and obtain a number of historical answer information output by the large language model based on the question to be answered.

[0033] In one or more embodiments of the present specification, after obtaining the information to be replied, the server may input the information to be replied into the large language model to obtain the current reply information output by the large language model.

[0034] Generally speaking, the large language model is a trained large language model with reply capabilities. Specifically, whether the large language model is trained can be defined based on standards such as the accuracy of the reply information output by the large language model. For example, if the accuracy of the large language model is 65% or above, the large language model is defined as trained, otherwise, the large language model is considered to be untrained. For large language models with lower accuracy, they can be trained first, and after improving the accuracy to a certain extent, the reply information generation method provided in this manual can be used to further improve the accuracy of the reply information. Of course, the reply information generation method provided in this manual can also be used directly, and during the generation of reply information, the process of training the large language model is not involved.

[0035] In addition, the server may also obtain several historical reply information output by the large language model based on the information to be replied, for subsequent determination of prompt information for the large language model. It is understandable that in each iteration, the server inputs the information to be replied into the large language model, obtains the current reply information, and before entering the next iteration, the current reply information will be determined as the historical reply information. For example, at the nth iteration, the current reply information 1 is obtained, and then, before the n+1th iteration, the current reply information n is determined as the historical reply information n-1.

[0036] It should be noted that, at the nth iteration, the historical reply information may include all reply information generated during the process from the 1st iteration to the n-1th iteration. Then, when the server obtains the historical reply information, it may only obtain a preset number of historical reply information, which can be set as needed, and this manual does not limit this. The server may also determine a number of historical reply information based on preset conditions, and the preset conditions may include that the similarity between the determined historical reply information is higher than a preset threshold. Specifically, feature analysis may be performed on all historical reply information, and for each historical reply information, the similarity between the historical reply information and the remaining historical reply information other than itself may be determined.

[0037] S104: Determine the reply information to be compared from the plurality of historical reply information.

[0038] Specifically, the server can use a voting mechanism to select a reply message with a higher consistency in reply content from a number of historical reply messages as the reply message to be compared. Consistency refers to the consistency of the content in the reply message. For example, a user enters a text and asks to determine the type of animal that appears in the text. If three historical reply messages are obtained, and there are three types of animals in all three reply messages, but the first type of animal is dog, cat, and rabbit, the second type is dog, sheep, and pig, and the third type is dog, sheep, and cow, then, through the voting mechanism, the reply message to be compared can be determined to have three types of animals, namely dog, sheep, and pig.

[0039] Of course, other methods may be used to determine the reply information to be compared from a number of historical reply information, including feature comparison methods, to determine the reply information with higher feature consistency of the historical reply information. Specifically, the feature analysis result of the determined historical reply information in step S104 may be obtained, and the final reply information may be determined based on the feature analysis result.

[0040] Determining historical response information with high consistency as the response information to be compared can ensure the stability of the results output by the large language model. The large language model will not output response information with huge differences due to small changes in subsequent input information, which improves the robustness of the large language model.

[0041] S106: Determine a current reply difference result according to the current reply information and the reply information to be compared; and determine the current reply information as historical reply information.

[0042] Figure 2 The self-consistent decoding strategy flow diagram provided for this manual is as follows: Figure 2 shown.

[0043] For the traditional self-consistent decoding strategy, after the server inputs the question to be answered into the large language model, it obtains n reply information, and uses the voting mechanism to determine the final reply information among the n reply information and display it. When voting, the large language model will focus on the content with high consistency in the reply content, that is, it follows the logic of minority obeys majority, thus ignoring the opinions that are inconsistent with the majority reply information in the decision-making process, causing the large language model to completely ignore the valuable information in the minority opinion, resulting in the problem of overconfidence.

[0044] The present specification also uses a voting mechanism. Therefore, in order to solve the problem of overconfidence while ensuring the robustness of the large language model, the present specification can also determine the current reply difference result based on the current reply information and the reply information to be compared. That is, the current reply information is compared with the reply information to be compared, and the difference between the current reply information and the reply information to be compared is determined to obtain the current difference result. Then, the current difference result is also used as subsequent input, so that the large language model pays attention to the difference information, that is, attaches importance to the valuable information in the minority opinion, thereby eliminating the problem of overconfidence.

[0045] In addition, in order to continuously obtain the current reply information for comparison in subsequent iterations, the server also needs to determine the current reply information as historical reply information.

[0046] S108: Input the current reply difference result and the question to be answered into the large language model, and re-determine the current reply information until a preset stop iteration condition is met.

[0047] In one or more embodiments of the present specification, the server may input the current reply difference result and the question to be answered into the large language model to obtain the output result of the large language model, and re-determine it as the current reply information until a preset stop iteration condition is met. The preset stop iteration condition may be to stop the iteration when the number of iterations reaches a preset number, or to stop the iteration when there is no difference between the current reply information and the reply information to be compared, etc. This specification does not impose any restrictions on this.

[0048] For traditional self-reflection methods, it is difficult for a large language model to identify errors in its own reply information. This is because traditional self-reflection methods only rely on feedback on a single reply information, and it is difficult to find errors hidden in the reply information. It may even generate illusionary modification opinions and ultimately output reply information with low accuracy. This manual obtains several historical reply information through repeated iterations, and determines the reply information to be compared, and compares it with the current reply information, so that the large language model can rely on multiple reply information for reflection, especially relying on the difference between the current reply information and the historical reply information for reflection, so that the large language model can identify possible errors in the reply output by itself, thereby improving the accuracy of the output.

[0049] S110: Obtain the current response information when the iteration is stopped, and display it.

[0050] For example, if the iteration stopping condition is that the iteration number reaches 4, the iteration stops. Then, the reply result output by the large language model includes historical reply information 1 to 3 and current reply information 4. The server can obtain the current reply information 4 and display it to the user.

[0051] based on Figure 1 The reply information generation method shown in the figure compares historical reply information with current reply information to determine the difference in replies, and inputs the difference and the question to be replied into the large language model again to prompt the large language model to pay attention to the difference in replies, thereby reducing overconfidence and improving the accuracy of the reply information output by the large language model. In addition, since the reply information to be compared belongs to historical reply information, it can also improve the stability of the reply information output by the large language model as the input of the next iteration, avoid excessive differences in the reply information output in each iteration, and improve the robustness of the large language model.

[0052] Figure 3 A flowchart of another method for generating a reply message provided in this manual is shown as follows: Figure 3 shown.

[0053] With respect to step S102, when the question to be answered is input into the large language model, the server may also obtain preset first prompt information, which is determined based on the application scenario of the large language model. For example, if the application scenario is a mathematical logic reasoning scenario, the constructed first prompt information needs to include mathematical scenario knowledge to prompt the large language model. For example, the first prompt information may be "please gradually reason out the result obtained by formula x". The existence of the prompt information can further improve the accuracy of the reply information output by the large language model.

[0054] Afterwards, the server may also perform resampling based on the preset first prompt information and the question to be answered to obtain a first resampling result. Resampling is mainly for the purpose of subsequently correcting and optimizing the generated reply information, ensuring that the sequence of the generated reply information is consistent and coherent in terms of syntax, logic and semantics, and also helps to further solve the illusion problem existing in the traditional self-reflection method.

[0055] Then, the server may input the first resampling result into the large language model to obtain the current reply information output by the large language model. Figure 3 In the step S104, based on the first prompt information, the question to be replied is resampled to obtain the first resampling result, and the result is input into the large language model to obtain the current reply information 1. If the current reply information 1 is the reply information obtained in the first iteration, then, since there is no historical reply information in the first iteration, there is no need to determine the reply information to be compared and compare it with the current reply information 1 to obtain the current reply difference result. Execute step S104 again, and the current reply information 2 is obtained in the second iteration. The historical reply information is the current reply information 1. Since the historical reply information only includes the current reply information 1 in the second iteration, the reply information 1 to be compared obtained by voting is still the current reply information 1. When S106 is then executed, the reply information 1 to be compared is compared with the current reply information 2 to obtain the current reply difference result 1. Similarly, in the third iteration, the reply information 2 to be compared is compared with the current reply information 3 to obtain the current reply difference result 2. In the fourth iteration, the reply information 3 to be compared is compared with the current reply information 4 to obtain the current reply difference result 3.

[0056] Before executing step S106, the server may also resample the current reply information to obtain the resampled current reply information. Then, when executing step S106, the server compares the resampled current reply information with the reply information to be compared to determine the current reply difference result.

[0057] With respect to step S106 , the server may also obtain prompt information to prompt the focus of the large language model for comparison during the comparison process, so as to further improve the accuracy of the reply information.

[0058] Specifically, the server can obtain several historical reply difference results, obtain a historical difference set, and determine the second prompt information based on the historical difference set. Specifically, it can be to determine the features in the historical difference set that account for a preset proportion, and determine the second prompt information based on the features. For example, there is always a certain feature in the historical difference set, indicating that there is a problem in the output of the feature by the large language model, then the second prompt information can be this feature, so that the large language model pays attention to whether the current reply information and the reply information to be compared still have differences in this feature when determining the reply difference. If the feature is the number of animal species, that is, in the historical difference set, the server determines that each historical reply difference result includes the number of animal species, indicating that the number of animal species determined by the large language model is always different, then the second prompt information can be "Please pay attention to the number of animal species in the question to be answered."

[0059] Then, the server can compare the current reply information with the reply information to be compared according to the second prompt information to determine the current reply difference result. This specification does not limit the number of historical reply difference results in the historical difference set, and it can be understood that the historical reply difference result is the reply difference result determined in any iteration process before the current iteration.

[0060] Figure 3 In the example, when the current reply information 3 is compared with the reply information 2 to be compared, the current reply difference result 1 points to the current reply difference result 2, indicating that the server can determine the prompt information 2 based on the current reply difference result 1, and compare according to the prompt information 2 to obtain the current reply difference result 2. Similarly, the prompt information 3 can be determined based on at least one of the current reply difference result 1 and the current reply difference result 2. Prompt information 1~3 is the second prompt information in this specification. Since there is no reply difference result in the first iteration, in the second iteration, the prompt information 1 can be a preset prompt template. This specification does not specifically limit the specific content of the preset prompt template, which can be obtained from experience.

[0061] Based on prompt information 1, the server compares reply information 1 to be compared with current reply information 2 to obtain current reply difference result 1. Similarly, based on prompt information 2, the server compares reply information 2 to be compared with current reply information 3 to obtain current reply difference result 2, and based on prompt information 3, the server compares reply information 3 to be compared with current reply information 4 to obtain current reply difference result 3.

[0062] Furthermore, the server may also compare the current reply information with the reply information to be compared according to the second prompt information to determine the feedback result, and determine the current reply difference result according to the feedback result and the historical difference set. In other words, the difference between the current reply information and the reply information to be compared is used as the feedback result, and the feedback result is merged with several historical reply difference results in the historical difference set to obtain a difference fusion result, and the difference fusion result is determined as the current reply difference result.

[0063] The fusion operation can make the current response difference result include the difference in the previous iteration of the current iteration. Then, in the next iteration of the current iteration, the difference in any previous iteration is still retained, so that the large language model can check whether it still has the problem of the previous iteration based on the response difference, further improve the accuracy of the output response information, and solve the problem of overconfidence.

[0064] Figure 4 This is a schematic diagram of the second prompt information provided in this manual, such as Figure 4 shown.

[0065] The specific content of the second prompt is "Given two alternative solutions to a problem, carefully analyze and compare the differences in their reasoning steps, and reflect on: 1) the specific differences in their reasoning steps and final response information; 2) the reasons for these differences.

[0066] Questions to be answered: [Questions to be answered] Two solutions: Solution 1: [Majority Voting] Solution 2: [Current reply information] If there is no difference, stop the iteration. If there is a difference, please describe the difference, identify the error in the response information, and explain the reason. Identify the key suggestions to prevent such errors and combine them with the response difference results to obtain a new response difference result. " Figure 4 The example below takes the case where the iteration is stopped when there is no difference between the current reply information and the reply information to be compared as the preset stop iteration condition. Figure 4 Solution 1 in the majority voting actually refers to the reply information to be compared determined by the voting medium. Feedback refers to executing step S108, that is, inputting the current reply difference result and the question to be answered into the large language model to re-determine the current reply information. It should be noted that when using the second prompt information, it is necessary to Figure 4 The part in the square brackets is replaced with the specific content in the corresponding scenario. For example, if the question to be answered is "Please determine the value of x in the following formula based on the following formula and specific numerical value", the question to be answered in the second prompt information is the content of the aforementioned question to be answered.

[0067] Then, when executing step S108, the server may also determine the third prompt information according to the current reply difference result, including extracting features from the current reply difference result, or directly using the current reply difference result as the third prompt information. Afterwards, based on the third prompt information and the question to be answered, resampling is performed to obtain a second resampling result. Finally, the server may input the second resampling result and the current reply difference result into the large language model to re-determine the current reply information. Figure 3 In the example, the current reply difference result 1 and the current reply difference result 2 are used as the third prompt information.

[0068] Figure 5 The third prompt information diagram provided in this manual is as follows: Figure 5 shown.

[0069] The third prompt message reads "Solve the following problems step by step, start each step with "Step", and make sure each step is separated by "\n\n", and end with "So the reply message is", followed by the reply message.

[0070] Consider integrating previous response variance results into your resolution process.

[0071] Questions to be answered: [Questions to be answered] Reply to message" In addition, if the preset condition for stopping iteration is not that there is no difference between the current reply information and the reply information to be compared, when the iteration is stopped, when there is no difference between the current reply information and the reply to be compared, resampling is performed based on the preset first prompt information and the question to be answered to obtain a first resampling result, and the first resampling result is input into the large language model to re-determine the current reply information.

[0072] It should be further explained that this specification can also calibrate the confidence of the output of the large language model. The confidence calibration of the large language model refers to the reasonable measurement of the uncertainty of the large language model's own output. An ideal large language model should have a reasonable estimate of the accuracy of its own answers, so as to remind users to pay attention to unreliable answers in low-confidence scenarios and improve the output with the help of external resources. It is understandable that in this specification, the large language model can be improved using various prompt information. This method of using prompt information and iteratively improving the output can also be called thought chain reasoning, which simulates the human reasoning process by dividing complex tasks into a series of logical reasoning steps, thereby stimulating the potential reasoning ability of the large language model. This method provides a structured mechanism for using large language models for complex reasoning tasks.

[0073] When calibrating the confidence, the server can obtain all the reply information generated during the iteration process to obtain a reply information set, which includes historical reply information, current reply information, and reply information to be compared. Generally speaking, the server can perform confidence calibration after obtaining the current reply when the iteration is stopped. The reply information set includes several historical reply information, one current reply information, and one reply information to be compared. Confidence calibration can also be performed once during each iteration, and this manual does not limit this.

[0074] Afterwards, the server can determine the reply distribution result of the reply information set, which can be the ratio of each type of feature in the reply information, or the number of identical reply information. For example, there are three reply information, in which the number of animal species is 3, 4, and 3 respectively, and the reply distribution result can be determined to be 66.6%. Since the current reply information in this specification is determined based on the difference in historical reply information, the accuracy of the reply information is also improved. Therefore, the confidence determined based on the reply distribution result is also more accurate, avoiding the occurrence of wrong predictions and giving a higher confidence.

[0075] Finally, the server can determine the confidence of the current reply information when the iteration is stopped according to the reply distribution result, and display it. The reply distribution result can be directly determined as the confidence of the current reply information when the iteration is stopped, or the reply distribution result can be statistically analyzed to obtain the analysis result, and then determined as the confidence of the current reply information when the iteration is stopped. This manual does not limit this.

[0076] Traditional sampling-based confidence calibration methods rely on "independent" repeated sampling and ignore the analysis of differences between response information. In contrast, this manual uses the self-reflection ability of the large language model on the response difference results to improve the confidence calibration ability of the large language. Answers that remain consistent after multiple comparisons and reflections are more reliable, while response information that changes frequently after comparisons and reflections reflects the inherent uncertainty of the large language model. Therefore, the distribution of multiple large language models generated based on iterative reflection is used as a measure of model confidence.

[0077] In summary, this manual combines the self-consistent decoding strategy and the self-reflection ability of the large language model, improves the accuracy of the reasoning decision of the large language model and alleviates the overconfidence phenomenon in the confidence evaluation process of the large language model, thereby making up for the respective limitations of the traditional self-consistent decoding strategy and self-reflection method.

[0078] In order to solve the problem that the large language model completely ignores the valuable information in the minority opinion in traditional self-consistent decoding, this manual iteratively compares the difference between the minority opinion and the majority opinion to find the inconsistency in the resampling process and the potential errors and illusions contained in the inconsistency. On the one hand, it prompts the large language model to pay attention to avoiding potential errors in the subsequent generation process, and on the other hand, it alleviates the phenomenon of overconfidence in confidence estimation by reflecting on the inconsistency.

[0079] In order to solve the problem of insufficient robustness of reflection faced by traditional self-reflection methods, this manual introduces a self-consistent logic of multiple iterations and final voting to increase the robustness of the reflection results. On the other hand, in order to solve the problem that traditional self-reflection methods are difficult to find errors, this manual generates multiple response messages and compares them to find potential errors. This method can effectively solve the problem of overconfidence and lack of robustness of large language models when finding potential errors.

[0080] After comprehensive experimental comparison, the reply information generation method (abbreviated as Mirror-Consistency) provided in this manual has significantly better performance than the traditional self-consistency solution (Self-Consistency). In the experiment, four large language models of different scales and sources were used: gpt-3.5-turbo, qwen-turbo, llama3-8B, and llama3-70B to ensure the universality of the experiment. Four open source data sets were used for evaluation, namely GSM8K, SVAMP, StrategyQA, and Date. The first two are mathematical logic reasoning tasks, and the latter two are multi-hop knowledge question-answering questions. The experimental results show that the reply information generation method provided in this manual improves the accuracy of reasoning in most tasks, especially in tasks involving complex multi-step mathematical reasoning.

[0081] In addition, the performance of the response information generation method provided in this description and the self-consistency scheme on the confidence assessment task is compared. In the experimental results obtained, the calibration curve corresponding to the response information generation method provided in this description is closer to the ideal curve in most cases. Especially when the traditional self-consistency scheme has overconfidence problems, such as when using qwen-turbo as an evaluation model, it further shows that the response information provided in this description can significantly improve the overconfidence problem of large language models.

[0082] The above is a reply information generation method provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding reply information generation device, such as Figure 6 As shown, the device comprises: The acquisition module 600 is used to acquire the question to be answered input by the user; The reply information determination module 602 is used to input the question to be replied to into the large language model, obtain the current reply information output by the large language model; and obtain a number of historical reply information output by the large language model based on the information to be replied to; A comparison information determination module 604 is used to determine the reply information to be compared from a plurality of historical reply information; The difference determination module 606 is used to determine the difference result of the current reply according to the current reply information and the reply information to be compared; and determine the current reply information as the historical reply information; Iteration module 608, used for inputting the current reply difference result and the question to be answered into the large language model, and re-determining the current reply information until a preset stop iteration condition is met; The display module 610 is used to obtain and display the current reply information when the iteration is stopped.

[0083] Optionally, the reply information determination module is specifically used to obtain a preset first prompt information; based on the preset first prompt information and the question to be answered, resampling is performed to obtain a first resampling result; the first resampling result is input into a large language model to obtain a current reply information output by the large language model.

[0084] Optionally, the device further comprises: The resampling module is used to resample the current reply information to obtain the resampled current reply information.

[0085] Optionally, the difference determination module 606 is specifically used to obtain several historical reply difference results to obtain a historical difference set; determine a second prompt information based on the historical difference set; and compare the current reply information with the reply information to be compared according to the second prompt information to determine the current reply difference result.

[0086] Optionally, the difference determination module 606 is specifically used to compare the current reply information with the reply information to be compared according to the second prompt information to determine the feedback result; and determine the current reply difference result according to the feedback result and the historical difference set.

[0087] Optionally, the iteration module 608 is specifically used to determine a third prompt information based on the current reply difference result; resample based on the third prompt information and the question to be answered to obtain a second resampling result; input the second resampling result and the current reply difference result into the large language model to re-determine the current reply information.

[0088] Optionally, the iteration module 608 is specifically used to resample based on the preset first prompt information and the question to be answered to obtain a first resampling result when there is no difference between the current reply information and the reply to be compared; input the first resampling result into the large language model to re-determine the current reply information.

[0089] Optionally, the device further comprises: The confidence determination module is used to obtain all the reply information generated during the iteration process to obtain a reply information set, which includes historical reply information, current reply information, and reply information to be compared; determine the reply distribution result of the reply information set; based on the reply distribution result, determine the confidence of the current reply information when stopping the iteration, and display it.

[0090] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 Provides a method for generating reply information.

[0091] This manual also provides Figure 7 The structural diagram of the electronic device shown in FIG. Figure 7 As shown, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Of course, in addition to the software implementation, this specification does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0092] In the 1990s, it was very clear whether the improvement of a technology was hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with hardware entity modules. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0093] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0094] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0095] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0096] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0097] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0098] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0099] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0100] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0101] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0102] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0103] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0104] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0105] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0106] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0107] The above description is only an embodiment of this specification and is not intended to limit this specification. For those skilled in the art, this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification should be included in the scope of the claims of this application.

Claims

1. A method for generating a reply message, the method comprising: Get the questions to be answered entered by the user; Inputting the question to be answered into a large language model to obtain current answer information output by the large language model; and obtaining a number of historical reply information output by the large language model based on the information to be replied; Determine the response information to be compared among a number of historical response information; Determine the difference result of the current reply according to the current reply information and the reply information to be compared; and determining the current reply information as historical reply information; Input the current reply difference result and the question to be answered into the large language model, and re-determine the current reply information until a preset stop iteration condition is met; Get the current reply information when the iteration stops and display it.

2. The method according to claim 1, inputting the question to be answered into a large language model to obtain current answer information output by the large language model, specifically comprising: Get the preset first prompt information; Based on the preset first prompt information and the question to be answered, resampling is performed to obtain a first resampling result; The first resampling result is input into a large language model to obtain current reply information output by the large language model.

3. The method according to claim 1, before determining the difference result of the current reply according to the current reply information and the reply information to be compared, the method further comprises: The current reply information is resampled to obtain the resampled current reply information.

4. The method according to claim 1, determining the difference result of the current reply according to the current reply information and the reply information to be compared, specifically comprising: Obtain several historical response difference results to obtain a historical difference set; Determining second prompt information based on the historical difference set; According to the second prompt information, the current reply information is compared with the reply information to be compared to determine the current reply difference result.

5. The method according to claim 4, wherein, according to the second prompt information, the current reply information is compared with the reply information to be compared to determine the difference result of the current reply, specifically comprising: According to the second prompt information, the current reply information is compared with the reply information to be compared to determine a feedback result; A current reply difference result is determined according to the feedback result and the historical difference set.

6. The method according to claim 1, inputting the current reply difference result and the question to be replied into the large language model to re-determine the current reply information, specifically comprising: Determining third prompt information according to the current response difference result; Based on the third prompt information and the question to be answered, resampling is performed to obtain a second resampling result; The second resampling result and the current reply difference result are input into the large language model to re-determine the current reply information.

7. The method according to claim 2, re-determining the current reply information, specifically comprising: When there is no difference between the current reply information and the reply to be compared, resampling is performed based on the preset first prompt information and the question to be answered to obtain a first resampling result; The first resampling result is input into the large language model to re-determine the current reply information.

8. The method of claim 1, further comprising: Acquire all reply information generated during the iteration process to obtain a reply information set, wherein the reply information set includes historical reply information, current reply information, and reply information to be compared; Determining a response distribution result of the response information set; According to the reply distribution result, the confidence of the current reply information when stopping iteration is determined and displayed.

9. A reply information generating device, the device comprising: The acquisition module is used to obtain the questions to be answered input by the user; A reply information determination module is used to input the question to be replied to into a large language model to obtain current reply information output by the large language model; and obtain a number of historical reply information output by the large language model based on the information to be replied to; A comparison information determination module, used to determine the reply information to be compared from a number of historical reply information; A difference determination module, used for determining a current reply difference result according to the current reply information and the reply information to be compared; and determining the current reply information as historical reply information; An iteration module, used for inputting the current reply difference result and the question to be answered into the large language model, and re-determining the current reply information until a preset stop iteration condition is met; The display module is used to obtain and display the current response information when the iteration is stopped.

10. In the device as described in claim 9, the reply information determination module is specifically used to obtain a preset first prompt information; based on the preset first prompt information and the question to be answered, resampling is performed to obtain a first resampling result; the first resampling result is input into the large language model to obtain the current reply information output by the large language model.

11. The apparatus of claim 9, further comprising: The resampling module is used to resample the current reply information to obtain the resampled current reply information.

12. In the device as described in claim 9, the difference determination module is specifically used to obtain a number of historical reply difference results to obtain a historical difference set; based on the historical difference set, determine the second prompt information; according to the second prompt information, compare the current reply information with the reply information to be compared to determine the current reply difference result.

13. In the device as described in claim 12, the difference determination module is specifically used to compare the current reply information with the reply information to be compared according to the second prompt information to determine the feedback result; and determine the current reply difference result according to the feedback result and the historical difference set.

14. The device as described in claim 9, wherein the iteration module is specifically used to determine a third prompt information based on the current reply difference result; resample based on the third prompt information and the question to be answered to obtain a second resampling result; input the second resampling result and the current reply difference result into the large language model to re-determine the current reply information.

15. The device as described in claim 10, wherein the iteration module is specifically used to resample based on the preset first prompt information and the question to be answered to obtain a first resampling result when there is no difference between the current reply information and the reply to be compared; and input the first resampling result into the large language model to re-determine the current reply information.

16. The apparatus of claim 9, further comprising: The confidence determination module is used to obtain all the reply information generated during the iteration process to obtain a reply information set, which includes historical reply information, current reply information, and reply information to be compared; determine the reply distribution result of the reply information set; based on the reply distribution result, determine the confidence of the current reply information when stopping the iteration, and display it.

17. A computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

18. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 8 when executing the program.

Images