Response text generation method applied to large language model and related equipment

Abstract

The embodiment of the invention belongs to the technical field of intelligent decision of artificial intelligence, and relates to a reply text generation method applied to a large language model and related equipment, and the method comprises the steps: generating a first reply text according to the large language model; performing confidence evaluation on the first reply text; if the confidence coefficient of the first reply text meets a preset confidence coefficient threshold value, taking the first reply text as the reply content of the question asked by the user; and if the confidence coefficient of the first reply text does not meet a preset confidence coefficient threshold value, constructing a PROMPT template and regenerating a second reply text to reply the question of the user. According to the method, the reply accuracy is improved, different reply strategies are made according to different confidence evaluation results by dynamically adjusting the PROMPT template, the user requirements are met to the maximum extent, in addition, the intelligent level and the user satisfaction degree of the dialogue system are improved, meanwhile, the risk of misunderstanding the reply is reduced, and the user experience is improved. And the overall service quality and efficiency are improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent decision-making technology of artificial intelligence, and in particular to a method for generating reply text applied to a large language model and related equipment. Background Art

[0002] With the rapid development of artificial intelligence technology, intelligent dialogue systems based on large language models have been widely used in multiple scenarios, such as customer service support, personalized recommendations, and automatic question answering. In these applications, the dialogue system built based on the large language model can generate corresponding responses based on the user's questions, thereby achieving the purpose of intelligent dialogue.

[0003] However, the applicant found that traditional reply generation methods generally have certain deficiencies. Specifically, large models tend to generate answers with high confidence, even if these answers may be inaccurate or wrong; due to limitations of training data, the model may not be up-to-date in knowledge in specific fields; the model may have limitations in understanding and generating content involving specific domain knowledge, emerging topics or personalized content; although the model is good at understanding and generating language, it still has deficiencies in understanding complex or deep contexts, etc. It can be seen that traditional reply generation methods have the problem of insufficient understanding ability resulting in inaccurate content of the generated reply text. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a reply text generation method and related equipment applied to a large language model, so as to solve the problem that the traditional reply generation method has insufficient understanding ability resulting in inaccurate content of the generated reply text.

[0005] In order to solve the above technical problems, the embodiment of the present application provides a reply text generation method applied to a large language model, which adopts the following technical solution:

[0006] Receiving a dialogue request carrying question information sent by a user terminal;

[0007] Inputting the question information into a large language model to generate a first answer text;

[0008] Calling N search formula classification models, and performing confidence evaluation operations on the first reply text according to the N search formula classification models, respectively, to obtain N confidence evaluation results, where N is an integer greater than or equal to 1;

[0009] Selecting the best evaluation result with the highest confidence from the N confidence evaluation results, and determining whether the best evaluation result meets a preset confidence threshold;

[0010] If the optimal evaluation result meets the preset confidence threshold, sending the first reply text as the target reply text to the user terminal;

[0011] If the optimal evaluation result does not meet the preset confidence threshold, generating a first PROMPT template according to the optimal evaluation result;

[0012] Inputting the first PROMPT template into the large language model to obtain a second reply text;

[0013] The second reply text is sent to the user terminal as the target reply text.

[0014] Furthermore, the step of sending the second reply text as the target reply text to the user terminal specifically includes the following steps:

[0015] Calculating the similarity between the second reply text and the first reply text according to a semantic similarity algorithm to obtain a similarity score;

[0016] If the similarity score is greater than or equal to a preset similarity threshold, sending the second reply text as the target reply text to the user terminal;

[0017] If the similarity score is less than the preset similarity threshold, regenerate a second PROMPT template according to the optimal evaluation result, and input the second PROMPT template into the large language model until the similarity score between the obtained third reply text and the first reply text is greater than or equal to the preset similarity threshold;

[0018] The third reply text is sent to the user terminal as the target reply text.

[0019] Furthermore, the step of calculating the similarity between the second reply text and the first reply text according to the semantic similarity algorithm to obtain a similarity score specifically includes the following steps:

[0020] Acquire all the words of the second reply text and the first reply text according to natural language processing technology to obtain a second text vocabulary set and a first text vocabulary set;

[0021] Determine the intersection of the second text vocabulary set and the first text vocabulary set according to a set operation algorithm to obtain a vocabulary overlap;

[0022] The similarity score is obtained according to the vocabulary overlap.

[0023] Furthermore, after the step of obtaining all the vocabulary of the second reply text and the first reply text according to the natural language processing technology to obtain the second text vocabulary set and the first text vocabulary set, and before the step of determining the intersection of the second text vocabulary set and the first text vocabulary set according to the set operation algorithm to obtain the vocabulary overlap, the following steps are also included:

[0024] Performing semantic analysis operations on the second text vocabulary set and the first text vocabulary set respectively according to the semantic analysis model to obtain a second semantic vocabulary set and a first semantic vocabulary set;

[0025] The step of determining the intersection of the second text vocabulary set and the first text vocabulary set according to the set operation algorithm to obtain the vocabulary overlap specifically includes the following steps:

[0026] The intersection of the second semantic vocabulary set and the first semantic vocabulary set is determined according to a set operation algorithm to obtain the vocabulary overlap.

[0027] Furthermore, if the similarity score is less than the preset similarity threshold, the second PROMPT template is regenerated according to the optimal evaluation result, and the second PROMPT template is input into the large language model until the similarity score between the third reply text and the first reply text is greater than or equal to the preset similarity threshold, specifically comprising the following steps:

[0028] Furthermore, after the step of sending the second reply text as the target reply text to the user terminal, the following steps are also included:

[0029] A sensitive word library is called, and a sensitive word filtering operation is performed on the second reply text according to the sensitive word library.

[0030] In order to solve the above technical problems, the embodiment of the present application further provides a reply text generation device applied to a large language model, which adopts the following technical solution:

[0031] A request receiving module, used for receiving a dialogue request carrying question information sent by a user terminal;

[0032] A first text generation module, used for inputting the question information into the large language model to generate a first reply text;

[0033] A confidence evaluation module, used to call N search formula classification models, and perform confidence evaluation operations on the first reply text according to the N search formula classification models, respectively, to obtain N confidence evaluation results, where N is an integer greater than or equal to 1;

[0034] A confidence judgment module, used to select the best evaluation result with the highest confidence among the N confidence evaluation results, and judge whether the best evaluation result meets a preset confidence threshold;

[0035] A first text output module, configured to send the first reply text as a target reply text to the user terminal if the optimal evaluation result meets the preset confidence threshold;

[0036] A first template generating module, configured to generate a first PROMPT template according to the optimal evaluation result if the optimal evaluation result does not meet the preset confidence threshold;

[0037] A second text generation module, used for inputting the first PROMPT template into the large language model to obtain a second reply text;

[0038] The second text output module is used to send the second reply text as the target reply text to the user terminal.

[0039] Furthermore, the second text output module includes:

[0040] A similarity calculation submodule, used to calculate the similarity between the second reply text and the first reply text according to a semantic similarity algorithm to obtain a similarity score;

[0041] A target text confirmation submodule, configured to send the second reply text as the target reply text to the user terminal if the similarity score is greater than or equal to a preset similarity threshold;

[0042] A template generation submodule, for regenerating a second PROMPT template according to the optimal evaluation result if the similarity score is less than the preset similarity threshold, and inputting the second PROMPT template into the large language model until the similarity score between the obtained third reply text and the first reply text is greater than or equal to the preset similarity threshold;

[0043] The second text output submodule is used to send the third reply text as the target reply text to the user terminal.

[0044] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0045] It includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the reply text generation method applied to a large language model as described above are implemented.

[0046] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0047] The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the reply text generation method applied to a large language model as described above.

[0048] The present application provides a reply text generation method applied to a large language model, comprising: receiving a dialogue request carrying question information sent by a user terminal; inputting the question information into the large language model to generate a first reply text; calling N retrieval classification models, and performing confidence assessment operations on the first reply text according to the N retrieval classification models, respectively, to obtain N confidence assessment results, wherein N is an integer greater than or equal to 1; screening the optimal assessment result with the highest confidence among the N confidence assessment results, and judging whether the optimal assessment result meets a preset confidence threshold; if the optimal assessment result meets the preset confidence threshold, sending the first reply text as the target reply text to the user terminal; if the optimal assessment result does not meet the preset confidence threshold, generating a first PROMPT template according to the optimal assessment result; inputting the first PROMPT template into the large language model to obtain a second reply text; and sending the second reply text as the target reply text to the user terminal. Compared with the existing technology, this application not only improves the accuracy of replies, but also dynamically adjusts the PROMPT template to formulate different reply strategies for different confidence assessment results to maximize user needs. In addition, it also improves the intelligence level of the dialogue system and user satisfaction, while reducing the risk of misunderstanding replies and improving the overall service quality and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0051] Figure 2 It is a flowchart of the implementation of the reply text generation method applied to a large language model provided in the first embodiment of the present application;

[0052] Figure 3It is a structural diagram of a reply text generation device applied to a large language model provided in Embodiment 2 of the present application;

[0053] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0055] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0056] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0057] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0058] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0059] The terminal devices 101, 102, and 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, desktop computers, etc.

[0060] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0061] It should be noted that the reply text generation method applied to a large language model provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the reply text generation device applied to a large language model is generally set in the server / terminal device.

[0062] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0063] Embodiment 1

[0064] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for generating a reply text applied to a large language model according to the present application. The method for generating a reply text applied to a large language model comprises: step S201, step S202, step S203, step S204, step S205, step S206, step S207 and step S208.

[0065] In step S201, a dialogue request carrying question information sent by a user terminal is received.

[0066] In the embodiments of the present application, the user terminal refers to a terminal device used to execute the image processing method for preventing document abuse provided by the present application. The user terminal can be a mobile terminal such as a mobile phone, a smart phone, a laptop computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a navigation device, etc., as well as a fixed terminal such as a digital TV, a desktop computer, etc. It should be understood that the examples of user terminals here are only for the convenience of understanding and are not used to limit the present application.

[0067] In step S202, the question information is input into the large language model to generate a first reply text.

[0068] In an embodiment of the present application, when a user asks a question through a terminal device, the present application first inputs the question into a large language model to generate a preliminary reply text, i.e., a first reply text, wherein the large language model may be constructed based on a Transformer architecture, such as GPT-3. It should be understood that the example of the large language model here is only for ease of understanding and is not used to limit the present application.

[0069] In step S203, N search formula classification models are called, and confidence evaluation operations are performed on the first reply text according to the N search formula classification models respectively to obtain N confidence evaluation results, where N is an integer greater than or equal to 1.

[0070] In the embodiment of the present application, the present application pre-trains several classification models corresponding to different application fields / scenarios, so as to achieve accurate judgment of specific field knowledge.

[0071] In an embodiment of the present application, the retrieval classification model is mainly used to analyze the first reply text generated above to obtain a confidence score related to the quality of the reply text. The score is calculated based on a pre-trained corpus and a specific algorithm. For example, the probability distribution is calculated using a softmax function to obtain the confidence level of the reply. As an example, the retrieval classification model can be BERT, RoBERTa or XLNet, etc. It should be understood that the examples of retrieval classification models here are only for ease of understanding and are not used to limit this application.

[0072] In step S204, the optimal evaluation result with the highest confidence is screened out from the N confidence evaluation results, and it is determined whether the optimal evaluation result meets a preset confidence threshold.

[0073] In an embodiment of the present application, after obtaining the confidence scores of several retrieval classification models, the confidence scores are compared to select the highest confidence score as the optimal evaluation result.

[0074] In the embodiment of the present application, the preset confidence threshold is mainly used to uniquely determine whether the response content of the optimal evaluation result is accurate and reliable, wherein the preset confidence threshold can be dynamically adjusted according to the specific situation. As an example, the confidence threshold can be 85, 90, 95, etc. It should be understood that the examples of confidence thresholds here are only for the convenience of understanding and are not used to limit the present application.

[0075] In step S205, if the optimal evaluation result meets the preset confidence threshold, the first reply text is sent to the user terminal as the target reply text.

[0076] In step S206, if the optimal evaluation result does not meet the preset confidence threshold, a first PROMPT template is generated according to the optimal evaluation result.

[0077] In an embodiment of the present application, if the confidence level of the optimal evaluation result is lower than the confidence level threshold, it means that the first reply text needs to be further optimized.

[0078] In an embodiment of the present application, a Prompt template optimized specifically for the question is generated based on the evaluation results. The template guides the language model to generate a more accurate response by combining the original question and the evaluation feedback.

[0079] In step S207, the first PROMPT template is input into the large language model to obtain a second reply text.

[0080] In an embodiment of the present application, the large language model is used again to process the optimized Prompt template to generate a second reply text.

[0081] In step S208, the second reply text is sent to the user terminal as the target reply text.

[0082] In an embodiment of the present application, a reply text generation method applied to a large language model is provided, including: receiving a dialogue request carrying question information sent by a user terminal; inputting the question information into the large language model to generate a first reply text; calling N retrieval classification models, and performing confidence assessment operations on the first reply text according to the N retrieval classification models, respectively, to obtain N confidence assessment results, wherein N is an integer greater than or equal to 1; screening the optimal assessment result with the highest confidence among the N confidence assessment results, and judging whether the optimal assessment result meets a preset confidence threshold; if the optimal assessment result meets the preset confidence threshold, sending the first reply text as the target reply text to the user terminal; if the optimal assessment result does not meet the preset confidence threshold, generating a first PROMPT template according to the optimal assessment result; inputting the first PROMPT template into the large language model to obtain a second reply text; and sending the second reply text as the target reply text to the user terminal. Compared with the existing technology, this application not only improves the accuracy of replies, but also dynamically adjusts the PROMPT template to formulate different reply strategies for different confidence assessment results to maximize user needs. In addition, it also improves the intelligence level of the dialogue system and user satisfaction, while reducing the risk of misunderstanding replies and improving the overall service quality and efficiency.

[0083] In some optional implementations of the embodiments of the present application, step S208 specifically includes the following steps:

[0084] Calculate the similarity between the second reply text and the first reply text according to the semantic similarity algorithm to obtain a similarity score;

[0085] If the similarity score is greater than or equal to the preset similarity threshold, the second reply text is sent to the user terminal as the target reply text;

[0086] If the similarity score is less than the preset similarity threshold, a second PROMPT template is regenerated according to the optimal evaluation result, and the second PROMPT template is input into the large language model until the similarity score between the third reply text and the first reply text is greater than or equal to the preset similarity threshold;

[0087] The third reply text is sent to the user terminal as the target reply text.

[0088] In an embodiment of the present application, the similarity of the two reply texts is first calculated using a semantic similarity algorithm based on BERT. Assuming that the preset similarity threshold is set to 80, if the similarity score obtained by the first calculation is only 75, it indicates that the similarity of the two texts is insufficient. It is necessary to use an automated analysis tool to compare the keyword usage and sentence structure of the two texts in detail to identify differences, such as the omission of keywords or the complexity of the sentence structure. Based on this analysis result, the second PROMPT template is automatically adjusted, such as adding missing keywords or simplifying the sentence structure. The adjusted template is input into the large language model again to generate a new third reply text. The BERT algorithm is used again to calculate the similarity between this text and the first reply text. If the new similarity score is increased to 82, that is, it exceeds the threshold, this text can be used as the target reply text; if it still does not meet the standard, the system will continue to iteratively optimize the PROMPT template and regenerate the text until the similarity meets the preset threshold.

[0089] In an embodiment of the present application, by continuously iteratively optimizing the PROMPT template to generate a reply text that meets a preset threshold, the accuracy and consistency of the reply text are ensured while improving processing efficiency. In addition, through continuous optimization and self-learning, the system can more accurately control the quality of semantic expression and user satisfaction.

[0090] In some optional implementations of the embodiments of the present application, the above step of sending the second reply text as the target reply text to the user terminal specifically includes the following steps:

[0091] Calculate the similarity between the second reply text and the first reply text according to the semantic similarity algorithm to obtain a similarity score;

[0092] If the similarity score is greater than or equal to the preset similarity threshold, the second reply text is sent to the user terminal as the target reply text;

[0093] If the similarity score is less than the preset similarity threshold, a second PROMPT template is regenerated according to the optimal evaluation result, and the second PROMPT template is input into the large language model until the similarity score between the third reply text and the first reply text is greater than or equal to the preset similarity threshold;

[0094] The third reply text is sent to the user terminal as the target reply text.

[0095] In an embodiment of the present application, the semantic similarity algorithm is mainly used to evaluate the semantic similarity between the second reply text and the first reply text, wherein the semantic similarity algorithm can be a similarity algorithm based on a bag-of-words model, a similarity algorithm based on word embedding, a similarity algorithm based on edit distance, a similarity algorithm based on syntactic analysis, or a similarity algorithm based on a knowledge graph, etc. It should be understood that the examples of semantic similarity algorithms given here are only for ease of understanding and are not used to limit the present application.

[0096] In some optional implementations of the embodiments of the present application, the above step of calculating the similarity between the second reply text and the first reply text according to the semantic similarity algorithm to obtain a similarity score specifically includes the following steps:

[0097] Acquire all the words of the second reply text and the first reply text according to natural language processing technology to obtain a second text vocabulary set and a first text vocabulary set;

[0098] Determine the intersection of the second text vocabulary set and the first text vocabulary set according to a set operation algorithm to obtain a vocabulary overlap;

[0099] The similarity score is obtained based on the word overlap.

[0100] In an embodiment of the present application, when performing text similarity analysis, natural language processing is first performed on the two texts to extract their respective vocabulary lists, so as to obtain two vocabulary sets, and then the intersection of the two sets is calculated to obtain the vocabulary overlap, wherein the vocabulary overlap can be calculated by dividing the number of words in the intersection by the smaller number of words in the set.

[0101] In some optional implementations of the embodiments of the present application, after the step of obtaining all the vocabulary of the second reply text and the first reply text according to the natural language processing technology to obtain the second text vocabulary set and the first text vocabulary set, and before the step of determining the intersection of the second text vocabulary set and the first text vocabulary set according to the set operation algorithm to obtain the vocabulary overlap, the following steps are also included:

[0102] Performing semantic analysis operations on the second text vocabulary set and the first text vocabulary set respectively according to the semantic analysis model to obtain a second semantic vocabulary set and a first semantic vocabulary set;

[0103] The above step of determining the intersection of the second text vocabulary set and the first text vocabulary set according to the set operation algorithm to obtain the vocabulary overlap specifically includes the following steps:

[0104] The intersection of the second semantic vocabulary set and the first semantic vocabulary set is determined according to a set operation algorithm to obtain a vocabulary overlap.

[0105] In an embodiment of the present application, the semantic analysis model mainly understands and parses the deep meaning of the text based on the position of the vocabulary in the original text and the contextual relationship. The semantic analysis model can be a deep semantic model (DSM), NL2SQL (natural language to SQL) or context and entity recognition, etc. It should be understood that the examples of semantic analysis models here are only for convenience of understanding and are not used to limit the present application.

[0106] In an embodiment of the present application, the deep meaning of all the words in the second reply text and the first reply text is obtained through a semantic analysis model, and the intersection is determined based on the words with deeper meanings, thereby effectively avoiding the problem of too low vocabulary overlap due to different words with the same meaning, and effectively improving the accuracy of calculating vocabulary overlap.

[0107] In some optional implementations of the embodiments of the present application, step S208 includes the following steps:

[0108] The sensitive word library is called, and sensitive word filtering operation is performed on the second reply text according to the sensitive word library.

[0109] In the embodiment of the present application, after the second reply text is generated, the sensitive word library can be called and the current sensitive word library can be obtained by querying the database to ensure that the filtering operation uses the latest sensitive information list. The sensitive word library is updated once a day to include the latest sensitive information list.

[0110] In an embodiment of the present application, the sensitive word filtering operation may be performed by utilizing a vocabulary matching algorithm in natural language processing technology, such as the KMP (Knuth Morris Pratt) algorithm, where the system detects the reply text word by word to ensure accurate identification of any sensitive words.

[0111] In the embodiment of the present application, once sensitive words are detected, the system will automatically start the filtering mechanism to replace these sensitive words with preset non-sensitive words.

[0112] In an embodiment of the present application, the modified text may be re-checked according to a text review algorithm to ensure that all sensitive words have been properly handled. This algorithm includes but is not limited to a support vector machine (SVM) model for high-accuracy text classification.

[0113] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0114] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0115] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0116] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0117] Embodiment 2

[0118] Further references Figure 3 , as a response to the above Figure 2The present application provides an embodiment of a reply text generation device applied to a large language model, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0119] like Figure 3 As shown, the reply text generation device 200 applied to the large language model of the embodiment of the present application includes:

[0120] The request receiving module 210 is used to receive a dialogue request carrying question information sent by a user terminal;

[0121] A first text generation module 220, used to input the question information into the large language model to generate a first answer text;

[0122] The confidence evaluation module 230 is used to call N search formula classification models, and perform confidence evaluation operations on the first reply text according to the N search formula classification models, to obtain N confidence evaluation results, where N is an integer greater than or equal to 1;

[0123] The confidence judgment module 240 is used to select the best evaluation result with the highest confidence from the N confidence evaluation results, and judge whether the best evaluation result meets the preset confidence threshold;

[0124] A first text output module 250, configured to send the first reply text as a target reply text to a user terminal if the optimal evaluation result meets a preset confidence threshold;

[0125] A first template generating module 260, configured to generate a first PROMPT template according to the optimal evaluation result if the optimal evaluation result does not meet a preset confidence threshold;

[0126] A second text generation module 270, for inputting the first PROMPT template into the large language model to obtain a second reply text;

[0127] The second text output module 280 is used to send the second reply text as the target reply text to the user terminal.

[0128] In an embodiment of the present application, a reply text generation device 200 applied to a large language model is provided, comprising: a request receiving module 210, for receiving a dialogue request carrying question information sent by a user terminal; a first text generation module 220, for inputting the question information into the large language model to generate a first reply text; a confidence evaluation module 230, for calling N search formula classification models, and performing confidence evaluation operations on the first reply text according to the N search formula classification models, respectively, to obtain N confidence evaluation results, wherein N is an integer greater than or equal to 1; a confidence judgment module 240, for screening confidence evaluation results from the N confidence evaluation results. The optimal evaluation result with the highest degree of confidence is selected, and it is judged whether the optimal evaluation result meets the preset confidence threshold; the first text output module 250 is used to send the first reply text as the target reply text to the user terminal if the optimal evaluation result meets the preset confidence threshold; the first template generation module 260 is used to generate the first PROMPT template according to the optimal evaluation result if the optimal evaluation result does not meet the preset confidence threshold; the second text generation module 270 is used to input the first PROMPT template into the large language model to obtain the second reply text; the second text output module 280 is used to send the second reply text as the target reply text to the user terminal. Compared with the prior art, the present application not only improves the accuracy of the reply, but also formulates different reply strategies for different confidence evaluation results by dynamically adjusting the PROMPT template to meet the needs of users to the greatest extent. In addition, the intelligence level and user satisfaction of the dialogue system are improved, while reducing the risk of misunderstanding the reply and improving the quality and efficiency of the overall service.

[0129] In some optional implementations of the embodiments of the present application, the second text output module 280 includes:

[0130] A similarity calculation submodule is used to calculate the similarity between the second reply text and the first reply text according to a semantic similarity algorithm to obtain a similarity score;

[0131] A target text confirmation submodule, for sending the second reply text as the target reply text to the user terminal if the similarity score is greater than or equal to a preset similarity threshold;

[0132] A template generation submodule, for regenerating a second PROMPT template according to the optimal evaluation result if the similarity score is less than a preset similarity threshold, and inputting the second PROMPT template into the large language model until the similarity score between the obtained third reply text and the first reply text is greater than or equal to the preset similarity threshold;

[0133] The second text output submodule is used to send the third reply text as the target reply text to the user terminal.

[0134] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of a computer device according to an embodiment of the present application.

[0135] The computer device 300 includes a memory 310, a processor 320, and a network interface 330 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 300 having components 310-330, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital processors (DSPs), embedded devices, etc.

[0136] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0137] The memory 310 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 310 can be an internal storage unit of the computer device 300, such as a hard disk or memory of the computer device 300. In other embodiments, the memory 310 can also be an external storage device of the computer device 300, such as a plug-in hard disk equipped on the computer device 300, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc. Of course, the memory 310 can also include both the internal storage unit of the computer device 300 and its external storage device. In the embodiment of the present application, the memory 310 is generally used to store the operating system and various application software installed on the computer device 300, such as computer-readable instructions for the reply text generation method applied to the large language model. In addition, the memory 310 can also be used to temporarily store various data that have been output or are to be output.

[0138] The processor 320 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 320 is generally used to control the overall operation of the computer device 300. In the embodiment of the present application, the processor 320 is used to run the computer-readable instructions stored in the memory 310 or process data, such as running the computer-readable instructions of the reply text generation method applied to the large language model.

[0139] The network interface 330 may include a wireless network interface or a wired network interface. The network interface 330 is generally used to establish a communication connection between the computer device 300 and other electronic devices.

[0140] The computer device provided by this application not only improves the accuracy of responses, but also formulates different response strategies for different confidence assessment results by dynamically adjusting the PROMPT template to meet user needs to the greatest extent possible. In addition, it also improves the intelligence level of the dialogue system and user satisfaction, while reducing the risk of misunderstanding responses and improving the overall service quality and efficiency.

[0141] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the reply text generation method applied to a large language model as described above.

[0142] The computer-readable storage medium provided by the present application not only improves the accuracy of responses, but also formulates different response strategies for different confidence assessment results by dynamically adjusting the PROMPT template to meet user needs to the greatest extent possible. In addition, it also improves the intelligence level of the dialogue system and user satisfaction, while reducing the risk of misunderstanding responses and improving the overall service quality and efficiency.

[0143] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0144] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A reply text generation method applied to a large language model, characterized in that: The steps include: Receiving a dialogue request carrying question information sent by a user terminal; Inputting the question information into a large language model to generate a first answer text; Calling N search formula classification models, and performing confidence evaluation operations on the first reply text according to the N search formula classification models, respectively, to obtain N confidence evaluation results, where N is an integer greater than or equal to 1; Selecting the best evaluation result with the highest confidence from the N confidence evaluation results, and determining whether the best evaluation result meets a preset confidence threshold; If the optimal evaluation result meets the preset confidence threshold, sending the first reply text as the target reply text to the user terminal; If the optimal evaluation result does not meet the preset confidence threshold, generating a first PROMPT template according to the optimal evaluation result; Inputting the first PROMPT template into the large language model to obtain a second reply text; The second reply text is sent to the user terminal as the target reply text.

2. The method for generating a reply text using a large language model according to claim 1, wherein: The step of sending the second reply text as the target reply text to the user terminal specifically includes the following steps: Calculating the similarity between the second reply text and the first reply text according to a semantic similarity algorithm to obtain a similarity score; If the similarity score is greater than or equal to a preset similarity threshold, sending the second reply text as the target reply text to the user terminal; If the similarity score is less than the preset similarity threshold, regenerate a second PROMPT template according to the optimal evaluation result, and input the second PROMPT template into the large language model until the similarity score between the obtained third reply text and the first reply text is greater than or equal to the preset similarity threshold; The third reply text is sent to the user terminal as the target reply text.

3. The method for generating a reply text using a large language model according to claim 2, wherein: The step of calculating the similarity between the second reply text and the first reply text according to the semantic similarity algorithm to obtain a similarity score specifically includes the following steps: Acquire all the words of the second reply text and the first reply text according to natural language processing technology to obtain a second text vocabulary set and a first text vocabulary set; Determine the intersection of the second text vocabulary set and the first text vocabulary set according to a set operation algorithm to obtain a vocabulary overlap; The similarity score is obtained according to the vocabulary overlap.

4. The method for generating a reply text using a large language model according to claim 3, wherein: After the step of obtaining all the vocabulary of the second reply text and the first reply text according to the natural language processing technology to obtain the second text vocabulary set and the first text vocabulary set, and before the step of determining the intersection of the second text vocabulary set and the first text vocabulary set according to the set operation algorithm to obtain the vocabulary overlap, the following steps are also included: Performing semantic analysis operations on the second text vocabulary set and the first text vocabulary set respectively according to the semantic analysis model to obtain a second semantic vocabulary set and a first semantic vocabulary set; The step of determining the intersection of the second text vocabulary set and the first text vocabulary set according to the set operation algorithm to obtain the vocabulary overlap specifically includes the following steps: The intersection of the second semantic vocabulary set and the first semantic vocabulary set is determined according to a set operation algorithm to obtain the vocabulary overlap.

5. The method for generating a reply text using a large language model according to claim 1, wherein: After the step of sending the second reply text as the target reply text to the user terminal, the following step is also included: A sensitive word library is called, and a sensitive word filtering operation is performed on the second reply text according to the sensitive word library.

6. A reply text generation device applied to a large language model, characterized in that: include: A request receiving module, used for receiving a dialogue request carrying question information sent by a user terminal; A first text generation module, used for inputting the question information into the large language model to generate a first reply text; A confidence evaluation module, used to call N search formula classification models, and perform confidence evaluation operations on the first reply text according to the N search formula classification models, respectively, to obtain N confidence evaluation results, where N is an integer greater than or equal to 1; A confidence judgment module, used to select the best evaluation result with the highest confidence among the N confidence evaluation results, and judge whether the best evaluation result meets a preset confidence threshold; A first text output module, configured to send the first reply text as a target reply text to the user terminal if the optimal evaluation result meets the preset confidence threshold; A first template generating module, configured to generate a first PROMPT template according to the optimal evaluation result if the optimal evaluation result does not meet the preset confidence threshold; A second text generation module, used for inputting the first PROMPT template into the large language model to obtain a second reply text; The second text output module is used to send the second reply text as the target reply text to the user terminal.

7. The reply text generation device for a large language model according to claim 6, characterized in that: The second text output module comprises: A similarity calculation submodule, used to calculate the similarity between the second reply text and the first reply text according to a semantic similarity algorithm to obtain a similarity score; A target text confirmation submodule, configured to send the second reply text as the target reply text to the user terminal if the similarity score is greater than or equal to a preset similarity threshold; A template generation submodule, for regenerating a second PROMPT template according to the optimal evaluation result if the similarity score is less than the preset similarity threshold, and inputting the second PROMPT template into the large language model until the similarity score between the obtained third reply text and the first reply text is greater than or equal to the preset similarity threshold; The second text output submodule is used to send the third reply text as the target reply text to the user terminal.

8. The reply text generation device for a large language model according to claim 7, characterized in that: The similarity calculation submodule includes: A vocabulary acquisition unit, used for acquiring all the vocabulary of the second reply text and the first reply text according to natural language processing technology to obtain a second text vocabulary set and a first text vocabulary set; an overlap acquisition unit, configured to determine the intersection of the second text vocabulary set and the first text vocabulary set according to a set operation algorithm to obtain a vocabulary overlap; A similarity calculation unit is used to obtain the similarity score according to the vocabulary overlap.

9. A computer device comprising a memory and a processor, characterized in that: The memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the reply text generation method applied to a large language model as described in any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the method for generating a reply text applied to a large language model as described in any one of claims 1 to 5.

Images