Method for realizing intelligent customer service based on large language model
By using multiple LLM Agents and knowledge bases in the customer service system for query and filtering, and clarifying user input, the problem of high cost in the existing technology and difficult to understand user intentions is solved, and high-quality user question answers are achieved .
Patent Information
- Application Number
- CN202411199348.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-11-22
AI Technical Summary
The existing customer service system based on large language models is costly and time-consuming, and external mount knowledge methods are difficult to understand the user's true intentions, resulting in a decrease in accuracy and relevance.
By introducing multiple LLM Agents into the customer service system, querying and filtering with the knowledge base, gradually clarifying user input, improving the accuracy and relevance of knowledge retrieval, and using LLM's text understanding and logical reasoning capabilities to enhance the dialogue and interaction mode.
It effectively improves the quality of responses of the customer service system when answering user questions, improves the accuracy and relevance of knowledge retrieval, and solves the problem that external knowledge base tools cannot understand user intentions.
Smart Images

Figure CN119003736A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of large model knowledge question answering technology, and in particular to a method for realizing intelligent customer service based on a large language model. Background Art
[0002] 1. The existing LLM-based customer service system can be described as combining the product's knowledge documents, using LLM's ability to understand and generate human language, and solving user problems by communicating with users.
[0003] Common LLM solutions combined with knowledge documents are as follows:
[0004] 1.1. Update model weights. Use product documents as data sets to pre-train or fine-tune LLM, store the knowledge in product documents in LLM parameters, and enable LLM to understand user questions and provide answers to relevant product questions. For updates to product knowledge documents, depending on the amount of updates, you can use methods such as training from scratch, continuous pre-training, and continuous fine-tuning to update model parameters, so that LLM can answer relevant questions more accurately based on the updated knowledge.
[0005] 1.2. Externally mount product knowledge. Use external tools to search for relevant content in product documents based on user input, and encapsulate the relevant content and user input into prompts as input for LLM. LLM answers user questions by understanding the content in the prompts. The way to search for document fragments is to index the document, and then process the user input and match it with the document index. Commonly used external tools include Langchain, etc.
[0006] 2. Existing solutions have achieved good results in the field of intelligent customer service. However, in the actual application of the LLM customer service system, we found that the above two technical solutions have the following problems:
[0007] 2.1. Large language models (LLMs) have a large number of parameters, and pre-training and fine-tuning are costly and time-consuming. If product documentation is updated frequently, costs will continue to increase and it will be difficult to ensure timeliness. In addition, the update of LLM weights may cause the model to become less stable, leading to problems such as hallucination and catastrophic forgetting. Staff need to invest extra time to adjust training data or model parameters to solve these problems.
[0008] 2.2. By using external knowledge loading, LLM does not need to update its own weights, which avoids the above problems. However, using external tools to obtain knowledge content related to user input does not fully utilize LLM's powerful language processing capabilities, and cannot fully capture the language rules and contextual information learned within LLM. Its accuracy and relevance will be reduced compared to the weight update method. In addition, users lack professional knowledge in related fields, so their original input sometimes cannot accurately reflect the essence of the problem and has a certain degree of randomness. When using external tools to retrieve knowledge related to user input, it is often impossible to understand the user's true intentions, resulting in a decrease in the relevance of the retrieved knowledge to the essence of the problem. Summary of the invention
[0009] In view of the shortcomings of the prior art, the present invention provides a method for realizing intelligent customer service based on a large language model, which solves the problems that the existing customer service system is costly and time-consuming, external mounting saves costs but is difficult to understand the user's true intentions, and has reduced relevance and accuracy.
[0010] To achieve the above objectives, the present invention is implemented through the following technical solutions: A method for realizing intelligent customer service based on a large language model, comprising the following steps:
[0011] Get user input and pass it to the LLM single question answering module;
[0012] The LLM single question answering module processes the user input and outputs the answer to the question;
[0013] If the user chooses to continue asking questions, the question and answer will be stored in the context module, and the user can continue to enter questions;
[0014] If the user chooses to be transferred to a manual customer service representative, the user will be transferred to a manual customer service representative.
[0015] If the user chooses that they have no other questions, they will enter the feedback module;
[0016] The context module synthesizes and integrates the user input and the previous text, and the integrated user question will be passed to the LLM single question answering module.
[0017] Preferably, the processing flow of the LLM single question answering module includes the following steps:
[0018] Pass the user input to the knowledge base query module to obtain the query results;
[0019] The query results are passed to the question clarification module along with the user input;
[0020] The question clarification module outputs the clarified user input, which is then passed to the knowledge base query module for a second query;
[0021] The second query result and the clarified user input are passed to the answer module, which outputs the answer to the user's question.
[0022] Preferably, the processing flow of the problem clarification module includes the following steps:
[0023] The knowledge base query results and user input are passed to the Filter Agent. The Filter Agent uses the text understanding and reasoning capabilities of the LLM to delete the unreasonable entries or entries that conflict with the user input in the knowledge base query results according to the user input content, merge the entries with the same meaning, and output the filtered query results.
[0024] If the number of filtered query results is greater than 1, they are passed to the question clarification dialogue module together with the user input. The question clarification dialogue module passes the relatively clear user input and the filtered query results to the Filter Agent to filter the query results again until the number of remaining query results is 1 or 0;
[0025] If the number of filtered query results is less than or equal to 1, the remaining query results and user input are passed to MergeAgent. Merge Agent summarizes and integrates multiple pieces of information to make the user input question more specific and clear, and outputs the clarified user input.
[0026] Preferably, the processing flow of the question-clarifying dialogue module includes the following steps:
[0027] The filtered query results and user input are passed to Questioner Agent. Based on the logical relationship between the query results and the user input, Questioner Agent infers the unclear content in the user input and selects the most critical question to ask the user.
[0028] The user's reply is passed to the Reply Judge Agent, which determines whether the user's reply is an answer to the previous question from the Questioner Agent or whether it raises a new question;
[0029] If the user asks a new question, the user input, Agent query, and user response are combined into a new user input, and the LLM single question answering module is recursively called;
[0030] If the user's answer is obtained, the user input, Agent query, and user reply are passed to the Merge Agent. The Merge Agent outputs a clearer user input and merges it with the filtered query results as the output of the question clarification dialogue module.
[0031] Preferably, the Filter Agent uses the following formula to delete the query results that conflict with the user input:
[0032] R f ={ri∈R|Sim(ri,Q)<θ1}
[0033] Among them, R f represents the filtered query result, R is the initial query result set, Ri is an item in the query result, Q is the user input, Sim(ri,Q) is the similarity function between the query result and the user input, and θ1 is the preset similarity threshold.
[0034] Preferably, the Merge Agent uses the following formula to merge multiple pieces of information into a specific and clear user input expressed by the following formula:
[0035] Q m =Merge(Q P ,R f )
[0036] Among them, Q m Indicates the user input after clarification, Q p Represents the previous user input paragraph, R f is the filtered query result set, and Merge is the merge function.
[0037] Preferably, the Questioner Agent uses the following formula to determine the key question in the user input:
[0038] qk=argmax q Uncertainty(q,R f )
[0039] Among them, qk represents the most critical question at the moment, q is the set of all possible questions, and Uncertainty is the uncertainty function, which is used to measure the ambiguity of each question relative to the filtered query results.
[0040] Preferably, the Reply Judge Agent uses the following formula to determine the type of user reply:
[0041] T r =Classify(R u)
[0042] Among them, T r Indicates the type of user reply, Classify is the classification function, R u The user's reply content.
[0043] Preferably, the final solution output by the LLM single question solution module is determined by the following formula:
[0044] A=Answer(Q m ,R m )
[0045] A is the final solution, Answer is the solution function, and R m is the final query result set obtained by querying the knowledge base.
[0046] Beneficial Effects
[0047] The present invention provides a method for realizing intelligent customer service based on a large language model. Compared with the prior art, it has the following beneficial effects:
[0048] In the present invention, the user input is clarified by asking the user through the customer service system. The knowledge retrieval mode based on the external mounted knowledge base and multiple LLM Agents in the solution processes the query results and user input through different LLM Agents, improves the participation of LLM in the knowledge retrieval process, and uses the large amount of knowledge reserves and logical reasoning capabilities of LLM itself to improve the accuracy and relevance of knowledge retrieval. The present invention effectively solves the problem that when relying on external knowledge base tools to obtain relevant knowledge from user input, the accuracy and relevance are significantly reduced compared to the weight update method; and the user lacks professional knowledge in related fields, and their initial input sometimes cannot accurately reflect the essence of the problem, and there is a certain randomness, which in turn affects the accuracy of knowledge base retrieval. The new interaction mode integrates different LLM Agents, giving full play to the capabilities of LLM in text understanding and logical reasoning, thereby improving the quality of customer service's responses when answering user questions. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1 The overall process of the LLM intelligent customer service system for realizing intelligent customer service based on a large language model proposed by the present invention;
[0050] Figure 2 A flowchart of an LLM single question answering module of a method for realizing intelligent customer service based on a large language model proposed by the present invention;
[0051] Figure 3 A flowchart of a question clarification module of a method for realizing intelligent customer service based on a large language model proposed by the present invention;
[0052] Figure 4 A flowchart of a question-clarifying dialogue module of a method for realizing intelligent customer service based on a large language model proposed by the present invention;
[0053] Figure 5 A flowchart of a hardware scenario for a method for realizing intelligent customer service based on a large language model proposed in the present invention. DETAILED DESCRIPTION
[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0055] See also Figure 1 - Figure 5 The present invention provides two technical solutions, which specifically include the following embodiments:
[0056] Example:
[0057] A method for implementing intelligent customer service based on a large language model comprises the following steps:
[0058] Get user input and pass it to the LLM single question answering module;
[0059] The LLM single question answering module processes the user input and outputs the answer to the question;
[0060] If the user chooses to continue asking questions, the question and answer will be stored in the context module, and the user can continue to enter questions;
[0061] If the user chooses to be transferred to a manual customer service representative, the user will be transferred to a manual customer service representative.
[0062] If the user chooses that they have no other questions, they will enter the feedback module;
[0063] The context module synthesizes and integrates the user input and the previous text, and the integrated user question will be passed to the LLM single question answering module.
[0064] The processing flow of the LLM single question answering module includes the following steps:
[0065] Pass the user input to the knowledge base query module to get the query result. For the knowledge base query module, a knowledge base needs to be established. The process of establishing a knowledge base usually includes three steps: entity discovery, coreference resolution, and relationship extraction. In addition, an end-to-end approach can be adopted, such as using a large language model (such as LLM) to directly perform one-step triple extraction. In order to enrich the information of the retrieved triples so that the LLM can make more accurate judgments, you can consider adding an additional item to the triple structure to store the file directory and paragraph structure of the source document from which the triples are extracted. Doing so can add more structured information to the triples in the knowledge base, thereby improving the accuracy and effect of subsequent natural language processing tasks. The triple structure design reference is as follows:
[0066] class KnowledgeGraphTriplet:
[0067] def__init__(self,subject,predicate,object,object_type,file_directory_structure):
[0068] self.header=file_directory_structure
[0069] self.subject = subject
[0070] self.predicate = predicate
[0071] self.object = object
[0072] self.object_type=object_type
[0073] Among them, subject, predicate, and object represent a triple structure. The header part stores the directory structure, which can be stored using a list structure to display the directory names from the root directory to the current paragraph. Object can store specific object text or a reference to an object, and its type is stored by object_type.
[0074] The tools available for building knowledge bases include Langchain, AllegroGraph, etc.
[0075] The query results are passed to the question clarification module along with the user input;
[0076] The question clarification module outputs the clarified user input, which is then passed to the knowledge base query module for a second query;
[0077] The second query result and the clarified user input are passed to the answer module, which outputs the answer to the user's question, and the user input is passed to the LLM single question answering module. Figure 2 As shown, the LLM single question answering module is entered, and the user input is passed to the knowledge base query module.
[0078] The knowledge base query method can adopt a three-step approach based on mention detection, entity disambiguation, and relationship classification, or a direct real-time query method. The query results are ranked according to relevance to select the most relevant k results. Here k should be slightly larger, for example, k is 30, to cover the user's possible real intentions as much as possible.
[0079] Query result example
[0080] triplet_1:
[0081] header = ["Video-on-demand service platform", "Client", "Instructions", "Video playback", "FAQ"]
[0082] subject = "Player not loaded"
[0083] predicate = "reason"
[0084] object=link_to_relevant_segment
[0085] object_type="segment_link"
[0086] triplet_2:
[0087] header = ["Video-on-demand service platform", "Client", "Instructions", "Video playback", "FAQ"]
[0088] subject = "Player error code 01"
[0089] predicate = "reason"
[0090] object=link_to_relevant_segment
[0091] object_type="segment_link"
[0092] triplet_3:
[0093] header = ["Video-on-demand service platform", "Web page", "Instructions", "Video playback", "FAQ"]
[0094] subject = "Player error code H03"
[0095] predicate = "reason"
[0096] object=link_to_relevant_segment
[0097] object_type="segment_link"
[0098] triplet_4:
[0099] header = ["Live broadcast service platform", "Client", "Instructions", "Video playback", "FAQ"]
[0100] subject = "Player error code 02"
[0101] predicate = "reason"
[0102] object=link_to_relevant_segment
[0103] object_type="segment_link" ......
[0105] Knowledge base related tools include langchain, ChatKBQA, etc. Vector databases for storing triples can use Faiss, Milvus, etc.
[0106] The processing flow of the problem clarification module includes the following steps:
[0107] The knowledge base query results and user input are passed to the Filter Agent, which uses the text understanding and reasoning capabilities of the LLM to delete unreasonable entries or entries that conflict with the user input in the knowledge base query results based on the user input content, merge entries with the same meaning, output the filtered query results, and pass the query results and user input to the question clarification module. Figure 3 As shown, the query results and user input are passed to the Filter Agent, which deletes the query results that are repeated or conflict with the user input and outputs the filtered query results. The following is an example of the filtered query results:
[0108] triplet_2:
[0109] header = ["Video-on-demand service platform", "Client", "Instructions", "Video playback", "FAQ"]
[0110] subject = "Player error code 01"
[0111] predicate = "reason"
[0112] object=link_to_relevant_segment
[0113] object_type="segment_link"
[0114] triplet_3:
[0115] header = ["Video-on-demand service platform", "Web page", "Instructions", "Video playback", "FAQ"]
[0116] subject = "Player error code H03"
[0117] predicate = "reason"
[0118] object=link_to_relevant_segment
[0119] object_type="segment_link"
[0120] triplet_4:
[0121] header = ["UHD Platform", "Client", "Instructions", "Video Playback", "FAQ"]
[0122] subject = "Player error code 02"
[0123] predicate = "reason"
[0124] object=link_to_relevant_segment
[0125] object_type="segment_link" ......
[0127] The "Player not loaded" in triplet_1 does not match the "Playback failed" input by the user and is deleted by the Filter Agent.
[0128] If the number of filtered query results is greater than 1, they are passed to the question clarification dialogue module together with the user input. The question clarification dialogue module passes the relatively clear user input and the filtered query results to the Filter Agent to filter the query results again until the number of remaining query results is 1 or 0;
[0129] If the number of filtered query results is less than or equal to 1, the remaining query results and user input are passed to MergeAgent. Merge Agent summarizes and integrates multiple pieces of information to make the user input question more specific and clear, and outputs the clarified user input. If the remaining number of filtered query results is greater than 1, the filtered query results and user input are passed to the question clarification dialogue module. Figure 4 As shown in the figure, the question clarification dialogue module is entered, and the filtered query results and user input are passed to the Questioner Agent. The Questioner Agent will infer the structure represented by the header part and the triples in the query results, and find the part with multiple choices. For example, the Questioner Agent splices the subject and predicate parts to the end of the header part. If the value of object_type is "str", the object is also spliced after the predicate. Then, the logical structure is inferred based on the spliced list and expressed in the form of a tree. Next, the Questioner Agent selects the most unclear part of the user input and converts this part into a question that the user needs to clarify. Finally, the question that needs to be clarified is converted into an Agent query in the tone of customer service and output.
[0130] The Agent asks in the example: "Thank you for your inquiry. In order to better help you, I need to confirm whether you are using our on-demand service platform or our live broadcast service platform?" The Agent asks the user, and the system waits for the user's reply. The user's reply is passed to the Reply Judge Agent. The Reply Judge Agent determines whether the user's reply raises a new question or answers the Agent's inquiry.
[0131] The user reply in the example is "How do I determine whether I am using the UHD platform or the on-demand platform?" Reply JudgeAgent determines that the user has raised a new question. The system uses the user input, Agent query, and user reply as context and passes them to the LLM single question answering module to answer the user's question recursively. After the recursively called LLM single question answering module completes the answer, it returns to 2.2.5.
[0132] In another example, the user reply is "on-demand service platform." The Reply Judge Agent determines that the user has not raised a new question, and the system passes the user input, agent query, and user reply to the Merge Agent. The Merge Agent uses the text understanding and summary capabilities of the LLM to merge and summarize the user input, agent query, and user reply, and output a relatively clear user input. The relatively clear user input in the example is "I can't watch XX video on the on-demand service platform. It keeps saying that the playback failed. What should I do?" The relatively clear user input is used as the output of the question clarification dialogue module and returned to the question clarification module, such as Figure 3 As shown in the figure, the output of the question clarification dialogue module is simply concatenated with the filtered query results and passed to the FilterAgent. If the number of remaining query results is less than or equal to 1, the user input input to the Filter Agent and the filtered query results are passed to the Merge Agent. The Merge Agent outputs the clarified user input. The clarified user input in the example is "I cannot watch XX video on the client of the on-demand service platform. It keeps prompting the playback error code 01. What should I do?" The clarified user input is used as the output of the question clarification module and returned to the LLM single question answering module.
[0133] The processing flow of the question-clarifying dialogue module includes the following steps:
[0134] The filtered query results and user input are passed to Questioner Agent. Based on the logical relationship between the query results and the user input, Questioner Agent infers the unclear content in the user input and selects the most critical question to ask the user.
[0135] The user's reply is passed to the Reply Judge Agent, which determines whether the user's reply is an answer to the previous question from the Questioner Agent or whether it raises a new question;
[0136] If the user asks a new question, the user input, Agent query, and user response are combined into a new user input, and the LLM single question answering module is recursively called;
[0137] If the user's answer is obtained, the user input, agent query, and user reply are passed to the Merge Agent. The Merge Agent outputs a relatively clear user input, which is merged with the filtered query results as the output of the question clarification dialogue module. The output of the question clarification module is passed to the knowledge base query module again. The knowledge base query results are ranked according to relevance to select the most relevant m results. Here, m should be slightly smaller, for example, m is 5, so as to select the most accurate result and avoid the situation in which the LLM cannot handle the prompt word because it is too long in subsequent operations.
[0138] Example of query results
[0139] triplet_1:
[0140] header = ["Video-on-demand service platform", "Client", "Instructions", "Video playback", "FAQ"]
[0141] subject = "Player error code 01"
[0142] predicate = "reason"
[0143] object=link_to_relevant_segment
[0144] object_type="segment_link"
[0145] triplet_2:
[0146] header = ["On-demand service platform", "Client", "Solutions to common problems"]
[0147] subject="Video player error code 01"
[0148] predicate = "solution steps"
[0149] object=link_to_relevant_segment
[0150] object_type="segment_link" ......
[0152] The system adds the relevant document fragments pointed to by the object in each triple in the query result into the triple, and passes it to the answer module together with the clarified user input. The answer module can be an LLM Agent, which understands the user's question, finds the answer in the query result or the knowledge stored in the LLM weight parameter, and finally outputs the answer to the question in the tone of customer service.
[0153] Example of solution:
[0154] “
[0155] Thank you for the information you provided. The possible cause of player error code 01 is XXX.
[0156] The reference solution steps are as follows:
[0157] 1.XXX 2.... ......
[0160] I hope this helps.
[0161] ”
[0162] The answer is the output of the LLM single-question answering module and returns the total process, such as Figure 1 As shown, the answer is stored in the context module, and the system then provides the user with three options: continue asking questions, no more questions, or transfer to a human. Example system prompt:
[0163] “
[0164] Did the AI customer service answer solve your problem? Please select the next step:
[0165] Transfer to manual
[0166] No more questions
[0167] Or continue to enter your needs in the dialog box and AI customer service will continue to serve you.
[0168] ”
[0169] If the user selects "transfer to manual" or "no other questions", the system will transfer the interface of the corresponding module.
[0170] If the user continues to input questions, the context module will pass the previous question and answer content and user input to the LLM single question answering module.
[0171] The Filter Agent uses the following formula to delete the query results that conflict with the user input:
[0172] R f ={ri∈R|Sim(ri,Q)<θ1}
[0173] Among them, R frepresents the filtered query result, R is the initial query result set, Ri is an item in the query result, Q is the user input, Sim(ri,Q) is the similarity function between the query result and the user input, and θ1 is the preset similarity threshold.
[0174] The Merge Agent uses the following formula to merge multiple pieces of information into specific and clear user inputs expressed by the following formula:
[0175] Q m =Merge(Q P ,R f )
[0176] Among them, Q m Indicates the user input after clarification, Q p Represents the previous user input paragraph, R f is the filtered query result set, and Merge is the merge function.
[0177] The Questioner Agent uses the following formula to determine the key question in the user input:
[0178] qk=argmax q Uncertainty(q,R f )
[0179] Among them, qk represents the most critical question at the moment, q is the set of all possible questions, and Uncertainty is the uncertainty function, which is used to measure the ambiguity of each question relative to the filtered query results.
[0180] The Reply Judge Agent uses the following formula to determine the type of user reply:
[0181] T r =Classify(R u)
[0182] Among them, T r Indicates the type of user reply, Classify is the classification function, R u The user's reply content.
[0183] The final solution output by the LLM single-question solution module is determined by the following formula:
[0184] A=Answer(Q m ,R m )
[0185] A is the final solution, Answer is the solution function, and Rm is the final query result set obtained by querying the knowledge base.
[0186] When working, first, the system obtains input from the user and passes it to the LLM (Large Language Model) single question answering module. The LLM single question answering module is the core part of the intelligent customer service system and is responsible for preliminary processing of the user's input. The user input is passed to the knowledge base query module, and the preliminary query results are obtained by querying the pre-built knowledge base. The construction of the knowledge base includes steps such as entity discovery, coreference resolution, and relationship extraction. These steps ensure that the knowledge base can contain structured information, such as triples (subject, predicate, object), to provide support for subsequent queries and answers. Next, the system enters the question clarification module. Here, the Filter Agent uses the text understanding and reasoning capabilities of the LLM to delete unreasonable or conflicting query results with the user input, and merge entries with the same meaning. If there are still multiple possible answers after filtering, the system will pass these results to the question clarification dialogue module. The Questioner Agent will infer the unclear parts of the user input and ask the user key questions to further confirm the user's needs. This process interacts with the user's real-time feedback, allowing the system to gradually converge to the most accurate answer. After the user replies, the ReplyJudge The agent analyzes the reply to determine whether the user has raised a new question. If the user raises a new question, the system will integrate it with the previous content as a new user input and recursively call the LLM single question answering module. This recursive calling mechanism enables the system to handle complex multi-round conversations and maintain context consistency. If the user's reply is an answer to a previous question, the Merge Agent will summarize and generalize the content of multiple information sources to generate a clearer user input and pass it to the answering module for final processing. The answering module is responsible for generating the final customer service reply. The LLM The agent generates a detailed answer by combining the query results in the knowledge base with the embedded model knowledge, and outputs it to the user in the tone of customer service. The system will also store the answer in the context module for the user to refer to in the next conversation. The intelligent customer service system can not only handle multiple questions of the user continuously, but also provide more services based on the user's choice. If the user chooses to continue asking questions after the answer, the system will integrate the previous question and answer content with the new question through the context module, and pass it to the LLM single question answering module for processing again. If the user chooses to transfer to manual customer service or end the conversation, the system will make the corresponding switch. This series of steps ensures that the intelligent customer service system can provide efficient, accurate and personalized services, and can continuously optimize its answering process through continuous interaction with users, thereby improving the user experience.
[0187] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the application should be included in the protection scope of the present application.
Claims
1. A method for implementing intelligent customer service based on a large language model, characterized in that: The following steps are involved: Get user input and pass it to the LLM single question answering module; The LLM single question answering module processes the user input and outputs the answer to the question; If the user chooses to continue asking questions, the question and answer will be stored in the context module, and the user can continue to enter questions; If the user chooses to be transferred to a manual customer service representative, the user will be transferred to a manual customer service representative. If the user chooses that they have no other questions, they will enter the feedback module; The context module synthesizes and integrates the user input and the previous text, and the integrated user question will be passed to the LLM single question answering module.
2. According to claim 1, a method for realizing intelligent customer service based on a large language model is characterized in that: The processing flow of the LLM single question answering module includes the following steps: Pass the user input to the knowledge base query module to obtain the query results; The query results are passed to the question clarification module along with the user input; The question clarification module outputs the clarified user input, which is then passed to the knowledge base query module for a second query; The second query result and the clarified user input are passed to the answer module, which outputs the answer to the user's question.
3. The method for realizing intelligent customer service based on a large language model according to claim 2, characterized in that: The processing flow of the problem clarification module includes the following steps: The knowledge base query results and user input are passed to the Filter Agent. The Filter Agent uses the text understanding and reasoning capabilities of the LLM to delete the unreasonable entries or entries that conflict with the user input in the knowledge base query results according to the user input content, merge the entries with the same meaning, and output the filtered query results. If the number of filtered query results is greater than 1, they are passed to the question clarification dialogue module together with the user input. The question clarification dialogue module passes the relatively clear user input and the filtered query results to the Filter Agent to filter the query results again until the number of remaining query results is 1 or 0; If the number of filtered query results is less than or equal to 1, the remaining query results and user input are passed to MergeAgent. MergeAgent summarizes and integrates multiple pieces of information to make the user input question more specific and clear, and outputs the clarified user input.
4. The method for realizing intelligent customer service based on a large language model according to claim 3, characterized in that: The processing flow of the question-clarifying dialogue module includes the following steps: The filtered query results and user input are passed to Questioner Agent. Based on the logical relationship between the query results and the user input, Questioner Agent infers the unclear content in the user input and selects the most critical question to ask the user. The user's reply is passed to ReplyJudgeAgent, which determines whether the user's reply is an answer to the previous question from Questioner Agent or a new question. If the user asks a new question, the user input, Agent query, and user response are combined into a new user input, and the LLM single question answering module is recursively called; If the user's answer is obtained, the user input, Agent query, and user reply are passed to the Merge Agent. The MergeAgent outputs a relatively clear user input, which is merged with the filtered query results as the output of the question clarification dialogue module.
5. The method for realizing intelligent customer service based on a large language model according to claim 3, characterized in that: The FilterAgent uses the following formula to delete the query results that conflict with the user input: Rf={ri∈RSim(ri,Q)<θ1} Among them, R f represents the filtered query result, R is the initial query result set, Ri is an item in the query result, Q is the user input, Sim(ri,Q) is the similarity function between the query result and the user input, and θ1 is the preset similarity threshold.
6. The method for realizing intelligent customer service based on a large language model according to claim 3, characterized in that: The Merge Agent uses the following formula to merge multiple pieces of information into specific and clear user inputs expressed by the following formula: Q m =Merge(Q P ,R f ) Among them, Q m Indicates the user input after clarification, Q p Represents the previous user input paragraph, R f is the filtered query result set, and Merge is the merge function.
7. The method for realizing intelligent customer service based on a large language model according to claim 4, characterized in that: The Questioner Agent uses the following formula to determine the key question in the user input: qk=argmax q Uncertainty(q,R f ) Among them, qk represents the most critical question at the moment, q is the set of all possible questions, and Uncertainty is the uncertainty function, which is used to measure the ambiguity of each question relative to the filtered query results.
8. The method for realizing intelligent customer service based on a large language model according to claim 4, characterized in that: The Reply Judge Agent uses the following formula to determine the type of user reply: T r =Classify(R u) Among them, T r Indicates the type of user reply, Classify is the classification function, R u The user's reply content.
9. The method for realizing intelligent customer service based on a large language model according to claim 1, characterized in that: The final solution output by the LLM single-question solution module is determined by the following formula: A=Answer(Q m ,R m ) A is the final solution, Answer is the solution function, and R m is the final query result set obtained by querying the knowledge base.
Citation Information
Patent Citations
Information retrieval method and device, medium and electronic equipment
CN111046154A
Processing method for intelligent customer service dialogue and related product
CN117312521A
Knowledge question-answering method based on big data and related products
CN117828048A
Medical question answering system based on large language model
CN117851558A
Model training method and question answering method for question answering system
CN118093841A
