Intelligent customer service system based on large language model and reinforcement learning

Abstract

The invention discloses an intelligent customer service system based on a large language model and reinforcement learning, which can be quickly established under the conditions of few or no session samples and limited knowledge base documents, and continuously improves the question response level in the continuous interaction process with customers and customer service. According to the technical scheme, the system comprises a customer service session management subsystem which is used for realizing interaction between customer service agent personnel and customers; the intelligent answering subsystem is used for generating recommended answering content and improving the answering level through reinforcement learning according to the message actually sent to the customer by the customer service staff and the feedback of the customer; the knowledge management subsystem is used for maintaining knowledge required in a customer service response process; and the monitoring display subsystem is used for displaying session experience and new knowledge generated in the operation process of the system, and recording the quality of recommended response content generated by the whole system and the response quality of each customer service seat personnel.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence applications, specifically to an intelligent customer service system based on large language models and reinforcement learning. Background Technology

[0002] With the rapid development of artificial intelligence technology, many enterprises have begun to use large language models to build intelligent customer service systems. Large language models, or simply large models, refer to machine learning models with a large number of parameters and complex computational structures. These models are typically built from deep neural networks and have billions or even hundreds of billions of parameters. As the training data and parameters of the model continue to expand until a certain critical scale is reached, it exhibits some unpredictable and more complex capabilities and characteristics. The model can automatically learn from the original training data and discover new, higher-level features and patterns; this ability is called "emergent capability." Through the natural language understanding capabilities emerging from large models, humans can already interact with them using prompts. When humans give instructions to the large model, it can provide the desired response according to the requirements of natural language.

[0003] The natural language understanding capabilities of large language models have been proven to partially replace human agents in answering customer questions. However, traditional intelligent customer service systems based on large models require enterprises to have a relatively complete document knowledge base and a large amount of historical conversation data as foundational data. Construction methods typically include fine-tuning the model itself through training and providing the model with relevant knowledge beyond the training data through Retrieval-Augmented Generation (RAG) technology. RAG refers to optimizing the output of large language models so that they can reference authoritative knowledge bases outside the training data source before generating responses. Regardless of the type of large model, it is trained on a large amount of real-world corpus, thus suffering from knowledge limitations and the inability to update knowledge in a timely manner. RAG technology, through a "retrieval-generation" model, retrieves relevant knowledge before the large model answers the question and feeds it into the large model as a prompt, enabling the large model to answer user questions more effectively based on this reference knowledge.

[0004] Both approaches have their limitations. Training the model can expand its inherent knowledge, enabling it to answer user questions more directly, but this is costly, time-consuming, and difficult to update. Retrieval-enhanced generation techniques partially solve the knowledge update problem, but they heavily rely on information within the knowledge base, requiring significant manpower to organize and correct documents. Furthermore, both of these solutions reach their system capacity limits after system construction, and subsequent knowledge base updates still require substantial manpower. Summary of the Invention

[0005] The following provides a brief overview of one or more aspects to offer a basic understanding of them. This overview is not an exhaustive summary of all conceived aspects, nor is it intended to identify key or decisive elements of all aspects, nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form to prepare for the more detailed descriptions that follow.

[0006] The purpose of this invention is to solve the aforementioned problems by providing an intelligent customer service system based on a large language model and reinforcement learning. This system can be rapidly built even with limited or no conversation samples and a limited knowledge base. Furthermore, it continuously improves its question-answering capabilities through ongoing interactions with customers and customer service representatives, ultimately reaching or even surpassing the performance of ordinary customer service agents. In addition, the system can help enterprises quickly expand their question-answering knowledge base and assist customer service managers in addressing issues such as outdated documentation and inadequate training for customer service agents.

[0007] The technical solution of this invention is as follows: This invention discloses an intelligent customer service system based on a large language model and reinforcement learning, the system comprising:

[0008] The customer service conversation management subsystem is used to enable interaction between customer service agents and customers;

[0009] The intelligent response subsystem generates recommended response content and improves response quality through reinforcement learning based on the messages actually sent to customers by customer service personnel and customer feedback.

[0010] The knowledge management subsystem is used to maintain the knowledge required for customer service responses; and

[0011] The monitoring and display subsystem is used to display the conversation experience and new knowledge generated by the system during operation, and to record the quality of the recommended response content generated by the entire system and the quality of the responses of each customer service agent.

[0012] The system operates through the following process:

[0013] Step 1: Obtain customer sessions from the customer service session management subsystem;

[0014] Step 2: The intelligent question-answering subsystem, in conjunction with the knowledge base managed by the knowledge management subsystem, generates corresponding recommended response content for customer service agents based on the customer conversation, and sends the corresponding notes to the customer service conversation management subsystem.

[0015] Step 3: Customer service representatives edit the recommended responses generated by the intelligent question-and-answer subsystem and send them to the customer.

[0016] Step 4: The intelligent question-and-answer subsystem obtains customer feedback on the response content;

[0017] Step 5: The intelligent question-and-answer subsystem, monitoring and display subsystem, and knowledge management subsystem evaluate the effectiveness based on customer feedback.

[0018] Step 6: Summarize and generalize the experience and knowledge gained from answering customer questions through the knowledge management subsystem, which will be used to enhance the response generation capability of the intelligent question-and-answer subsystem in Step 2.

[0019] According to an embodiment of the intelligent customer service system based on a large language model and reinforcement learning according to the present invention, step 2 further includes:

[0020] Step 2-1: The message response judgment module in the intelligent question-and-answer subsystem determines whether a response is required for the customer message. If so, proceed to step 2-2; otherwise, proceed to step 2-7.

[0021] Step 2-2: The question rewriting module of the knowledge management subsystem rewrites and optimizes the questions in the customer conversation messages by generating similar questions and generating simulated answers;

[0022] Steps 2-3: The document retrieval module of the knowledge management subsystem retrieves the most relevant results from the document knowledge base and the question-answering knowledge base respectively through text embedding retrieval.

[0023] Steps 2-4: The relevance ranking module of the knowledge management subsystem re-ranks the knowledge in the search results according to the degree of relevance;

[0024] Steps 2-5: The response generation module of the intelligent question-answering subsystem and the question experience knowledge reading and writing module of the knowledge management subsystem generate response content based on experience prompts and relevant knowledge;

[0025] Steps 2-6: The response generation module of the intelligent question-answering subsystem and the fact conflict knowledge reading and writing module of the knowledge management subsystem use the fact conflict knowledge base to perform similarity matching between each historical conflict knowledge and the response content currently generated by the big language model. If a resolved fact conflict knowledge is matched, the original response sentence is corrected through the big language model and a note is generated. Then, steps 2-8 are executed.

[0026] Step 2-7: The response sending module of the intelligent question-and-answer subsystem returns an empty response and adds the reason why no reply is needed as a note, and then proceeds to step 2-8;

[0027] Steps 2-8: The response sending module of the intelligent question-and-answer subsystem returns the recommended response content and remarks generated in the above steps to the customer service session management subsystem, which then displays the recommended response content and remarks to the customer service agents.

[0028] According to an embodiment of the intelligent customer service system based on large language model and reinforcement learning of the present invention, in step 3, the customer service agent may choose not to modify the recommended response content and send it directly to the customer through the message sending module of the customer service session management subsystem, or may completely rewrite the response content in the response editing module of the customer service session management subsystem and send it to the customer.

[0029] According to an embodiment of the intelligent customer service system based on large language model and reinforcement learning of the present invention, in step 4, if the customer does not reply within a set time, an empty reply is used to enter the subsequent effect evaluation process; otherwise, the customer reply message received by the message receiving module is used to enter the subsequent effect evaluation process, and the customer reply message is sent to the response generation module, which determines whether a reply is needed.

[0030] According to an embodiment of the intelligent customer service system based on a large language model and reinforcement learning according to the present invention, step 5 further includes:

[0031] Step 5-1: The effectiveness evaluation module of the intelligent question-answering subsystem provides feedback scores based on completion and sentiment.

[0032] Step 5-2: The indicator recording module of the monitoring and display subsystem records the feedback score as the customer service agent's score, and determines whether to execute step 5-3 or step 5-4 based on the customer service agent's score result.

[0033] Step 5-3: Execute the positive evaluation sub-process;

[0034] Step 5-4: The failure experience analysis module of the intelligent question-answering subsystem and the problem experience knowledge reading and writing module of the knowledge management subsystem analyze customer feedback to generate conversation experience.

[0035] According to an embodiment of the intelligent customer service system based on a large language model and reinforcement learning according to the present invention, step 5-3 further includes:

[0036] Step 5-3-1: The similarity calculation module of the intelligent question answering subsystem calculates the similarity between the generated response and the actual response;

[0037] Step 5-3-2: The indicator recording module of the monitoring and display subsystem scores the generated effect based on the degree of similarity;

[0038] Step 5-3-3: The clause splitting module of the intelligent question-answering subsystem splits both the generated response and the actual response into clauses;

[0039] Step 5-3-4: The similarity calculation module of the intelligent question answering subsystem matches each clause: the similarity algorithm is used again to match each clause;

[0040] Step 5-3-5: The effect evaluation module of the intelligent question-answering subsystem performs the following processing: If there are more actual response clauses than generated response clauses, execute the process of adding question-answer knowledge pairs;

[0041] Step 5-3-6: The following processing is implemented by the question experience generation module of the intelligent question answering subsystem, the question standardization module of the knowledge management subsystem, and the question experience knowledge reading and writing module of the knowledge management subsystem: If there are more generated response clauses than actual response clauses, they will be deleted when a similar situation is encountered again;

[0042] Step 5-3-7: The fact conflict analysis module of the intelligent question answering subsystem performs the following processing: If the clauses can match but are not completely identical, then the two clauses are processed separately according to whether the facts of the two clauses match as determined by the large language model: If the facts match, the differences between the two clauses are analyzed through the large language model to generate an experience for the current problem; if the facts do not match, a fact conflict knowledge base is established for analysis.

[0043] According to an embodiment of the intelligent customer service system based on a large language model and reinforcement learning according to the present invention, step 5-3-5 further includes:

[0044] Step 5-3-5-1: The document retrieval module of the knowledge management subsystem searches whether the knowledge described in the clause exists in the document knowledge base;

[0045] Step 5-3-5-2: The fact conflict analysis module of the intelligent question answering subsystem performs the following processing: If similar information exists in the knowledge base, the large language model is used to determine whether the information in the knowledge base is consistent with the information in the customer service response;

[0046] Step 5-3-5-3: If the previous step finds that the input knowledge is inconsistent with the knowledge in the document knowledge base, the fact conflict analysis sub-process of step 5-3-7 will be called.

[0047] Step 5-3-5-4: The question standardization module and the question-answer pair knowledge reading and writing module of the knowledge management subsystem standardize the question-answer pairs and add them to the question-answer pair knowledge base;

[0048] Step 5-3-5-5: The question-and-answer pair knowledge reading and writing module of the knowledge management subsystem stores the question-and-answer pairs into a temporary question-and-answer pair knowledge base;

[0049] Step 5-3-5-6: The question-and-answer knowledge reading and writing module of the knowledge management subsystem performs the following processing: obtain the number of times similar answers appear in the temporary question-and-answer knowledge base. If the number of appearances is greater than the threshold, the knowledge is considered to be commonly used customer service knowledge, and the knowledge will be added to the question-and-answer knowledge base in step 5-3-5-4. Otherwise, the process ends.

[0050] According to an embodiment of the intelligent customer service system based on a large language model and reinforcement learning according to the present invention, step 5-3-7 further includes:

[0051] Step 5-3-7-1: The fact conflict knowledge reading and writing module of the knowledge management subsystem stores different factual descriptions of the same issue into the conflict knowledge base and automatically assigns an auto-incrementing conflict event number;

[0052] Step 5-3-7-2: The fact conflict early warning module of the monitoring and display subsystem sends a conflict early warning message to remind customer service agent management personnel;

[0053] Step 5-3-7-3: The customer service representative manager marks the correct facts;

[0054] Step 5-3-7-4: The fact conflict knowledge reading and writing module of the knowledge management subsystem marks the fact conflict as resolved and records the correct information;

[0055] Step 5-3-7-5: The document knowledge reading and writing module, the question-and-answer pair knowledge reading and writing module of the knowledge management subsystem, and the similarity calculation module of the intelligent question-and-answer subsystem update the corresponding document knowledge base and question-and-answer pair knowledge base.

[0056] According to an embodiment of the intelligent customer service system based on a large language model and reinforcement learning according to the present invention, step 6 further includes:

[0057] Step 6-1: The problem experience knowledge reading and writing module of the knowledge management subsystem stores the session experience;

[0058] Step 6-2: The question experience is standardized and stored by the question standardization module of the knowledge management subsystem, the question experience knowledge reading and writing module of the knowledge management subsystem, and the similarity calculation module of the intelligent question answering subsystem;

[0059] Step 6-3: Extract general experience from the similarity calculation module of the intelligent question answering subsystem and the question experience knowledge reading and writing module of the knowledge management subsystem;

[0060] Step 6-4: The problem experience is summarized by the problem experience knowledge reading and writing module of the knowledge management subsystem.

[0061] Compared with the prior art, the present invention has the following advantages:

[0062] 1. This invention fully utilizes the natural language understanding capabilities of large models through carefully designed prompts, enabling the rapid construction of an entire intelligent customer service system without the need to train the large model.

[0063] 2. This invention uses reinforcement learning (RL) instead of supervised learning methods based on raw corpora, enabling the system to ultimately achieve service results that surpass those of customer service agents.

[0064] 3. This invention uses the actual customer service response content and customer response content as feedback for reinforcement learning, rather than using only customer service response content as training samples as in traditional solutions.

[0065] 4. This invention enhances the model strategy through textual experience (rather than fine-tuning model parameters), which can significantly improve the quality of responses generated by large models and is highly interpretable. The generated experience can also be provided to customer service agents as a reference.

[0066] 5. This invention can automatically and quickly expand the knowledge base content (including: question-and-answer pair knowledge base, conflict fact knowledge base, and problem experience knowledge base) without additional manual organization, and can improve customer service agent management personnel to supplement the original document knowledge base based on failure experience.

[0067] 6. This invention can help customer service agent managers discover knowledge conflicts (problems with untimely knowledge updates), which is something that cannot be achieved through general large model training schemes. At the same time, it can also help customer service agent managers evaluate and manage the performance of customer service agents. Attached Figure Description

[0068] The above-described features and advantages of the present invention will be better understood after reading the following detailed description of embodiments of the present disclosure in conjunction with the accompanying drawings. In the drawings, components are not necessarily drawn to scale, and components having similar related characteristics or features may have the same or similar reference numerals.

[0069] Figure 1 The diagram illustrates a schematic of an embodiment of the intelligent customer service system based on a large language model and reinforcement learning according to the present invention.

[0070] Figure 2 It shows Figure 1 The diagram shows the main flow of the system.

[0071] Figure 3 It shows Figure 2 The diagram shows the operation of the sub-process for generating recommended responses in the main system flow.

[0072] Figure 4 It shows Figure 2 The diagram shows the operation of the sub-process for effect evaluation in the main process of the system.

[0073] Figure 5 It shows Figure 2 The diagram shows the operation of the positive evaluation sub-process in the main system flow.

[0074] Figure 6 It shows Figure 2 The diagram shows the operation of the sub-process of adding question-and-answer knowledge pairs in the main process of the system.

[0075] Figure 7 It shows Figure 2 The diagram shows the operation of the fact conflict analysis sub-process in the main system flow.

[0076] Figure 8 It shows Figure 2 The diagram shows the operation of the sub-process of summarizing session experience in the main system flow. Detailed Implementation

[0077] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should be noted that the aspects described below with reference to the accompanying drawings and specific embodiments are merely exemplary and should not be construed as limiting the scope of protection of the present invention in any way.

[0078] Figure 1 This illustrates the principle of an embodiment of the intelligent customer service system based on a large language model and reinforcement learning according to the present invention. Please refer to [link / reference]. Figure 1 The system in this embodiment includes: a customer service conversation management subsystem, an intelligent response subsystem, a knowledge management subsystem, and a monitoring and display subsystem.

[0079] The customer service conversation management subsystem is used to realize the interaction between customer service agents and customers, including a customer message receiving module, a recommended response display module, a response editing module, a message sending module, and a conversation storage module.

[0080] The message receiving module of the customer service conversation management subsystem is configured to provide an entry point for customer inquiries, typically a web page, mobile app, etc.

[0081] The message sending module of the customer service conversation management subsystem is configured to display customer service replies on the user interface, typically a web page, mobile app, etc.

[0082] The recommended response display module of the customer service conversation management subsystem is configured to display the recommended response content and remarks generated by the system to customer service agents. This content is only available to internal customer service agents within the enterprise.

[0083] The response editing module of the customer service conversation management subsystem is configured to provide an interface for customer service agents to edit responses.

[0084] The session storage module of the customer service session management subsystem is configured to store sessions using a traditional relational database (such as MySQL).

[0085] The intelligent question-and-answer subsystem generates recommended responses and continuously improves its response level through reinforcement learning based on the messages actually sent to customers by customer service personnel and customer feedback, so as to achieve an effect close to or even surpassing that of customer service agents.

[0086] The response generation module of the intelligent question-answering subsystem is configured to: execute Figure 3 The sub-process for generating recommended responses is shown.

[0087] The response sending module of the intelligent question-and-answer subsystem is configured to send the generated recommended response and related notes to the customer service session management subsystem, which can be done via HTTP protocol or RPC.

[0088] The message response judgment module of the intelligent question-answering subsystem is configured as follows: to judge whether a customer message needs to be replied to. For example, the Doubao-Seed-1.6 large language model is used for judgment. The input instruction (Prompt) is: "The following is the dialogue content between the customer and the customer service: {1}. Your task is to judge whether the last message of the customer needs to be replied to by you. You only need to reply to the customer's inquiry about the company's business. If the customer's message is an inquiry about the company's business, return yes. If the customer's message is feedback on the previous conversation without asking new questions or is just small talk, return no and explain why no reply is needed. The return is in JSON format: {"need_reply":[1],"note":[2]}, where [1] is a boolean type, true means that a reply is needed, false means that no reply is needed, and [2] is the note content, explaining why no reply is needed." This embodiment can also use other large models or models that are specifically pre-trained for this type of task.

[0089] The performance evaluation module of the intelligent question-answering subsystem is configured as follows: Execute Figure 4 The effect evaluation sub-process is shown below. The completion rate and sentiment scores are both completed by the Doubao-Seed-1.6 model, with the following instructions: "The following is a dialogue between a customer and customer service. Your task is to determine whether the problem in the original conversation has been resolved based on the customer feedback. Please return a value between -1 and 1, where 1 represents resolved, -1 represents unresolved, and 0 represents indeterminate. Note that this value only needs to determine whether the problem in the original conversation has been resolved. If a new problem is raised in subsequent conversations that did not appear in the original conversation, then the problem in the original conversation is considered resolved. The original conversation is: {1}, and the customer feedback is: {2}. Only one value needs to be returned." Here, {1} represents the original conversation content (up to the customer service representative's last response), and {2} represents the customer's subsequent feedback. "The following is a dialogue between a customer and customer service. Your task is to determine the customer's emotional inclination based on the 'Customer Feedback' section. Please return a value between -1 and 1, where 1 represents customer satisfaction, -1 represents customer dissatisfaction, and 0 represents indeterminacy. Note that the original dialogue is for reference only; you only need to determine the customer's emotional inclination in the 'Customer Feedback' section. The 'Original Dialogue' is: {1}, and the 'Customer Feedback' is: {2}. You only need to return a single value." Here, {1} represents the original dialogue content (up to the customer service representative's last response), and {2} represents the customer's subsequent feedback.

[0090] The similarity calculation module of the intelligent question answering subsystem is configured as follows: For the similarity calculation of two single sentences, Levenshtein distance can be used. Assuming lev(s1,s2) represents the Levenshtein distance between two sentences, the similarity calculation formula is: 1 - (lev(s1,s2) / max(len(s1),len(s2))). For retrieval tasks, the Sentence-BERT model can be used to convert all sentences into sentence vectors, and then the cosine similarity between the vectors can be calculated.

[0091] The clause splitting module of the intelligent question-answering subsystem is configured as follows: It uses the Doubao-Seed-1.6 large-scale model for judgment, and the input prompt is: "Please help me split the input sentence into multiple independent 'fact clauses.' Each clause must correspond to a core fact in the original sentence (i.e., an independent, verifiable unit of information), and the clauses must not repeat or contain redundant information. At the same time, the integrity of the clauses must be ensured, and related facts cannot be separated. The sentence to be split is: {1}." Here, {1} is the clause to be split. This embodiment can also use other large-scale language models or improve the model's accuracy through pre-training specifically for this type of task, but even without additional training, large-scale language models can still perform this type of task well.

[0092] The fact conflict analysis module of the intelligent question-answering subsystem is configured as follows: It uses the Doubao-Seed-1.6 large model for judgment. The input prompt is: "Please help me analyze whether there are any conflicts in the facts described in the following two sentences. Please note that the two sentences are answers to the same question; therefore, if there are inconsistencies, it indicates a conflict. If there is a conflict, please generate two different descriptions, corresponding to the two sides of the facts respectively. If there is no conflict, return 'no conflict'. The two sentences to be judged for conflict are:"

[0093] Sentence 1: {1}

[0094] Sentence 2: {2}”

[0095] Here, {1} and {2} represent the two input statements that need to be judged to have conflicting facts. Additionally, multiple examples can be added to the instructions to improve the model's accuracy, or other models or models specifically pre-trained for this type of task can be used. However, even without additional training, the general-purpose large language model can still perform this type of task well.

[0096] The question experience generation module of the intelligent question answering subsystem is configured to use the Doubao-Seed-1.6 large model.

[0097] The failure experience analysis module of the intelligent question-answering subsystem is configured as follows: It uses the Doubao-Seed-1.6 large model, and the input prompt is: "The original conversation between the customer and customer service is: {1}, the customer's last question is: {2}, the customer service's response is: {3}, and the customer's feedback is: {4}. Based on the complete conversation content, analyze the customer service's response from the customer's feedback information to identify areas for improvement regarding the customer's last question. Summarize the improvements concisely." Here, {1} represents all historical conversation messages before the customer's last question, {2} represents the customer's last question, {3} represents the customer service's response, and {4} represents the customer's feedback.

[0098] The knowledge management subsystem is used to maintain the knowledge required in the customer service response process. It is an auxiliary system to the intelligent question-and-answer subsystem. The knowledge base managed by the knowledge management subsystem includes a document knowledge base, a question-and-answer pair knowledge base, a fact conflict knowledge base, and a problem experience knowledge base. Among them, the question-and-answer pair knowledge base contains formal question-and-answer pair knowledge and temporary question-and-answer pair knowledge, the fact conflict knowledge base contains resolved fact conflicts and unresolved fact conflicts, and the problem experience knowledge base contains positive problem experience and failed problem experience.

[0099] The problem experience knowledge read / write module of the knowledge management subsystem is configured to store problem experience knowledge and provide read / write methods. Text vectors can be stored using Milvus or ElasticSearch vector databases. The original text information can be associated with metadata in the vector database, and an additional copy of the original text information is stored in a relational database (e.g., MySQL).

[0100] The question rewriting module of the knowledge management subsystem is configured as follows: Generating different wording of the question: This is accomplished using the Doubao-Seed-1.6 model. The input instruction (Prompt) is: "Please generate two different wordings based on the customer's question: {1}, but the meaning must not be changed." Generating simulated responses: This is also accomplished using the Doubao-Seed-1.6 model. The input instruction (Prompt) is: "Please generate a simulated response based on the customer's question: {1}. The simulated response does not need to rely on any factual basis; it only needs to conform to the logic of answering the question in form."

[0101] The document retrieval module of the knowledge management subsystem is configured to retrieve the k documents most similar to the input text using the similarity retrieval function of the vector database.

[0102] The relevance ranking module of the knowledge management subsystem is configured to re-rank the retrieved results and the input text using either the bge-reranker model or the cohere-rerank model.

[0103] The question-and-answer pair knowledge reading and writing module of the knowledge management subsystem is configured to store question-and-answer pair knowledge and provide reading and writing methods. Text vectors can be stored using Milvus or ElasticSearch vector databases. The original text information can be associated with metadata in the vector database, and an additional copy of the original text information is stored in a relational database (e.g., MySQL).

[0104] The document knowledge reading and writing module of the knowledge management subsystem is configured to store document knowledge and provide reading and writing methods. Text vectors can be stored using Milvus or ElasticSearch vector databases. The original text information can be associated with metadata in the vector database, and an additional copy of the original text information is stored in a relational database (e.g., MySQL).

[0105] The fact conflict knowledge read / write module of the knowledge management subsystem is configured to store fact conflict knowledge and provide read / write methods. Text vectors can be stored using Milvus or ElasticSearch vector databases. The original text information can be associated with metadata in the vector database, and an additional copy of the original text information is stored in a relational database (e.g., MySQL).

[0106] The question standardization module of the knowledge management subsystem is configured as follows: The goal of the question standardization module is to aggregate similar questions together as much as possible, so that diverse user questions can be accurately matched with a small number of corresponding standard questions, so as to better provide users with the answers they want. The Doubao-Seed-1.6 large model is used to complete the task. The input instruction is: "The dialogue content between the customer and the customer service is: {1}, and the customer's last question is: {2}. Please identify the intent of the customer's last question and summarize the customer's question in a short sentence and extract the keywords in the customer's intent. Note that the summarized question should not contain user privacy content, but should cover the customer's original intent as much as possible. The returned format is JSON: {"Question Summary":[1],"Keywords":[2]}, where [1] is the generated question summary and [2] is an array of keywords."

[0107] The monitoring and display subsystem is used to display the conversation experience and new knowledge generated during system operation, and to record the quality of the recommended response content generated by the entire system as well as the quality of the responses of each customer service agent.

[0108] The system in this embodiment reads, indexes, and stores existing knowledge documents through the document knowledge reading and writing module in the knowledge management subsystem, constructing a document knowledge base as a reference for the intelligent question-answering subsystem to generate recommended responses. Retrieval-enhanced generation (RAG) technology is generally used here, and can be implemented using langchain, llamaIndex, or by writing a custom document encoder; no specific method is limited here.

[0109] The more complete the initial document knowledge base of the system in this embodiment, the better the initial experience of the system. However, even if there is no knowledge document base in the initial stage, the system can still run normally and will not affect the final effect that the system can achieve.

[0110] The model metric display module of the monitoring and display subsystem is configured to display the effect metrics of the current system's response generation, including the adoption rate of the generated response (a response whose similarity to the generated response is greater than a threshold is considered adopted) and the overall score of the model.

[0111] The customer service agent metrics display module of the monitoring and display subsystem is configured to display the number of conversations handled by each customer service agent on that day, response performance scores, etc.

[0112] The indicator recording module of the monitoring and display subsystem is configured to record customer service agent ratings and model-generated response ratings, and store them using a relational database (such as MySQL).

[0113] The fact conflict early warning module of the monitoring and display subsystem is configured to send early warning information by calling an email or instant messaging tool (such as DingTalk) interface. The information will include a link to the fact conflict handling interface.

[0114] The fact conflict display module of the monitoring and display subsystem is configured to display the fact conflict ID number, two versions of the conflicting fact description, and a blank input box. It provides customer service managers with the function of selecting the correct fact or the blank input box. If the blank input box is selected, it means that the customer service manager believes that neither description is correct and the correct fact needs to be entered manually.

[0115] The fact conflict handling module of the monitoring and display subsystem is configured as follows: receive the conflict event ID and the correct facts marked by the operator, call the issue standardization module to obtain the standard issue, store the standard issue and the correct facts in the fact conflict knowledge base and mark the status of the fact conflict as resolved.

[0116] Figure 2 It shows Figure 1 The system's main workflow is shown in the diagram. (Reference) Figure 2 , combined Figure 1 The system modules shown below will now be explained in terms of the steps of the main system operation process.

[0117] Step 1: The system obtains the client session.

[0118] This step is completed by the message receiving function of the customer service conversation management subsystem, which provides an entry point for customers to send inquiry messages through the company's official website or a dedicated customer client application.

[0119] Step 2: The system generates a recommended response.

[0120] This step is completed by the intelligent question-answering subsystem and the knowledge management subsystem. The intelligent question-answering subsystem combines the knowledge base managed by the knowledge management subsystem (including document knowledge base, question-answer pair knowledge base, fact conflict knowledge base, and problem experience knowledge base) to generate recommended response content for customer service agents, and generates corresponding notes information to send to the customer service conversation management subsystem.

[0121] The recommended answers generated by the intelligent question-and-answer subsystem can be edited by customer service agents in the next step to generate a reply, but the notes cannot be edited. Notes typically include: experience summarized from previous responses to similar user questions, factual conflicts encountered previously (information prone to errors in the answer), and links to relevant knowledge base content within the reply.

[0122] Before generating a response, the system first determines whether the content in the customer's conversation requires a reply. If it determines that no reply is needed, the generated recommended response will be empty, and the remarks will explain why the system determined that the message does not need a reply.

[0123] Step 2: Detailed processing Figure 3 As shown, the details are as follows.

[0124] Step 2-1: The message response judgment module in the intelligent question-and-answer subsystem determines whether a response is required for the customer message. If so, proceed to step 2-2; otherwise, proceed to step 2-7.

[0125] Step 2-2: The question rewriting module of the knowledge management subsystem rewrites and optimizes the questions in the customer conversation messages by generating similar questions and generating simulated answers, so as to improve the matching effect in the subsequent retrieval process.

[0126] Steps 2-3: The document retrieval module of the knowledge management subsystem retrieves the most relevant results from the document knowledge base and the question-answering knowledge base respectively using the traditional text embedding retrieval method, for example, the 5 most relevant results.

[0127] Steps 2-4: The relevance ranking module of the knowledge management subsystem reorders the knowledge in the search results according to the degree of relevance in order to reduce noise information and improve the speed and accuracy of subsequent large language model to generate response content.

[0128] Steps 2-5: The response generation module of the intelligent question-and-answer subsystem and the question experience knowledge reading and writing module of the knowledge management subsystem generate response content based on experience prompts and relevant knowledge.

[0129] The specific processing for steps 2-5 is as follows:

[0130] First, historical and general experiences related to the current problem are retrieved and extracted from the experience knowledge base. Then, the original problem, the extracted experience, and the relevant knowledge obtained in the previous step are fed into the large language model as prompts, which then generate recommended responses.

[0131] Large language models can be, for example, the Doubao-Seed-1.6 model, or other large language models such as Deepseek.

[0132] For example, the input prompt might be: "Customer Question: {1}, Based on historical experience, for this type of question, the customer might want: {2}, and should pay attention to the following in the answer: {3}, with the most relevant knowledge fragment being: {4}. Please generate the corresponding answer to the customer's question based on the above content. In the answer, please strictly adhere to the facts in the knowledge fragment and do not fabricate facts." Here, {1} is the customer's original question, {2} and {3} are summaries of experience learned from history for this or general questions, and {4} is the relevant knowledge fragment after being reordered in the previous step. Simultaneously, "Based on historical experience, for this type of question, the customer might want: {2}, and should pay attention to the following in the answer: {3}, with the most relevant knowledge fragment being: {4}." will also be added to the notes returned to the customer service representative.

[0133] Steps 2-6: The response generation module of the intelligent question-answering subsystem and the fact-conflict knowledge reading and writing module of the knowledge management subsystem, using the fact-conflict knowledge base, perform similarity matching between each piece of historical conflict knowledge (whether resolved or unresolved) and the currently generated response content of the large language model. If a resolved fact-conflict knowledge is matched, the original response sentence is corrected through the large language model and a note is generated. Then, steps 2-8 are executed.

[0134] This step improves the accuracy of the large language model's output by limiting the return format and using a one-shot approach. Alternatively, a few-shot approach can be used, which provides multiple samples to further improve the reliability of the returned content.

[0135] Example as follows:

[0136] If the matched knowledge is already resolved, the instruction sent to the Doubao-Seed-1.6 large model is as follows: "The currently generated response content is: "{1}", and the confirmed fact is: "{2}". If there are any discrepancies between the response content and the confirmed fact, please correct them and generate a note. If the information in the response content is correct, do not change the original response; simply add a note. The returned format is:

[0137] The corrected response is: [1]

[0138] Note: [2]”'

[0139] [1] is the corrected response data, and [2] is the added remarks.

[0140] Here is an example:

[0141] The generated response currently states: "Our partner bank is the Shanghai branch of the Industrial and Commercial Bank of China." It has been confirmed that "Our company also partners with the Beijing branch of China Merchants Bank." The response will return:

[0142] Revised response: Our partner bank is China Merchants Bank (Beijing).

[0143] Note: A potentially erroneous fact has been discovered: our partner bank is China Merchants Bank in Beijing, not Industrial and Commercial Bank of China in Shanghai.

[0144] If the matched information is an unresolved conflict, the instruction sent to the Doubao-Seed-1.6 large model is as follows: "The currently generated response content is: "{1}", and the fact that is prone to error is: "{2}". Please generate a note explaining the possible error.

[0145] Here is an example:

[0146] The currently generated response states: "Our partner bank is the Shanghai Industrial and Commercial Bank of China." A potentially error-prone detail is found: "It has not been confirmed whether our company partners with the Beijing China Merchants Bank or the Shanghai Industrial and Commercial Bank of China." The response will return:

[0147] Note: A potentially error-prone fact has been discovered: it has not been confirmed whether our partner bank is China Merchants Bank in Beijing or Industrial and Commercial Bank of China in Shanghai.

[0148] Step 2-7: The response sending module of the intelligent question-and-answer subsystem returns an empty response and adds the reason why no reply is needed as a note, and then proceeds to step 2-8.

[0149] Steps 2-8: The response sending module of the intelligent question-and-answer subsystem returns the recommended response content and remarks generated in the above steps to the customer service session management subsystem, which then displays the recommended response content and remarks to the customer service agents.

[0150] Step 3: The system sends a response.

[0151] Customer service agents edit the recommended responses generated by the intelligent Q&A subsystem and displayed by the recommended response display module before sending them to the customer. Agents can choose to send the recommended responses directly to the customer via the message sending module of the customer service conversation management subsystem without modification, or they can completely rewrite the responses using the response editing module within the customer service conversation management subsystem before sending them to the customer.

[0152] Optionally, before sending the response to the customer, the system can recheck the content for errors or rewrite the response based on previous conversation experience, prompting the user for confirmation before sending it to the customer. To avoid excessive disruption to customer service personnel, this optional function can be enabled after the system's intelligence level reaches a high level.

[0153] Step 4: The system receives feedback.

[0154] This step is implemented by the performance evaluation module of the intelligent question-and-answer subsystem. After the customer service agent sends the response to the customer, the customer usually provides further feedback, or may not reply because the problem has been resolved. If the customer does not reply within 24 hours, an empty reply is used to proceed with the subsequent performance evaluation process. Otherwise, the customer's reply message received by the message receiving module is used to proceed with the subsequent performance evaluation process. The customer's reply message may have raised a new question, so the system will also send the customer's reply message to the response generation module, which will determine whether a reply is needed.

[0155] This process is triggered when a previous response is received or when no response is received after 24 hours. The 24-hour timeout is a configurable option.

[0156] Step 5: The system performs an effectiveness evaluation.

[0157] This step is completed by the performance evaluation module of the intelligent question-answering subsystem. Upon entering this process, the system may receive three types of customer feedback: empty feedback, feedback related to previous questions, and feedback on new topics. The system will assess the completion status of previous questions and the customer's emotional inclination based on the feedback information, score the effectiveness of the response generation and customer service response, summarize experience, improve the question-answer pair knowledge base, and highlight factual conflicts. The specific process will be detailed in subsequent sub-processes. This step is implemented based on the intelligent question-answering subsystem, the monitoring and display subsystem, and the knowledge management subsystem.

[0158] Specific procedures for effect evaluation Figure 4 As shown, the following is a detailed description of it.

[0159] Step 5-1: The performance evaluation module of the intelligent question-answering subsystem provides feedback scores based on completion and sentiment. The evaluation process is as follows:

[0160] a) The large language model is used to determine whether the customer's reply can be considered as an answer to the corresponding question. The large language model will give three results: yes, no, or cannot be determined, with corresponding scores of 1, -1, and 0 points, respectively. In particular, if the feedback is empty (no reply after timeout) or the customer asks a new question, the problem is considered to be solved, and the corresponding score is 1.

[0161] b) The large language model is used to determine the sentiment of the customer's reply message. The large language model will give three results: negative, neutral or positive, with corresponding scores of -1, 0 and 1 points respectively.

[0162] c) Add the above scores together to get the final score for the current feedback, which is a maximum of 2 points and a minimum of -2 points.

[0163] Step 5-2: The indicator recording module of the monitoring and display subsystem records the feedback score as the customer service agent's score: Since the customer feedback message is based on the actual response content of the customer service agent, the feedback score can be used as the score of the customer service agent's current conversation performance. The score can be used as a reference for customer service management personnel when managing the team.

[0164] If the above score is greater than 0, it means that the customer is relatively satisfied or the problem has been resolved. The actual response can be used as the standard answer to evaluate whether the generated recommended content has reached the level of the customer service agent. This will lead to the positive evaluation sub-process in step 5-3 (which will be described in detail later).

[0165] If the above score is less than 0, it means that the customer is not satisfied with the content actually sent by the customer service personnel or the customer's problem has not been resolved. In this case, it is necessary to analyze the customer's feedback to find directions for improvement, i.e., to proceed with step 5-4.

[0166] Step 5-3: Positive evaluation sub-process.

[0167] Step 5-4: The failure experience analysis module of the intelligent question-answering subsystem and the question experience knowledge reading and writing module of the knowledge management subsystem analyze customer feedback to generate conversation experience. All conversation information up to the current point and the customer's last feedback message are provided to the large language model. The language model analyzes whether the customer's last feedback message explicitly points to areas for improvement. If so, a failure experience for that type of question is generated. The failure experiences obtained here may be issues that customer service agents and the intelligent customer service system cannot resolve (potentially requiring improvements to product quality or functionality), and therefore will be stored separately in the failure experience knowledge base for subsequent analysis by customer service managers.

[0168] The positive evaluation sub-process of step 5-3 above is referenced. Figure 5 As shown, the details of each process are described below.

[0169] Step 5-3-1: The similarity calculation module of the intelligent question-answering subsystem calculates the similarity between the generated response and the actual response: calculate the similarity between the actual response content of the customer service and the recommended response content generated by the system, and obtain a similarity value between 0 and 1. The larger the value, the more similar the response content is.

[0170] Step 5-3-2: The indicator recording module of the monitoring and display subsystem scores the generated response based on similarity. If the similarity between the generated response and the actual response exceeds a threshold (generally set at around 0.8-0.9 to ensure sufficient similarity), the system-generated recommended response will receive the same score as the actual response. Otherwise, it will receive a score of 0. If the generated response is identical to the actual response, no further improvement is needed, and the positive evaluation sub-process ends. Otherwise, proceed to the next step.

[0171] Step 5-3-3: The clause splitting module of the intelligent question answering subsystem splits both the generated response and the actual response into clauses: the large language model is called to split both the generated response and the actual response into clauses.

[0172] Step 5-3-4: The similarity calculation module of the intelligent question answering subsystem matches each clause: The similarity algorithm is used again to match each clause in order to determine whether all clauses in the actual response sentence are mentioned in the recommended response sentence, and whether any clauses in the recommended response sentence do not appear in the actual response sentence.

[0173] Step 5-3-5: The effect evaluation module of the intelligent question-answering subsystem performs the following processing: If there are more actual response clauses than generated response clauses, it indicates that the generated recommended response content has missing information. In this case, it is necessary to execute the question-answering knowledge pair sub-process. The detailed steps of this question-answering knowledge pair sub-process will be described later.

[0174] Step 5-3-6: The following processing is implemented by the question experience generation module of the intelligent question answering subsystem, the question standardization module of the knowledge management subsystem, and the question experience knowledge reading and writing module of the knowledge management subsystem: If there are more generated response clauses than actual response clauses, it indicates that the large language model has generated redundant information, which should be deleted when a similar situation is encountered again. Therefore, the system will generate an experience for this type of question, in the form of a tuple {question, experience}, where the question is the standardized result of the customer's original question, and the experience is "When the customer inquires about this question, there is no need to reply with information about {1}.", where {1} is a summary of redundant information generated by the Doubao-Seed-1.6 large model, and then stored in the experience knowledge base.

[0175] Step 5-3-7: The fact conflict analysis module of the intelligent question answering subsystem performs the following processing: If the clauses can match but are not completely identical, it means that the information generated by the large language model is valuable, but there is still room for optimization in the expression or some key information is biased. The two situations will be processed separately.

[0176] Determine whether the facts stated in the two clauses match using a large language model:

[0177] The first scenario: If the facts match, then it is generally a problem of expression. By analyzing the differences between the two sentences using a large language model, an experience can be generated to address the problem.

[0178] The second scenario: If the facts are inconsistent, it could be due to an outdated knowledge base, incorrect information provided by customer service, or a misleading model. Both scenarios can be resolved by establishing a fact conflict knowledge base. The fact conflict analysis sub-process will be detailed later.

[0179] The process of adding question-and-answer knowledge pairs in step 5-3-5 is referenced. Figure 6 As shown.

[0180] Step 5-3-5-1: The document retrieval module of the knowledge management subsystem performs the following processing: Checking if the knowledge described in the clause exists in the document knowledge base: The knowledge in the currently input question-answer pair is not mentioned in the system-generated response content. Therefore, it may be knowledge that does not exist in the document knowledge base, or it may exist in the document knowledge base but was not retrieved correctly due to the previous question-based retrieval. Therefore, this step first checks whether similar information exists in the document base.

[0181] Step 5-3-5-2: The fact conflict analysis module of the intelligent question-answering subsystem performs the following processing: If similar information exists in the knowledge base, the large language model is used to determine whether the information in the knowledge base is consistent with the information in the customer service response. This step is included because the previous step, when searching for similar documents, might retrieve content with very similar expressions but completely opposite semantics.

[0182] Step 5-3-5-3: If the previous step found that the input knowledge is inconsistent with the knowledge in the document knowledge base, it must be that there is an error in the information on one side, and the fact conflict analysis sub-process will be called.

[0183] Step 5-3-5-4: The question standardization module and the question-answer pair knowledge reading and writing module of the knowledge management subsystem shall perform the following processing: Otherwise, it means that the knowledge already exists in the knowledge base, but the relevant answer cannot be accurately matched by the question. Therefore, the question-answer pair shall be standardized and added to the question-answer pair knowledge base through this step.

[0184] Step 5-3-5-5: The question-and-answer knowledge reading and writing module of the knowledge management subsystem performs the following processing: If the relevant knowledge cannot be retrieved from the knowledge base in step 5-3-5-1, it is possible that the knowledge is not recorded in the document library and is additional knowledge possessed by the customer service representative. To avoid information errors by the customer service representative, the question-and-answer pair is first stored in a temporary question-and-answer knowledge base.

[0185] Step 5-3-5-6: The question-and-answer knowledge reading and writing module of the knowledge management subsystem performs the following processing: It determines whether similar answers have appeared multiple times in the temporary question-and-answer knowledge base. If the number of appearances exceeds a threshold, the knowledge is considered to be frequently used customer service knowledge, and it will be added to the question-and-answer knowledge base in step 5-3-5-4. Otherwise, the process ends.

[0186] Refer to the fact conflict analysis sub-process in step 5-3-7 Figure 7 As shown.

[0187] Step 5-3-7-1: The fact conflict knowledge reading and writing module of the knowledge management subsystem performs the following processing: different fact descriptions of the same issue are stored in the conflict knowledge base. This step stores two conflicting descriptions in the database and automatically assigns an auto-incrementing conflict event number.

[0188] Step 5-3-7-2: The fact conflict early warning module of the monitoring and display subsystem performs the following processing: sending a conflict early warning message to remind customer service agents, for example, by notifying customer service agents that a conflict has occurred through internal instant messaging tools, email, or SMS.

[0189] Step 5-3-7-3: The following processing is achieved by the fact conflict display module, the fact conflict handling module, and the fact conflict knowledge reading and writing module of the knowledge management subsystem within the monitoring and display subsystem: Customer service agent managers mark the correct facts. This further includes: Customer service agent managers can access the monitoring and display subsystem via a link in the alert message. The system will display the original problem and conflict information based on the conflict event number in the link; customer service agents can then select the correct information or input information they believe to be correct.

[0190] Step 5-3-7-4: The fact conflict knowledge reading and writing module of the knowledge management subsystem will mark the fact conflict as resolved and record the correct information.

[0191] Step 5-3-7-5: The document knowledge reading and writing module, the question-and-answer pair knowledge reading and writing module of the knowledge management subsystem, and the similarity calculation module of the intelligent question-and-answer subsystem update the corresponding document knowledge base and question-and-answer pair knowledge base: The system will scan all document knowledge bases and question-and-answer pair knowledge bases and correct the places that involve the corresponding knowledge.

[0192] Step 6: The system summarizes the conversation experience to enhance the response generation capability of the intelligent question-and-answer subsystem in Step 2.

[0193] In the previous process, the experience and knowledge gained in answering customer questions will be summarized and organized through the knowledge management subsystem to avoid making the experience documents too lengthy. The specific process will be detailed in subsequent sub-processes.

[0194] For detailed processing of step 6, please refer to the following: Figure 8 As shown, each processing step is described in detail below.

[0195] Step 6-1: The problem experience knowledge reading and writing module of the knowledge management subsystem performs session experience storage: the experience identified for customer problems is stored in the original problem experience log library in the form of {problem, experience} tuples.

[0196] Step 6-2: The problem experience standardization and storage are performed by the problem standardization module, the problem experience knowledge reading and writing module, and the similarity calculation module of the intelligent question answering subsystem. The problem is standardized, and then its similarity is calculated with existing problems in the problem experience knowledge base. If similar problems exist, the current experience is associated with the corresponding similar problem. Otherwise, the current problem is recorded as a new problem category.

[0197] Step 6-3: Extract general experience from the similarity calculation module of the intelligent question answering subsystem and the question experience knowledge reading and writing module of the knowledge management subsystem: calculate the similarity between the experience of the current question and the experience of other questions in the question experience knowledge base. If the current experience has similar experience for multiple questions (the threshold is generally set to 5 or above), then the current experience and other experiences are summarized and stored as general experience.

[0198] Step 6-4: The problem experience knowledge reading and writing module of the knowledge management subsystem performs problem experience summarization: After associating experience with similar problems, similar experiences may have already been added. Therefore, this step requires clustering the associated experiences (using the DBSCAN algorithm), and then summarizing the n clusters with the most entries (n is a threshold, which can be adjusted). When the recommendation response generation module retrieves experience knowledge, the problem experience knowledge reading and writing module will return the summarized experience corresponding to the problem instead of the original problem experience recorded in step 6-1, and will also return the general experience summarized in the previous step.

[0199] Although the methods described above are illustrated and depicted as a series of actions for the sake of simplicity, it should be understood and appreciated that these methods are not limited by the order of the actions, as some actions may occur in a different order and / or concurrently with other actions from the illustrations and descriptions herein or not illustrated and described herein but which may be understood by those skilled in the art, according to one or more embodiments.

[0200] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps are described above in a generalized manner in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the invention.

[0201] The various illustrative logic blocks, modules, and circuits described in conjunction with the embodiments disclosed herein can be implemented or performed using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but in alternatives, it may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core, or any other such configuration.

[0202] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of both. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor such that the processor can read and write information to / from the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and storage medium may reside as discrete components in the user terminal.

[0203] In one or more exemplary embodiments, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functionality may be stored or transmitted as one or more instructions or code on or through a computer-readable medium. A computer-readable medium includes both computer storage media and communication media, encompassing any medium that facilitates the transfer of a computer program from one location to another. A storage medium may be any available medium accessible to a computer. By way of example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and is accessible to a computer. Any connection is also legitimately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of a medium. As used in this article, disk and disc include compact discs (CDs), laser discs, optical discs, digital multi-purpose discs (DVDs), floppy disks, and Blu-ray discs. Disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of these should also be included within the scope of computer-readable media.

[0204] The prior description of this disclosure is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not intended to be limited to the examples and designs described herein, but should be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An intelligent customer service system based on large language models and reinforcement learning, characterized in that, The system includes: The customer service conversation management subsystem is used to enable interaction between customer service agents and customers; The intelligent response subsystem generates recommended response content and improves response quality through reinforcement learning based on the messages actually sent to customers by customer service personnel and customer feedback. The knowledge management subsystem is used to maintain the knowledge required for customer service responses. as well as The monitoring and display subsystem is used to display the conversation experience and new knowledge generated by the system during operation, and to record the quality of the recommended response content generated by the entire system and the quality of the responses of each customer service agent. The system operates through the following process: Step 1: Obtain customer sessions from the customer service session management subsystem; Step 2: The intelligent question-answering subsystem, in conjunction with the knowledge base managed by the knowledge management subsystem, generates corresponding recommended response content for customer service agents based on the customer conversation, and sends the corresponding notes to the customer service conversation management subsystem. Step 3: Customer service representatives edit the recommended responses generated by the intelligent question-and-answer subsystem and send them to the customer. Step 4: The intelligent question-and-answer subsystem obtains customer feedback on the response content; Step 5: The intelligent question-and-answer subsystem, monitoring and display subsystem, and knowledge management subsystem evaluate the effectiveness based on customer feedback. Step 6: Summarize and generalize the experience and knowledge gained from answering customer questions through the knowledge management subsystem, which will be used to enhance the response generation capability of the intelligent question-and-answer subsystem in Step 2.

2. The intelligent customer service system based on large language models and reinforcement learning according to claim 1, characterized in that, Step 2 further includes: Step 2-1: The message response judgment module in the intelligent question-and-answer subsystem determines whether a response is required for the customer message. If so, proceed to step 2-2; otherwise, proceed to step 2-7. Step 2-2: The question rewriting module of the knowledge management subsystem rewrites and optimizes the questions in the customer conversation messages by generating similar questions and generating simulated answers; Steps 2-3: The document retrieval module of the knowledge management subsystem retrieves the most relevant results from the document knowledge base and the question-answering knowledge base respectively through text embedding retrieval. Steps 2-4: The relevance ranking module of the knowledge management subsystem re-ranks the knowledge in the search results according to the degree of relevance; Steps 2-5: The response generation module of the intelligent question-answering subsystem and the question experience knowledge reading and writing module of the knowledge management subsystem generate response content based on experience prompts and relevant knowledge; Steps 2-6: The response generation module of the intelligent question-answering subsystem and the fact conflict knowledge reading and writing module of the knowledge management subsystem use the fact conflict knowledge base to perform similarity matching between each historical conflict knowledge and the response content currently generated by the big language model. If a resolved fact conflict knowledge is matched, the original response sentence is corrected through the big language model and a note is generated. Then, steps 2-8 are executed. Step 2-7: The response sending module of the intelligent question-and-answer subsystem returns an empty response and adds the reason why no reply is needed as a note, and then proceeds to step 2-8; Steps 2-8: The response sending module of the intelligent question-and-answer subsystem returns the recommended response content and remarks generated in the above steps to the customer service session management subsystem, which then displays the recommended response content and remarks to the customer service agents.

3. The intelligent customer service system based on large language models and reinforcement learning according to claim 1, characterized in that, In step 3, customer service agents can either choose not to modify the recommended response content and send it directly to the customer through the message sending module of the customer service session management subsystem, or completely rewrite the response content through the response editing module of the customer service session management subsystem and send it to the customer.

4. The intelligent customer service system based on large language models and reinforcement learning according to claim 1, characterized in that, In step 4, if the customer does not respond within the set time, an empty response is used to proceed to the subsequent performance evaluation process; otherwise, the customer response message received by the message receiving module is used to proceed to the subsequent performance evaluation process, and the customer response message is sent to the response generation module, which then determines whether a response is required.

5. The intelligent customer service system based on large language models and reinforcement learning according to claim 1, characterized in that, Step 5 further includes: Step 5-1: The effectiveness evaluation module of the intelligent question-answering subsystem provides feedback scores based on completion and sentiment. Step 5-2: The indicator recording module of the monitoring and display subsystem records the feedback score as the customer service agent's score, and determines whether to execute step 5-3 or step 5-4 based on the customer service agent's score result. Step 5-3: Execute the positive evaluation sub-process; Step 5-4: The failure experience analysis module of the intelligent question-answering subsystem and the problem experience knowledge reading and writing module of the knowledge management subsystem analyze customer feedback to generate conversation experience.

6. The intelligent customer service system based on large language models and reinforcement learning according to claim 5, characterized in that, Step 5-3 further includes: Step 5-3-1: The similarity calculation module of the intelligent question answering subsystem calculates the similarity between the generated response and the actual response; Step 5-3-2: The indicator recording module of the monitoring and display subsystem scores the generated effect based on the degree of similarity; Step 5-3-3: The clause splitting module of the intelligent question-answering subsystem splits both the generated response and the actual response into clauses; Step 5-3-4: The similarity calculation module of the intelligent question answering subsystem matches each clause: the similarity algorithm is used again to match each clause; Step 5-3-5: The effect evaluation module of the intelligent question-answering subsystem performs the following processing: If there are more actual response clauses than generated response clauses, execute the process of adding question-answer knowledge pairs; Step 5-3-6: The following processing is implemented by the question experience generation module of the intelligent question answering subsystem, the question standardization module of the knowledge management subsystem, and the question experience knowledge reading and writing module of the knowledge management subsystem: If there are more generated response clauses than actual response clauses, they will be deleted when a similar situation is encountered again; Step 5-3-7: The fact conflict analysis module of the intelligent question answering subsystem performs the following processing: If the clauses can match but are not completely identical, then the two clauses are processed separately according to whether the facts of the two clauses match as determined by the large language model: If the facts match, the differences between the two clauses are analyzed through the large language model to generate an experience for the current problem; if the facts do not match, a fact conflict knowledge base is established for analysis.

7. The intelligent customer service system based on large language models and reinforcement learning according to claim 6, characterized in that, Step 5-3-5 further includes: Step 5-3-5-1: The document retrieval module of the knowledge management subsystem searches whether the knowledge described in the clause exists in the document knowledge base; Step 5-3-5-2: The fact conflict analysis module of the intelligent question answering subsystem performs the following processing: If similar information exists in the knowledge base, the large language model is used to determine whether the information in the knowledge base is consistent with the information in the customer service response; Step 5-3-5-3: If the previous step finds that the input knowledge is inconsistent with the knowledge in the document knowledge base, the fact conflict analysis sub-process of step 5-3-7 will be called. Step 5-3-5-4: The question standardization module and the question-answer pair knowledge reading and writing module of the knowledge management subsystem standardize the question-answer pairs and add them to the question-answer pair knowledge base; Step 5-3-5-5: The question-and-answer pair knowledge reading and writing module of the knowledge management subsystem stores the question-and-answer pairs into a temporary question-and-answer pair knowledge base; Step 5-3-5-6: The question-and-answer knowledge reading and writing module of the knowledge management subsystem performs the following processing: obtain the number of times similar answers appear in the temporary question-and-answer knowledge base. If the number of appearances is greater than the threshold, the knowledge is considered to be commonly used customer service knowledge, and the knowledge will be added to the question-and-answer knowledge base in step 5-3-5-4. Otherwise, the process ends.

8. The intelligent customer service system based on large language models and reinforcement learning according to claim 6, characterized in that, Step 5-3-7 further includes: Step 5-3-7-1: The fact conflict knowledge reading and writing module of the knowledge management subsystem stores different factual descriptions of the same issue into the conflict knowledge base and automatically assigns an auto-incrementing conflict event number; Step 5-3-7-2: The fact conflict early warning module of the monitoring and display subsystem sends a conflict early warning message to remind customer service agent management personnel; Step 5-3-7-3: The customer service representative manager marks the correct facts; Step 5-3-7-4: The fact conflict knowledge reading and writing module of the knowledge management subsystem marks the fact conflict as resolved and records the correct information; Step 5-3-7-5: The document knowledge reading and writing module, the question-and-answer pair knowledge reading and writing module of the knowledge management subsystem, and the similarity calculation module of the intelligent question-and-answer subsystem update the corresponding document knowledge base and question-and-answer pair knowledge base.

9. The intelligent customer service system based on large language models and reinforcement learning according to claim 1, characterized in that, Step 6 further includes: Step 6-1: The problem experience knowledge reading and writing module of the knowledge management subsystem stores the session experience; Step 6-2: The question experience is standardized and stored by the question standardization module of the knowledge management subsystem, the question experience knowledge reading and writing module of the knowledge management subsystem, and the similarity calculation module of the intelligent question answering subsystem; Step 6-3: Extract general experience from the similarity calculation module of the intelligent question answering subsystem and the question experience knowledge reading and writing module of the knowledge management subsystem; Step 6-4: The problem experience is summarized by the problem experience knowledge reading and writing module of the knowledge management subsystem.

Images