Customer service processing method based on semantic retrieval model and text generation language model
By combining semantic search model and text generation language model, intelligent customer service can effectively reduce development costs and complexity, improve question-and-answer accuracy and response speed, and solve the problems of high development costs and unstable large language models in broadband fault handling. .
Patent Information
- Application Number
- CN202411453817.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-17
- Publication Date
- 2025-01-14
AI Technical Summary
Intelligent customer service has high development costs in broadband fault handling, and the instability of large language models limits its application, resulting in complex and high cost.
Using a combination of semantic retrieval model and text generation language model, we can collect broadband fault Q&A recordings, form step-by-step guide nodes, configure standard speech, and use GRU, LSTM or GPT models to train generation dialogues to achieve user broadband fault problems The solution.
It reduces the complexity of technical configuration and development costs during the development process of intelligent customer service, improves the accuracy and response speed of customer service Q&A, and enhances the user experience.
Abstract
Description
Technical Field
[0001] The present invention relates to the field of AI data processing technology, and in particular to a customer service processing method based on a semantic retrieval model and a text generation language model. Background Art
[0002] Customer Service In the field of broadband fault handling, the value of intelligent customer service is particularly prominent. When users report broadband problems, intelligent customer service can respond immediately by asking a series of preset questions, such as whether the router has been restarted, checking the line connection, etc., to make preliminary diagnoses and quickly screen out solutions to common problems without users having to wait for a long time for manual customer service to intervene. At the same time, it can provide 24-hour uninterrupted service every day of the year, ensuring that users' urgent needs can be responded to in a timely manner during non-working hours, thereby enhancing user satisfaction. At the same time, the processing of most basic problems can be automated, which can significantly reduce the workload of manual customer service, allowing them to focus on solving more complex technical problems or providing deeper user care. However, the development of broadband fault robots based on intelligent customer service requires a lot of work, usually involving speech configuration, entity extraction, intent understanding, dialogue management, etc., with high development costs and is not conducive to operation and maintenance. With the emergence of large language models, the industry has developed solutions to integrate large language models into intelligent customer service. For example, the entire user dialogue process is written in text as part of the prompt as a knowledge background. However, due to the lack of generation ability, large language models are unstable under low probability conditions, which limits the development of large language models. If large language models are used to solve single intent understanding and entity recognition, the advantages are not obvious, which puts intelligent customer service into an optimization dead loop. In order to solve the above technical pain points, this patent proposes an intelligent customer service solution based on a semantic retrieval model combined with a text generation language model. Although this patent is more controllable, the scope of use of this patent is: the user dialogue content does not include entity information, such as personal mobile phone number, ID number, name, mobile phone brand number and other relatively private or personal related content, and focuses more on providing users with general technical solutions. Summary of the invention
[0003] There are currently two main ways to implement intelligent customer service. One is based on a small model (such as Bert), etc., through intent understanding, entity recognition, semantic retrieval, process management and other technologies. The process is usually cumbersome. The other is based on a large language model. There are two main mainstream methods for solving the problem through a large language model. One is to describe the conversation process in text, write it into the system, and use the large language model to guide users to talk according to the process. The other is to let the large language model play a single point of intent understanding and entity recognition capabilities. However, the large language model is usually not stable. To date, although intelligent customer service has been implemented in many scenarios, it is still very difficult to implement intelligent customer service and the development cost is high. This patent aims to provide a convenient intelligent customer service solution. The user conversation content does not include entity information, such as personal mobile phone number, ID number, name, mobile phone brand number and other relatively private or personal related content. It mainly provides users with a solution to broadband failure problems by talking to them.
[0004] The present invention solves the problems of the prior art by adopting the following technical solution, a customer service processing method based on a semantic retrieval model and a text generation language model, comprising the following steps: S1: Joint broadband fault resolution process: Collect broadband fault customer service Q&A recordings, sort them out to form a broadband fault customer service Q&A resolution process, form step-by-step guidance nodes through sentence-by-sentence questions and answers, and form a process until the problem is solved to form a set of broadband fault type processes; select different recordings to form multiple sets of broadband fault type process sets; S2: Broadband fault resolution process script configuration: Configure scripts for step-by-step guidance nodes based on sentence-by-sentence questions and answers. Use a semantic retrieval model to map the user's answers to standard scripts. Perform multiple rounds of mapping of step-by-step guidance nodes in the process to standard scripts. S3: Create a speech token: The standard speech of the step-by-step guidance node in S2 is converted into a token; and each standard speech is stored in a line, consisting of a reserved [CLS] text start character and a [PAD] text filler character; [PAD] is a token for a scenario of a troubleshooting process. When the length of the token is less than the maximum length maxlen, the [PAD] filler character is used to fill it; the token for the intelligent customer service to terminate the conversation also includes the [EOS] text terminator, and [EOS] means that the intelligent customer service terminates the conversation, that is, reaches the leaf node; S4: Model construction: Use GRU, LSTM or GPT text generation model to train the tokens in S3, and then train to obtain historical training text to complete the prediction of dialogue skills, that is, to solve the user's broadband failure problem.
[0005] S5: Intelligent customer service application: The historical training texts trained in S3 are composed of a historical training speech set, which is used for customer service consultation. If the consultation content is the standard speech of the step-by-step guidance node, the question is answered by giving the historical training speech set; if the consultation content is not the standard speech of the step-by-step guidance node, steps S1 to S4 are looped again to form a new historical training text and add it to the historical training speech set.
[0006] The cosine similarity threshold of the standard speech in the semantic retrieval model in S2 is 0.85. Each new customer service consultation and process set is processed through the semantic retrieval model. Text cosine similarity ≥ 0.85 is standard speech, which is answered using the process set and step-by-step guidance nodes; text cosine similarity < 0.85 is non-standard speech, and speech tokens are created through steps S3 to S5 and the historical training speech set is supplemented.
[0007] Beneficial effects of the invention: This patent proposes an intelligent customer service solution that combines a semantic retrieval model with a text generation model, aiming to solve the problems of excessive script configuration and high development costs in the traditional customer service development process, and at the same time solve the instability problem of large language models. The semantic retrieval model and text generation model trained by this patent are combined and applied with each other, so that the intelligent customer service only needs to configure the business process, and each process node only needs to configure a standard script. An innovative small-scale intelligent customer service solution that combines a small semantic retrieval model with a text generation model is proposed, aiming to avoid the problems of script configuration and intent model training in the intelligent customer service development process, that is, through the combination of a semantic retrieval model and a text generation model. It effectively solves the common problem of low accuracy of the semantic retrieval model, improves the adaptability of customer service question and answer implementation, and at the same time adopts the method of semantic retrieval similarity threshold judgment to improve user customer service question and answer favorability and response speed. DETAILED DESCRIPTION
[0008] A customer service processing method based on a semantic retrieval model and a text generation language model comprises the following steps: S1: Joint broadband fault resolution process: Collect broadband fault customer service Q&A recordings, sort them out to form a broadband fault customer service Q&A resolution process, form step-by-step guidance nodes through sentence-by-sentence questions and answers, and form a process until the problem is solved to form a set of broadband fault type processes; select different recordings to form multiple sets of broadband fault type process sets; S2: Broadband fault resolution process script configuration: Configure scripts for step-by-step guidance nodes based on sentence-by-sentence questions and answers. Use a semantic retrieval model to map the user's answers to standard scripts. Perform multiple rounds of mapping of step-by-step guidance nodes in the process to standard scripts. S3: Create a speech token: The standard speech of the step-by-step guidance node in S2 is converted into a token; and each standard speech is stored in a line, consisting of a reserved [CLS] text start character and a [PAD] text filler character; [PAD] is a token for a scenario of a troubleshooting process. When the length of the token is less than the maximum length maxlen, the [PAD] filler character is used to fill it; the token for the intelligent customer service to terminate the conversation also includes the [EOS] text terminator, and [EOS] means that the intelligent customer service terminates the conversation, that is, reaches the leaf node; S4: Model construction: Use GRU, LSTM or GPT text generation model to train the tokens in S3, and then train to obtain historical training text to complete the prediction of dialogue skills, that is, to solve the user's broadband failure problem.
[0009] S5: Intelligent customer service application: The historical training texts trained in S3 are composed of a historical training speech set, which is used for customer service consultation. If the consultation content is the standard speech of the step-by-step guidance node, the question is answered by giving the historical training speech set; if the consultation content is not the standard speech of the step-by-step guidance node, steps S1 to S4 are looped again to form a new historical training text and add it to the historical training speech set.
[0010] The cosine similarity threshold of the standard speech in the semantic retrieval model in S2 is 0.85. Each new customer service consultation and process set is processed through the semantic retrieval model. Text cosine similarity ≥ 0.85 is standard speech, which is answered using the process set and step-by-step guidance nodes; text cosine similarity < 0.85 is non-standard speech, and speech tokens are created through steps S3 to S5 and the historical training speech set is supplemented.
[0011] The specific technical requirements of the case are explained as follows: S1: Broadband fault resolution process: Industry experts will sort out the broadband fault resolution process. For example, if a user reports that the wireless network cannot be connected, the robot will determine whether it is a single device or all devices. If it is a single device, the robot will consult the prompt information of the failure to connect to the Internet, guide the user step by step to the leaf node of the process, and provide the user with the solution of the final leaf node. At this point, processes for various broadband fault types will be formed.
[0012] S2: Broadband fault resolution process speech configuration: Configure the speech for the questions that the robot or user needs to answer or inquire in the process. The difference between this patent and the traditional small model is that it only needs to configure a user or robot standard speech in the broadband fault resolution process. The main innovation of this patent is that it uses an intelligent customer service solution that combines semantic retrieval models and text generation models such as GPT, without the need to configure a lot of speech like previous intelligent customer service solutions. When the content of the user's answer is not standard, the semantic similarity retrieval model will be used to map the content of the user's answer to the standard speech, so as to continue to complete the subsequent multiple rounds of dialogue.
[0013] S3: Create a speech token: The token used in this patent is slightly different from the token used in the industry. The token used in this patent is the standard speech of the robot and user in S2, that is, the user's standard speech (one sentence) is a token. That is, the token used in this patent is not a character, a node, a function name, or a plug-in name, but a standard speech (a complete sentence). The speech of all process nodes in S2 (newline characters are replaced with \n), including the solutions of leaf nodes, are all saved as tokens, and each speech or solution is stored in one line. At the same time, the [CLS] text start character and the [PAD] text filler are reserved. [PAD] means that when the length of the token of a scenario of a troubleshooting process is less than the maximum length maxlen, the [PAD] filler is used to fill it. [EOS] means that the intelligent customer service terminates the conversation, that is, when it reaches the leaf node, the robot terminates the conversation after giving an answer. This patent stipulates that max_len=100 is the maximum length of the input model stipulated in this patent, that is, the stipulated model stipulates that the robot and the user will interact for a maximum of max_len / 2=50 rounds. At the same time, this patent focuses on solving 5,000 user failure scenarios.
[0014] S3: Model construction: The text generation language model used in the technical solution of this patent can be a text generation model such as GRU, LSTM, GPT, etc. This is a mature technology in the industry. The model structure is not the innovation of this patent and will not be explained in this patent. In order to explain the specific technical solution of this patent, this patent records the text generation language model as TextGenerationLM (i.e., any one of GRU, LSTM, GPT). The training process of the text generation language model of this patent is to let TextGenerationLM predict new robot speech based on historical robot speech and user speech, output it to the user, and provide a technical solution for broadband fault resolution. The robot speech and user speech are equivalent to a token in the large language model, which is the innovation of this patent. This patent turns the process of broadband fault into a token sequence of speech, and then trains a TextGenerationLM to complete the prediction of robot speech, that is, to solve the user broadband fault problem.
[0015] S4: Specific use of intelligent customer service: The TextGenerationLM trained by S3 only achieves the purpose of predicting the current robot's speech based on historical robot speech and historical user speech. However, in the actual problem-solving process, users have a variety of language expressions. This patent combines the semantic retrieval model and the text generation TextGenerationLM to complete a small broadband fault-solving customer service robot. Specifically: First, the robot outputs the speech to consult the user about the details of the specific fault, and then the user outputs his own text information. However, the user input content is not a defined speech token, so TextGenerationLM cannot be used. It must first pass through the semantic similarity retrieval model to retrieve the speech with the highest similarity in the semantic vector matrix of the user-configured speech. This speech can be used as a token of the text generation model. This speech is the actual speech of the user input text generation language model. Because this speech is the actual speech of the robot designer, that is, an actual token, it can complete the robot's speech prediction and output it to the user. By analogy, the specific fault-solving implementation process of this patent is completed. However, the words with the highest similarity on the front-end page may not be the actual words of the user through the semantic retrieval model. If the semantic cosine similarity threshold of the standard words retrieved by the user text words is greater than 0.85, the front-end page directly outputs the robot words configured by the model prediction. If the cosine similarity threshold is less than 0.85, the actual words with the highest semantic similarity searched for the words are still used as actual tokens and input into the text generation language model to predict the robot words. At this time, the front-end page directly outputs the robot words configured by the model prediction and adds a sentence. The answer is "I can't accurately judge your true intention. According to your question, I provide the following answer:" + "The text generation language model predicts the robot standard words" + "\n, please confirm whether my answer is correct". The front-end page configures a confirmation button. If the user enters a negative, the model history information is cleared and the conversation is restarted. "The text generation language model predicts the robot standard words" represents the robot words manually configured in advance. The above method completes the final solution to the broadband failure problem.
[0016] Glossary Semantic retrieval: Vectorized semantic retrieval uses deep learning technology to convert text data into vector form, and then achieves more accurate retrieval by comparing the similarity of these vectors. The core of this method is to map the abstract meaning of language into mathematical space, so that computers can understand and process semantic information.
[0017] GRU model: GRU (Gated Recurrent Unit) is a variant of a recurrent neural network (RNN) for sequence data. It aims to solve the gradient vanishing or gradient exploding problems of traditional RNN when processing long sequence data. The core idea of GRU is to introduce two gating mechanisms: Update Gate and Reset Gate. These two gating mechanisms can control the flow of information, thereby helping the network better capture long-term dependencies. The GRU language model can be used for text generation tasks with small-scale datasets.
[0018] GPT model: GPT is a generative pre-trained model based on the Transformer decoder architecture. The core of the GPT model lies in its powerful language generation and understanding capabilities. It captures the regularity of language by learning on a large amount of text data, and can then be used for various natural language processing tasks, such as text generation, translation, question answering, summarization, etc.
[0019] Transformer: Transformer is a deep learning model based on the self-attention mechanism. It performs well in processing long sequence data and is particularly suitable for tasks such as machine translation that require processing global dependencies. The Transformer model uses an attention mechanism that allows the model to establish connections between different positions, thereby better understanding and processing the input sequence. Transformer consists of two parts: encoder and decoder, each of which consists of multiple layers.
[0020] Token: In the language model, a token is a basic minimum unit of text. It can be a word, character, punctuation mark, or any string segmented according to specific rules. It can also be a manually defined token, such as a tool name, function name, or sentence.
[0021] Intelligent customer service: Intelligent customer service is a system that uses natural language processing, machine learning, speech recognition and other technologies to simulate and perform customer service agent functions. It can automatically handle customer inquiries, provide information, solve problems, and even conduct a certain degree of emotional communication to achieve the purpose of improving service efficiency, reducing costs, and enhancing user experience.
Claims
1. A customer service processing method based on a semantic retrieval model and a text generation language model, characterized in that The steps include: S1: Joint broadband fault resolution process: Collect broadband fault customer service Q&A recordings, sort them out to form a broadband fault customer service Q&A resolution process, form step-by-step guidance nodes through sentence-by-sentence questions and answers, and form a process until the problem is resolved, forming a set of processes for broadband fault types; Select different recordings to form multiple sets of process sets for broadband fault types; S2: Broadband fault resolution process script configuration: Configure scripts for step-by-step guidance nodes based on sentence-by-sentence questions and answers. Use a semantic retrieval model to map the user's answers to standard scripts. Perform multiple rounds of mapping of step-by-step guidance nodes in the process to standard scripts. S3: Create a speech token: The standard speech of the step-by-step guidance node in S2 is converted into a token; and each standard speech is stored in a line, consisting of a reserved [CLS] text start character and a [PAD] text filler character; [PAD] is a token for a scenario of a troubleshooting process. When the length of the token is less than the maximum length maxlen, the [PAD] filler character is used to fill it; the token for the intelligent customer service to terminate the conversation also includes the [EOS] text terminator, and [EOS] means that the intelligent customer service terminates the conversation, that is, reaches the leaf node; S4: Model construction: Use the GRU, LSTM or GPT text generation model to train the tokens in S3, and then train to obtain historical training text to complete the prediction of dialogue skills, that is, to solve the user's broadband failure problem; S5: Intelligent customer service application: The historical training texts completed through S3 training form a historical training script set, which is used for customer service consultation. If the consultation content is a standard script for step-by-step guidance nodes, the historical training script set is used to answer questions; If the consultation content is not the standard script of the step-by-step guidance node, steps S1 to S4 are looped again to form a new historical training text and add it to the historical training script set.
2. The customer service processing method based on the semantic retrieval model and the text generation language model according to claim 1 is characterized in that The cosine similarity threshold of the standard speech in the semantic retrieval model in S2 is 0.
85. Each new customer service consultation and process set is processed through the semantic retrieval model. Text cosine similarity ≥ 0.85 is standard speech, which is answered using the process set and step-by-step guidance nodes; text cosine similarity < 0.85 is non-standard speech, and speech tokens are created through steps S3 to S5 and the historical training speech set is supplemented.