A customer service system for assisting and evaluating responses to extremely long conversations.

Abstract

The present invention provides a customer service super-long conversation reply assistance and evaluation system, including: processing the current conversation and the conversation set obtained by merging the nearest multiple conversations of the current conversation through a summary text screening model, respectively screening out matching summary texts, and the conversation key point summary extraction model summarizes and extracts the matching summary text and conversation set, and generates conversation key points and field labels. The answer matching model processes the conversation key points and screens out matching answers. The customer service evaluation model converts the customer service's reply and the matching answer into a reply vector and an answer vector, calculates the similarity between the two, and deducts points from the customer service once when it is lower than the first preset similarity threshold, and scores the customer service based on the total deduction points. The system can help customer service personnel or telephone sales personnel quickly summarize and extract the core information in super-long conversations, accurately capture the main needs of users, and based on this, make response prompts to customer service and evaluate whether the customer service complies with SOP specifications.

Description

Technical Field

[0001] The present invention relates to the fields of customer service and artificial intelligence technology, and in particular to a customer service ultra-long conversation reply assistance and evaluation system, an electronic device and a readable storage medium. Background Art

[0002] In the customer service industry, there are some difficulties in handling extremely long conversations. It is difficult for customer service to obtain comprehensive information and capture key information, making it difficult to accurately judge user needs. The services provided cannot fully meet user needs, resulting in a poor user experience. Therefore, there is an urgent need for technology that can handle extremely long conversations and summarize and extract key points.

[0003] With the development of natural language understanding technology, text understanding and key point extraction technology based on the attention framework has become increasingly widely used. However, both traditional NLP technology and current large model technology have some problems in understanding and summarizing the context of extremely long conversations:

[0004] 1. Context window limitation: Most LLMs are limited to a fixed context window size when processing long texts, which means that the model can only process text fragments of a certain length at the same time, resulting in the possibility of loss of important information.

[0005] 2. Computational complexity: The computational complexity of the self-attention mechanism in the Transformer model increases quadratically when processing long texts, which limits the model's ability to process long texts.

[0006] 3. Context fragmentation: When processing long texts, splitting the text into multiple segments may lead to fragmentation of context information and loss of association information between text segments.

[0007] 4. Long-distance dependency capture: Although the Transformer structure can theoretically capture long-distance dependencies, in practical applications, due to the limitations of the context window, the model may not be able to effectively handle long-distance semantic relationships.

[0008] 5. Information retrieval and memory: LLM needs to effectively retrieve and remember key information in long text understanding, but the current model has insufficient performance in this regard.

[0009] 6. Model optimization and acceleration: In order to enable the model to process longer texts, the model needs to be optimized to reduce computing time and resource consumption, which includes model compression, acceleration, and optimization of the attention mechanism.

[0010] 7. Lack of long text datasets: There is a lack of sufficiently large and diverse long text datasets to train and evaluate models, which limits the improvement of model performance and the scope of application.

[0011] Therefore, the current super-long conversation processing technology still lacks comprehensiveness and accuracy in summarizing and extracting the key points, making it difficult to accurately judge user needs. The services provided cannot fully meet user needs, resulting in poor user experience. In addition, due to differences in experience and capabilities of different customer service staff, some customer service staff may not be able to provide satisfactory solutions to users based on the key point information, and the services provided are still inadequate. Therefore, there is an urgent need for technology that can further provide customer service with response prompts based on the key point prompts, as well as technology that evaluates whether customer service replies are standardized. Summary of the invention

[0012] In view of the above problems, the present invention is proposed to provide a customer service ultra-long conversation response assistance and evaluation system, an electronic device and a readable storage medium that overcome the above problems or at least partially solve the above problems.

[0013] The present invention provides a customer service long conversation reply assistance and evaluation system, comprising:

[0014] Summary text vector database, used to store the vectors corresponding to the summary texts of historical long conversations;

[0015] A summary text vector pairing database, used to store vector-summary text data pairs; the vector-summary text data pairs are the vectors corresponding to each summary text of the historical super-long conversation and the corresponding summary text;

[0016] Question vector database, used to store vectors corresponding to questions in customer service specification documents;

[0017] A question-answer pairing database, used to store vector-answer data pairs; the vector-answer data pairs are the vector corresponding to each question in the customer service specification document and the corresponding answer in the customer service specification document;

[0018] A summary text screening model is used to receive a current conversation and a conversation set obtained by merging multiple conversations closest to the current conversation, process the current conversation and the conversation set, and screen out matching summary texts according to a summary text vector database and a summary text vector pairing database, and input the matching summary text and the conversation set into a conversation key point summary extraction model;

[0019] A dialogue key points summary extraction model, used to perform summary extraction on the matched summary text and the dialogue set to generate dialogue key points and domain labels;

[0020] An answer matching model is used to process the key points of the conversation and screen out matching answers based on the question vector database and the question-answer pairing database, and provide them to the customer service as a response prompt;

[0021] A customer service evaluation model is used to obtain a customer service reply, convert the reply into a reply vector, convert the matched answer into an answer vector, and calculate the similarity between the reply vector and the answer vector, and if the similarity between the reply vector and the answer vector is lower than a first preset similarity threshold, deduct points from the customer service, and score the customer service based on the total deductions.

[0022] Optionally, the answer matching model is used to:

[0023] Converting the dialogue key points into dialogue key point vectors, and calculating the similarity between the dialogue key point vectors and each question vector in a question vector database, and taking question vectors exceeding a second preset similarity threshold as question vectors that match the dialogue key point vectors;

[0024] The answer corresponding to the question vector matching the conversation key point vector is searched in the question-answer pairing database, and provided to the customer service as a response prompt.

[0025] Optionally,

[0026] The answer pairing model is also used to receive a customer service specification document constructed in the form of a question-answer pair, convert each question in the customer service specification document into a question vector, classify the questions according to the field to which the questions belong and label them with field labels, store the question vectors in a question vector database, and store the question vector-answer data pairs in a question-answer pairing database.

[0027] Optionally,

[0028] The customer service over-long conversation response assistance and evaluation system is also used to optimize customer service specification documents based on the points deducted from the customer service during the conversation, as well as user satisfaction, complaint rate, and transaction amount.

[0029] Optionally, the summarized text screening model is used to:

[0030] Converting the current conversation into a current conversation vector, and searching the summary text vector database for a summary text vector that matches the current conversation vector;

[0031] Converting the dialog set into a dialog set vector, and searching the summary text vector database for a summary text vector that matches the dialog set vector;

[0032] Searching the summary text corresponding to the summary text vector matching the current conversation vector from the summary text vector pairing database to obtain the current conversation summary text;

[0033] Searching the summary text corresponding to the summary text vector that matches the dialogue set vector from the summary text vector pairing database to obtain the dialogue set summary text;

[0034] The current conversation summary text, the conversation set summary text and the conversation set are input into a conversation key point summary extraction model.

[0035] Optionally,

[0036] The summarized text screening model is used to calculate the similarity between the current conversation vector and each summarized text vector in the summarized text vector database, and use the summarized text vector exceeding a third preset similarity threshold as the summarized text vector matched by the current conversation vector;

[0037] The summarized text screening model is used to calculate the similarity between the dialog set vector and each summarized text vector in the summarized text vector database, and to use the summarized text vector exceeding a fourth preset similarity threshold as the summarized text vector matched by the dialog set vector.

[0038] Optionally,

[0039] The dialogue key points summary extraction model is also used to receive historical super-long dialogues, construct dialogue segment windows of various lengths and steps for each historical super-long dialogue through a sliding window mechanism, summarize the dialogue segments in each window, summarize each dialogue segment, generate a summary text for each dialogue segment and input it into the summary text screening model;

[0040] The summarized text screening model is also used to convert the summarized text of each dialogue segment into a summarized text vector of each dialogue segment, and store the summarized text vectors in corresponding tables of a summarized text vector database according to the window length and step size, and store the summarized text vector-summary text data pairs in corresponding tables of a summarized text vector pairing database according to the window length and step size.

[0041] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the electronic device executes the computer program, it loads the customer service long conversation response assistance and evaluation system as described in any one of the embodiments of the present invention.

[0042] The present invention also provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the customer service long conversation reply assistance and evaluation system as described in any one of the embodiments of the present invention is loaded.

[0043] The present invention includes the following advantages:

[0044] The customer service super-long dialogue reply assistance and evaluation system of the present invention receives the current dialogue and the dialogue set obtained by merging the nearest multiple dialogues of the current dialogue through a summary text screening model, processes the current dialogue and the dialogue set, and screens out matching summary texts according to a summary text vector database and a summary text vector matching database, and inputs the matching summary text and the dialogue set into a dialogue key point summary extraction model. The dialogue key point summary extraction model summarizes and extracts the matching summary text and the dialogue set, and generates dialogue key points and domain labels. The answer matching model processes the dialogue key points and screens out matching answers according to a question vector database and a question-answer matching database, and provides them to the customer service as a response prompt. The customer service evaluation model obtains the customer service's reply, converts the reply into a reply vector, converts the matching answer into an answer vector, and calculates the similarity between the reply vector and the answer vector, and if the similarity between the reply vector and the answer vector is lower than a first preset similarity threshold, the customer service is deducted once, and the customer service is scored based on the total deduction. The system can help customer service staff or telephone sales staff quickly summarize and extract the core information in long conversations, accurately capture the main needs and key information of users, and based on this, provide response tips to customer service and evaluate whether customer service complies with SOP specifications. By combining the current conversation with the content of multiple conversations closest to it and using the screening mechanism of summarized text, the comprehensiveness and accuracy of conversation analysis are ensured, and accurate response tips and response standardization evaluation are provided, thereby improving the quality and efficiency of service. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a structural block diagram of a customer service super-long conversation reply assistance and evaluation system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0047] Reference Figure 1 , shows a structural block diagram of a customer service super-long conversation reply assistance and evaluation system provided in an embodiment of the present invention, which may specifically include the following modules:

[0048] Summary text vector database, used to store the vectors corresponding to the summary texts of historical long conversations;

[0049] A summary text vector pairing database, used to store vector-summary text data pairs; the vector-summary text data pairs are the vectors corresponding to each summary text of the historical super-long conversation and the corresponding summary text;

[0050] Question vector database, used to store vectors corresponding to questions in customer service specification documents;

[0051] A question-answer pairing database, used to store vector-answer data pairs; the vector-answer data pairs are the vector corresponding to each question in the customer service specification document and the corresponding answer in the customer service specification document;

[0052] A summary text screening model is used to receive a current conversation and a conversation set obtained by merging multiple conversations closest to the current conversation, process the current conversation and the conversation set, and screen out matching summary texts according to a summary text vector database and a summary text vector pairing database, and input the matching summary text and the conversation set into a conversation key point summary extraction model;

[0053] A dialogue key points summary extraction model, used to perform summary extraction on the matched summary text and the dialogue set to generate dialogue key points and domain labels;

[0054] An answer matching model is used to process the key points of the conversation and screen out matching answers based on the question vector database and the question-answer pairing database, and provide them to the customer service as a response prompt;

[0055] A customer service evaluation model is used to obtain a customer service reply, convert the reply into a reply vector, convert the matched answer into an answer vector, and calculate the similarity between the reply vector and the answer vector, and if the similarity between the reply vector and the answer vector is lower than a first preset similarity threshold, deduct points from the customer service, and score the customer service based on the total deductions.

[0056] The customer service super-long dialogue reply assistance and evaluation system of the present invention may include a summary text vector database, a summary text vector pairing database, a question vector database, a question-answer pairing database, a summary text screening model, a dialogue key point summary extraction model, an answer matching model and a customer service evaluation model, wherein the summary text vector pairing database and the question-answer pairing database may be a kv database, and the text and the summary text vector form a data pair with the summary text vector as the key and the summary text as the value; the question vector and the answer form a data pair with the question vector as the key and the answer as the value. The summary text screening model and the answer matching model may be a text2vector model with an attention mechanism based on a transformer architecture, and the dialogue key point summary extraction model may be an LLM model.

[0057] During the conversation between the user and the customer service, the user generates a new conversation. The length of the new conversation can be defined by the user and includes at least one conversation. The new conversation of the user is provided to the summary text screening model, and the past n conversations of the current new conversation are merged and provided to the summary text screening model. The summary text screening model can receive the current conversation and the conversation set obtained by merging the nearest multiple conversations of the current conversation, process the current conversation and the conversation set, and filter out the summary text matching the current conversation and the conversation set according to the data pre-stored in the summary text vector database and the summary text vector pairing database, and input the matching summary text and the conversation set into the conversation key point summary extraction model. The conversation key point summary extraction model can summarize and extract the matching summary text and conversation set, and generate conversation key points and field labels. In order to provide response prompts to the customer service, the answer matching model can process the conversation key points and filter out the matching answers according to the question vector database and the question answer pairing database, and provide them to the customer service as response prompts. After the customer service responds, if it is necessary to check whether the customer service can respond according to the SOP instructions, the customer service evaluation model can obtain the customer service's response, convert the customer service's response into a response vector, convert the matching answer into an answer vector, and calculate the similarity between the response vector and the answer vector, and then determine whether the similarity between the response vector and the answer vector is lower than the first preset similarity threshold. If the similarity between the response vector and the answer vector is lower than the first preset similarity threshold, or even the opposite, the customer service will be deducted once, and the customer service will be scored based on the total deduction. By combining the current conversation with the contents of multiple conversations closest to it, and through the screening mechanism of the summarized text, the comprehensiveness and accuracy of the conversation analysis are ensured, and then accurate response prompts and response normative evaluation are provided, which improves the quality and efficiency of the service.

[0058] In one embodiment of the present invention, the summarized text screening model is used to:

[0059] Converting the current conversation into a current conversation vector, and searching the summary text vector database for a summary text vector that matches the current conversation vector;

[0060] Converting the dialog set into a dialog set vector, and searching the summary text vector database for a summary text vector that matches the dialog set vector;

[0061] Searching the summary text corresponding to the summary text vector matching the current conversation vector from the summary text vector pairing database to obtain the current conversation summary text;

[0062] Searching the summary text corresponding to the summary text vector that matches the dialogue set vector from the summary text vector pairing database to obtain the dialogue set summary text;

[0063] The current conversation summary text, the conversation set summary text and the conversation set are input into a conversation key point summary extraction model.

[0064] In one embodiment of the present invention,

[0065] The summarized text screening model is used to calculate the similarity between the current conversation vector and each summarized text vector in the summarized text vector database, and use the summarized text vector exceeding a third preset similarity threshold as the summarized text vector matched by the current conversation vector;

[0066] The summarized text screening model is used to calculate the similarity between the dialog set vector and each summarized text vector in the summarized text vector database, and to use the summarized text vector exceeding a fourth preset similarity threshold as the summarized text vector matched by the dialog set vector.

[0067] In the embodiment of the present invention, the summary text screening model first converts the current conversation and the conversation set into the current conversation vector and the conversation set vector respectively. These two vectors are numerical representations in high-dimensional space and can capture the semantic features of the conversation. Then, the summary text vector matching the current conversation vector and the conversation set vector is searched in the summary text vector database, which can be achieved by calculating the similarity between the vectors, and the summary text vector with high similarity is found as the matching result. Then, the summary text vector matching database is used to retrieve the corresponding summary text, that is, the current conversation summary text and the conversation set summary text according to the matching summary text vector found. Finally, the summary text screening model inputs the current conversation summary text, the conversation set summary text and the conversation set into the conversation key points summary extraction model for subsequent processing.

[0068] During the matching process, the summary text screening model determines the matching result by calculating the similarity between the current conversation vector and each summary text vector in the summary text vector database, as well as the similarity between the conversation set vector and each summary text vector in the summary text vector database.

[0069] In order to filter out suitable matches, preset similarity thresholds are set for the model. Different thresholds are set to adapt to conversations of different lengths and complexities: the third preset similarity threshold is used to filter matches for the current conversation vector, and the fourth preset similarity threshold is used to filter matches for the conversation set vector. Only when the similarity of the summary text vector exceeds the corresponding threshold will it be selected as a match result.

[0070] Through vector conversion and similarity calculation, the model can more accurately find summary text that is closer to the current conversation and conversation set content, provide more valuable information for summarizing and extracting the key points of the conversation, and ensure the accuracy of conversation analysis. In addition, the vectorized representation and efficient similarity calculation algorithm enable the model to quickly complete the matching task, reduce processing time, and improve the response speed of the overall system.

[0071] Specifically, the conversation process between the user and the customer service is as follows:

[0072] (1) A user or customer service initiates a new conversation.

[0073] (2) Send the conversation to the text2vector model to generate a vector. Search for matches in the tables corresponding to different vector databases. The result is [X], where [X] is a dictionary list in the form of [{vector1: score1}, {vector2: score2}]. Take the part whose score value is greater than the set threshold thrX. thrX is obtained through manual experience observation and is also related to the corpus trained by the text2vector model.

[0074] (3) Combine the past n conversations (n ​​is usually short, for example, 3 to 5) and send them to the text2vector model to generate a vector. Search for matches in the tables corresponding to different vector databases. The result [Y] is the same as [X]. Take the part whose score value is greater than the set threshold thrY.

[0075] (4) The two lists [X] and [Y] respectively search for the corresponding tables in the kv database to obtain the corresponding dialogue summary text, forming [X_dl] and [Y_dl].

[0076] (5) Send [X_dl], [Y_dl], the most recent n conversations, and the set prompt words as a whole character string Z to the LLM interface, so that LLM can summarize what the current customer is concerned about, what the core of the problem is, and what field the conversation belongs to in the company's internal sop (such as greetings, product introductions, customer situation surveys, etc.), and add field labels. The present invention does not limit prompt words, for example: 1. Please extract the conversation tags {Greetings: {{}}, Product introduction: {{}}, Customer situation survey: {{history}}} according to the following conversation and in the prompt format, the user conversation is as follows: {{dialogue}}; 2. The following conversation belongs to a customer situation survey. The company's customer guidance requirements for customer situation surveys are as follows {{customer survey}}. Please judge whether the consultant meets the requirements according to the above requirements. If there are any non-compliance, please list them.

[0077] In this round of conversation, the user and customer service can use the attention mechanism to match any records related to the current topic in the past historical conversations as chat records into the current attention window.

[0078] In one embodiment of the present invention,

[0079] The dialogue key points summary extraction model is also used to receive historical super-long dialogues, construct dialogue segment windows of various lengths and steps for each historical super-long dialogue through a sliding window mechanism, summarize the dialogue segments in each window, summarize each dialogue segment, generate a summary text for each dialogue segment and input it into the summary text screening model;

[0080] The summarized text screening model is also used to convert the summarized text of each dialogue segment into a summarized text vector of each dialogue segment, and store the summarized text vectors in corresponding tables of a summarized text vector database according to the window length and step size, and store the summarized text vector-summary text data pairs in corresponding tables of a summarized text vector pairing database according to the window length and step size.

[0081] The system of the present invention is a multi-level external memory system, which can construct a summary text vector database and a summary text vector pairing database by processing historical super-long conversations in advance. Specifically, the conversation is windowed and a sliding window mechanism with different lengths and steps is constructed. For example, one sliding window is 20 in length and 10 in step, so every 10 messages takes the past 20 conversation histories. The sliding window is continuously performed as the conversation continues.

[0082] For the conversation history that meets the sliding window and step size conditions, specific prompt words are constructed based on LLM to summarize the conversation. After summarization, the vector is generated using the text2vector model. It is stored in the vector database. The text and vector form a data pair, which is stored in the kv database, with the vector as the key and the text as the value. The present invention does not limit the prompt words. For example, please briefly summarize the following conversation {{dialogue}} based on the user's emotions, the user's main communication demands, the user's physical summary, the user's conversation topic, and how the consultant responds.

[0083] Data from different windows are stored in different tables and used to build different levels of memory. Long windows represent long memory compression, and short windows represent short memory compression.

[0084] In one embodiment of the present invention, the answer matching model is used to:

[0085] Converting the dialogue key points into dialogue key point vectors, and calculating the similarity between the dialogue key point vectors and each question vector in a question vector database, and taking question vectors exceeding a second preset similarity threshold as question vectors that match the dialogue key point vectors;

[0086] The answer corresponding to the question vector matching the conversation key point vector is searched in the question-answer pairing database, and provided to the customer service as a response prompt.

[0087] When the current conversation is the user speaking, the string Z is concatenated, and the current concatenated result Z is used by LLM to identify the focus of the current user's question, and then the focus is output as a sentence s to completely restate the user's words (natural language). For example, the user asks: [Can I eat this thing with my current physical condition? ], but the actual user mentioned last month that he had stomach discomfort recently and the doctor asked him to eat a light diet for a month. This thing refers to an endocrine regulating product from the company that she bought last month. The sentence s is vectorized by text2vector, and then the customer service specification knowledge base is searched for the N vectors closest to this vector, and the thresholds need to be greater than thrZ. Finally, the value corresponding to the matched key is extracted as the recommended reply and solution for the matching problem. In this way, customer service can be managed in a standardized manner.

[0088] In one embodiment of the present invention,

[0089] The answer pairing model is also used to receive a customer service specification document constructed in the form of a question-answer pair, convert each question in the customer service specification document into a question vector, classify the questions according to the field to which the questions belong and label them with field labels, store the question vectors in a question vector database, and store the question vector-answer data pairs in a question-answer pairing database.

[0090] You can collect the company's customer service specification sop in advance and build a customer service work knowledge base. The knowledge base is also built with a vector database and a kv database. The customer service manager uploads a customer service specification sop document constructed in the form of question and answer pairs to the system. The system uses the text2vector model to vectorize the questions in the document, and the vectors are stored in the vector database, and the vector and answer pairs are stored in the kv database.

[0091] In the customer service knowledge base, different questions are labeled with corresponding domain labels to increase the accuracy of content matching and add prior knowledge of a domain during recall, which is convenient for SOP scoring.

[0092] In one embodiment of the present invention,

[0093] The customer service super-long conversation reply assistance and evaluation system is also used to optimize the customer service specification document based on the points deducted from the customer service in the conversation, as well as user satisfaction, complaint rate, and transaction amount.

[0094] In the present invention, the points deducted by the customer service staff during the communication process with the customer can be compared and analyzed with customer satisfaction, complaint rate, transaction amount, etc., to optimize the SOP and improve the correlation between the SOP and customer satisfaction and transaction amount. The customer service specification document obtained after optimization to a high correlation can be used as a basis for automatic scoring of individual customer service staff. In addition, the system of the present invention can also further optimize the customer service specification document in the application based on the points deducted from the customer service staff during the conversation, as well as user satisfaction, complaint rate, and transaction amount.

[0095] Based on the same inventive concept, another embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored on the memory, wherein the processor executes the computer program to load the customer service long conversation response assistance and evaluation system as described in any one of the embodiments of the present invention.

[0096] Specifically, the electronic device includes: a memory and a processor, the memory and the processor are connected via a bus communication, a computer program is stored in the memory, and the computer program can be run on the processor, thereby loading the customer service long conversation response assistance and evaluation system described in any one of the first aspects of the embodiments of the present invention.

[0097] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0098] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0099] Based on the same inventive concept, another embodiment of the present invention provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the customer service long conversation response assistance and evaluation system as described in any one of the first aspects of the embodiments of the present invention is loaded.

[0100] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0101] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, devices, electronic devices, storage media or computer program products. Therefore, embodiments of the present invention may take the form of complete hardware embodiments, complete software embodiments, or embodiments combining software and hardware. Moreover, embodiments of the present invention may take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CDROM, optical storage, etc.) containing computer-usable program codes.

[0102] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0103] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0104] The above is a detailed introduction to a customer service extra-long conversation reply assistance and evaluation system, electronic device and readable storage medium provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and its core idea of ​​the present application; at the same time, for those of ordinary skill in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application. The above embodiments are only preferred embodiments cited to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or changes made by those skilled in the art on the basis of the present invention are all within the scope of protection of the present invention.

Claims

1. A customer service long conversation reply assistance and evaluation system, characterized in that: include: Summary text vector database, used to store the vectors corresponding to the summary texts of historical long conversations; A summary text vector pairing database, used to store vector-summary text data pairs; the vector-summary text data pairs are the vectors corresponding to each summary text of the historical super-long conversation and the corresponding summary text; Question vector database, used to store vectors corresponding to questions in customer service specification documents; A question-answer pairing database, used to store vector-answer data pairs; the vector-answer data pairs are the vector corresponding to each question in the customer service specification document and the corresponding answer in the customer service specification document; A summary text screening model is used to receive a current conversation and a conversation set obtained by merging multiple conversations closest to the current conversation, process the current conversation and the conversation set, and screen out matching summary texts according to a summary text vector database and a summary text vector pairing database, and input the matching summary text and the conversation set into a conversation key point summary extraction model; A dialogue key points summary extraction model, used to perform summary extraction on the matched summary text and the dialogue set to generate dialogue key points and domain labels; An answer matching model is used to process the key points of the conversation and screen out matching answers based on the question vector database and the question-answer pairing database, and provide them to the customer service as a response prompt; A customer service evaluation model is used to obtain a customer service reply, convert the reply into a reply vector, convert the matched answer into an answer vector, and calculate the similarity between the reply vector and the answer vector, and if the similarity between the reply vector and the answer vector is lower than a first preset similarity threshold, deduct points from the customer service, and score the customer service based on the total deductions.

2. The customer service long conversation reply assistance and evaluation system according to claim 1 is characterized in that: The answer matching model is used to: Converting the dialogue key points into dialogue key point vectors, and calculating the similarity between the dialogue key point vectors and each question vector in a question vector database, and taking question vectors exceeding a second preset similarity threshold as question vectors that match the dialogue key point vectors; The answer corresponding to the question vector matching the conversation key point vector is searched in the question-answer pairing database, and provided to the customer service as a response prompt.

3. The customer service long conversation reply assistance and evaluation system according to claim 1 is characterized in that: The answer pairing model is also used to receive a customer service specification document constructed in the form of a question-answer pair, convert each question in the customer service specification document into a question vector, classify the questions according to the field to which the questions belong and label them with field labels, store the question vectors in a question vector database, and store the question vector-answer data pairs in a question-answer pairing database.

4. The customer service long conversation reply assistance and evaluation system according to claim 1 is characterized in that: The customer service over-long conversation response assistance and evaluation system is also used to optimize customer service specification documents based on the points deducted from the customer service during the conversation, as well as user satisfaction, complaint rate, and transaction amount.

5. The customer service long conversation reply assistance and evaluation system according to claim 1 is characterized in that: The summarized text screening model is used to: Converting the current conversation into a current conversation vector, and searching the summary text vector database for a summary text vector that matches the current conversation vector; Converting the dialog set into a dialog set vector, and searching the summary text vector database for a summary text vector that matches the dialog set vector; Searching the summary text corresponding to the summary text vector matching the current conversation vector from the summary text vector pairing database to obtain the current conversation summary text; Searching the summary text corresponding to the summary text vector that matches the dialogue set vector from the summary text vector pairing database to obtain the dialogue set summary text; The current conversation summary text, the conversation set summary text and the conversation set are input into a conversation key point summary extraction model.

6. The customer service long conversation reply assistance and evaluation system according to claim 5 is characterized in that: The summarized text screening model is used to calculate the similarity between the current conversation vector and each summarized text vector in the summarized text vector database, and use the summarized text vector exceeding a third preset similarity threshold as the summarized text vector matched by the current conversation vector; The summarized text screening model is used to calculate the similarity between the dialog set vector and each summarized text vector in the summarized text vector database, and to use the summarized text vector exceeding a fourth preset similarity threshold as the summarized text vector matched by the dialog set vector.

7. The customer service long conversation reply assistance and evaluation system according to claim 1 is characterized in that: The dialogue key points summary extraction model is also used to receive historical super-long dialogues, construct dialogue segment windows of various lengths and steps for each historical super-long dialogue through a sliding window mechanism, summarize the dialogue segments in each window, summarize each dialogue segment, generate a summary text for each dialogue segment and input it into the summary text screening model; The summarized text screening model is also used to convert the summarized text of each dialogue segment into a summarized text vector of each dialogue segment, and store the summarized text vectors in corresponding tables of a summarized text vector database according to the window length and step size, and store the summarized text vector-summary text data pairs in corresponding tables of a summarized text vector pairing database according to the window length and step size.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the electronic device executes the computer program, the customer service long conversation response assistance and evaluation system as described in any one of claims 1 to 7 is loaded.

9. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the customer service ultra-long conversation response assistance and evaluation system as described in any one of claims 1 to 7 is loaded.

Images