Customer service quality evaluation method, device, equipment, medium and program product

Abstract

The application relates to a customer service quality evaluation method, device, equipment, medium and program product. The method comprises the following steps: obtaining a dialogue flow of a customer service and a user; performing text recognition on the dialogue flow to obtain dialogue text, and processing the dialogue text to obtain text features; performing audio recognition on the dialogue flow to obtain audio features; and performing evaluation according to the text features and the audio features to obtain customer service quality. The method can improve evaluation accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a customer service quality evaluation method, device, equipment, medium and program product. BACKGROUND

[0002] The function of telephone banking has gradually changed from service support to customer management. With the expansion of embedded marketing and outbound marketing in the customer service center, the quality consistency of business handling is the focus of the customer service center, and how to improve the service quality of customer service is the inevitable requirement of modern customer-oriented service. However, the traditional customer scoring mode is less and less valued by customers, and the actual operation personnel have many evasive means. Therefore, it is particularly important to improve an objective and fair evaluation system that is helpful to the dialogue technique.

[0003] The scoring of the traditional technology for customer service includes the following two kinds: one is to evaluate the service quality based on the scoring of the user, but the user scoring is not comprehensive and missed, and it occupies the time of the user, which is not objective; the second is emotion recognition based on artificial intelligence, which maps the emotion to the corresponding service quality, in addition, the classification of emotion is different for different companies, and it is difficult for many classification annotation personnel to distinguish; the mapping from emotion to service quality has information loss, and cannot guide the improvement of service quality end to end. At the same time, there are two mainstream ways of emotion recognition at present, one is text-based processing, which obtains corresponding emotional information according to the text. This method loses the voice information, resulting in inaccurate analysis effect; the other is emotion recognition based on voice, but the text information is lost.

[0004] Therefore, the scoring accuracy of the current customer service scoring system needs to be improved. SUMMARY

[0005] Therefore, it is necessary to provide a customer service quality evaluation method, device, equipment, medium and program product capable of improving the evaluation accuracy in view of the above technical problems.

[0006] In a first aspect, the present application provides a customer service quality evaluation method, which comprises:

[0007] obtaining a dialogue flow between a customer service and a user;

[0008] performing text recognition on the dialogue flow to obtain dialogue text, and processing the dialogue text to obtain text features;

[0009] performing audio recognition on the dialogue flow to obtain audio features;

[0010] evaluating according to the text features and the audio features to obtain customer service quality.

[0011] In one of the embodiments, the processing of the dialogue text obtains text features, including:

[0012] The dialogue text is segmented to obtain segmented words;

[0013] The analysis is calculated to obtain word features;

[0014] The word features are processed to obtain sentence features;

[0015] According to the sentence features and the corresponding position of the current sentence in the dialogue text, the text features are obtained.

[0016] In one of the embodiments, the audio recognition of the dialogue flow obtains audio features, including:

[0017] The dialogue flow is audio feature extracted, and the audio features include one or more of volume change matrix, speech rate change matrix and tone change matrix.

[0018] In one of the embodiments, the evaluation according to the text features and the audio features obtains the quality of customer service, including:

[0019] The text features and the audio features are input into the pre-trained evaluation model to obtain the quality of customer service; wherein the evaluation model is trained according to historical service data and customer ratings.

[0020] In one of the embodiments, the method further comprises:

[0021] Obtaining the operation data of the customer service on the business system according to the dialogue flow;

[0022] Matching the operation data with the text features;

[0023] When the operation data and the text features do not match, the operation corresponding to the operation data is blocked;

[0024] When the operation data and the text features match, the operation according to the operation data is continued.

[0025] In one of the embodiments, the matching of the operation data with the text features comprises:

[0026] The operation data is matched with the corresponding intent and slot value in the text features.

[0027] In one of the embodiments, the method further comprises:

[0028] Emotion information is obtained by performing emotion recognition on the voice of the user in a dialogue flow between the user and the intelligent customer service;

[0029] When the emotion information does not meet the requirement, a communication channel between the user and the manual customer service is established.

[0030] In one of the embodiments, the method further includes:

[0031] According to the quality of the customer service of the manual customer service, a corresponding customer service script is obtained;

[0032] The script is optimized to obtain a script template, and the script template is used to guide the intelligent customer service to provide customer service.

[0033] In a second aspect, the application further provides a customer service quality evaluation device, and the device includes:

[0034] A dialogue flow acquisition module is configured to acquire a dialogue flow between a customer service and a user;

[0035] A text feature recognition module is configured to perform text recognition on the dialogue flow to obtain dialogue text, and perform processing on the dialogue text to obtain text features;

[0036] An audio feature recognition module is configured to perform audio recognition on the dialogue flow to obtain audio features;

[0037] An evaluation module is configured to evaluate the text features and the audio features to obtain the quality of the customer service.

[0038] In a third aspect, the application further provides a computer device including a memory and a processor, the memory stores a computer program, and the processor implements the steps of the method in any one of the above embodiments when executing the computer program.

[0039] In a fourth aspect, the application further provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the method in any one of the above embodiments when executed by a processor.

[0040] In a fifth aspect, the application further provides a computer program product including a computer program, and the computer program implements the steps of the method in any one of the above embodiments when executed by a processor.

[0041] The customer service quality evaluation method, device, equipment, medium and program product described above obtain a dialogue flow of the customer and the user, perform text recognition on the dialogue flow to obtain dialogue text, perform audio recognition on the dialogue flow to obtain audio data, and then evaluate the customer service quality according to the dialogue text and the audio data, so as to realize end-to-end processing, on the one hand, without the need for the user to score, and on the other hand, without the need for the emotion to be recognized in advance, so that information is not lost, and the accuracy of the evaluation is ensured. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 An application environment diagram of the customer service quality evaluation method in an embodiment;

[0043] Figure 2 A flowchart of the customer service quality evaluation method in an embodiment;

[0044] Figure 3 A model diagram of a text feature extraction flow in an embodiment;

[0045] Figure 4 A whole architecture diagram of the customer service quality evaluation method in an embodiment;

[0046] Figure 5 A framework diagram of the customer service quality evaluation method in an embodiment;

[0047] Figure 6 A structural block diagram of the customer service quality evaluation device in an embodiment;

[0048] Figure 7 An internal structure diagram of the computer equipment in an embodiment. DETAILED DESCRIPTION

[0049] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0050] The customer service quality evaluation method provided in the embodiments of the present application can be applied in an application environment as shown in Figure 1 The terminal 102 communicates with the server 104 through a network. The data storage system can store data required to be processed by the server 104. The data storage system can be integrated on the server 104, or placed on a cloud or other network server.

[0051] The terminal 102 can have a conversation with the customer service through the server 104 to obtain customer service provided by the customer service. The server 104 can obtain a conversation flow of the customer service and the user, perform text recognition on the conversation flow to obtain conversation text, process the conversation text to obtain text features, perform audio recognition on the conversation flow to obtain audio features, and evaluate the text features and the audio features to obtain the quality of the customer service. In this way, end-to-end processing is realized. On the one hand, the user does not need to perform scoring, and on the other hand, the emotion does not need to be recognized in advance, so that information is not lost, and the accuracy of the evaluation is ensured.

[0052] The terminal 102 can be various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices, but is not limited thereto. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle-mounted device, and the like. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, and the like. The server 104 can be implemented by a single server or a server cluster composed of multiple servers.

[0053] In one embodiment, as shown in Figure 2 , a customer service quality evaluation method is provided. The method is applied to the server in Figure 1 , and includes the following steps:

[0054] S202: Obtain a conversation flow of the customer service and the user.

[0055] Specifically, the user and the customer service have a conversation through an interactive voice response (IVR) module, so that the conversation flow can be obtained from the interactive voice response module.

[0056] It should be noted that the customer service includes intelligent customer service and manual customer service. The intelligent customer service refers to customer service that provides customer service according to a script template and pre-set business logic, and the manual customer service refers to a real person customer service.

[0057] The server refers to the entire back end, which can include the interactive voice response module, so that the customer service can have a conversation with the user through the interactive voice response module.

[0058] S204: Perform text recognition on the conversation flow to obtain conversation text, and process the conversation text to obtain text features.

[0059] Specifically, the dialogue text refers to the natural language obtained by converting speech during a dialogue. Preferably, the server includes an ASR (Automatic Speech Recognition) module, which is connected to the interactive voice response module. The ASR module is used to read the dialogue stream between the customer service representative and the user in the interactive voice response module and convert it into dialogue text. Preferably, the ASR module converts the dialogue stream into dialogue text similar to "User A: aaaa; User B: bbb…", which not only obtains the dialogue but also indicates who spoke each sentence.

[0060] Text features are obtained by processing the dialogue text. Preferably, text features can be represented by semantic vectors. More preferably, the server can input the dialogue text into a pre-trained semantic recognition model for processing to obtain semantic vectors.

[0061] For ease of understanding, the semantic vector includes not only the features of the current sentence but also contextual features, such as the features of the previous sentence. Optionally, the server can first segment each sentence into words, then generate word features corresponding to each segment using the BERT model. Concatenating these word features yields the sentence representation, which is then input into a pre-trained sentence feature extraction model to obtain sentence features. Finally, the sentence features, along with features from previous sentences, are input into a discourse semantic extraction model to extract text features. Therefore, the resulting text features can encompass the features of the entire dialogue flow without loss of features.

[0062] S206: Perform audio recognition on the dialogue stream to obtain audio features.

[0063] Specifically, the audio features include one or more of the volume variation matrix, speech rate variation matrix, and tone variation matrix. The server can directly obtain the dialogue stream from the interactive speech response module and then input the dialogue stream into the feature extraction part of the pre-trained evaluation model or a separate audio feature extraction model to extract the audio features of the dialogue stream.

[0064] S208: Customer service quality is assessed based on text and audio features.

[0065] Specifically, customer service quality can be expressed through a score or a rating system; no specific restrictions are imposed here.

[0066] In this embodiment, text features and audio features are input into a pre-trained evaluation model for calculation to obtain customer service quality. The evaluation model is trained based on historical service data and customer ratings. This eliminates the need for prior emotion recognition; it directly evaluates customer service quality based on audio and text features, containing more information and thus achieving higher accuracy.

[0067] Specifically, the server extracts specific feature information to define the feature matrix of audio text information features. At the same time, the scoring of customer service performance is based on existing information in the existing system. This directly links customer service quality with the feature matrix, enabling end-to-end model annotation, training, and prediction.

[0068] The training method for the evaluation model includes: acquiring historical customer service data, which includes historical dialogue streams between customer service representatives and users; extracting corresponding historical text features and historical audio features from this historical dialogue stream as described above; using these historical text features and historical audio features as input to the evaluation model to be trained; obtaining user ratings or grades for this historical customer service data from the system, or quality control ratings or grades for this historical customer service data; and using these ratings or grades as output to the evaluation model to be trained. Thus, the evaluation model is trained based on historical text features, historical audio features, and corresponding ratings or grades. Optionally, the server can also optimize the evaluation model for continuous iteration, thereby guiding the generation of script templates.

[0069] The aforementioned customer service quality assessment method acquires the dialogue flow between customer service representatives and users; performs text recognition on the dialogue flow to obtain the dialogue text; and performs audio recognition on the dialogue flow to obtain audio data. In this way, the customer service quality is assessed based on the dialogue text and audio data, achieving end-to-end processing. On the one hand, users do not need to score the service, and on the other hand, there is no need to identify emotions beforehand, thus avoiding information loss and ensuring the accuracy of the assessment.

[0070] In one embodiment, the dialogue text is processed to obtain text features, including: segmenting the current sentence in the dialogue text to obtain word segments; calculating and analyzing the corresponding word features; processing the word features to obtain sentence features; and processing the sentence features and the position of the corresponding current sentence in the dialogue text to obtain text features.

[0071] Specifically, in combination Figure 3 As shown, Figure 3 This is a model diagram of the text feature extraction process in one embodiment. In this embodiment, to solve the problem of misinterpretation, the meaning of words is associated with the context, and a deep neural network is used to achieve text evaluation at the chapter level.

[0072] wherein Figure 3 For example, it includes three sentences, and each sentence is segmented to obtain specific words in the sentence, wherein the segmentation can be obtained based on existing analysis methods. Preferably, the segmentation logic can be obtained according to predefined template words to improve segmentation accuracy.

[0073] wherein the server inputs the segmentation into a pre-trained bert model to obtain the word features corresponding to the segmentation, preferably word vectors, and then splices the word vectors to obtain a sentence vector, inputs the sentence vector into a convolutional neural network to obtain a sentence representation, i.e., a sentence feature, and inputs the sentence feature or the sentence feature together with the previous sentence feature into a Gated Recurrent neural network to obtain a chapter representation, i.e., a text feature.

[0074] For the convenience of understanding, take the dialogue flow "Customer Service: Hello, sir. I'm glad to serve you. Customer: Hello, I want to buy An Enjoy. Customer Service: OK, how long do you want to buy?" as an example for illustration, wherein the three sentences correspond to Figure 3 the middle front and middle back word representations, and each sentence obtains its word representation through a bert model (a general text conversion vector model), that is, the vector at the bottom of the figure. Then the sentence representation vector of each sentence is obtained through sentence semantic combination. Finally, the chapter representation is obtained through chapter semantic combination, and finally sent to the evaluation model for scoring.

[0075] In one embodiment, audio recognition is performed on the dialogue flow to obtain audio features, including: performing audio feature extraction on the dialogue flow, and the audio features include one or more of a volume change matrix, a speech speed change matrix, and a tone change matrix.

[0076] Specifically, the audio features include one or more of a volume change matrix, a speech speed change matrix, and a tone change matrix, wherein the server can directly obtain the dialogue flow from the interactive voice response module, and then input the dialogue flow into the feature extraction part of the pre-trained evaluation model or a separate audio feature extraction model to extract the audio features of the dialogue flow.

[0077] In the above embodiment, through the quantitative coding of the explicit feature matrix of the voice text of the customer service and the user, the service score is automatically collected through this means, and a closed-loop optimization of the quality of customer service is formed, replacing the incomplete coverage and subjective randomness of the user's subjective scoring, and the service quality of the customer service is effectively improved.

[0078] In one of the embodiments, the customer service evaluation method further comprises: obtaining operation data of the customer service on the business system according to the dialogue flow; matching the operation data with the text features; when the operation data does not match the text features, blocking the operation corresponding to the operation data; and when the operation data matches the text features, continuing to operate according to the operation data.

[0079] Specifically, the operation data refers to operation data obtained by the customer service operating the business system according to the user's indication in the process of communication with the user. For example, the operation data corresponding to "I want to buy 10000 yuan of An Enjoy Win wealth management product for three years" can include opening the business system corresponding to the An Enjoy Win wealth management product and inputting 10000 yuan and 3 years of data.

[0080] The server can extract the operation data of the customer service on the business system, and then match the operation data with the text features. When the operation data does not match the text features, the operation corresponding to the operation data is blocked; and when the operation data matches the text features, the operation is continued according to the operation data.

[0081] More specifically, the customer service handles business in the business system through communication, the business system outputs handling content according to handling and outputs it to the semantic engine; the semantic engine compares the user's intention and the customer's handling content, calculates the similarity of the two according to the semantic metric, or matches whether it meets the intention through the rule engine, scores the degree of meeting the intention; according to the intention matching score, it is decided whether to block the business handling.

[0082] Specifically, if the customer wants to buy a 10000 yuan wealth management product, but the customer service mistakenly buys 1000 yuan, the amount comparison does not pass, and the business is prompted to block, at this time the business cannot purchase for the customer, only the corresponding monitoring personnel can release it to continue trading, which can effectively prevent risks.

[0083] In order to enable those skilled in the art to fully understand, the following examples are described:

[0084] The customer service will operate the business system in the process of communication, and the server can obtain the product information purchased by the customer, mainly detecting what product the customer wants to buy and how much money to buy, etc., and matching and comparing through the data returned by the business system.

[0085] The rule engine is a collection of countless semantics, such as a greeting rule including "Hello" "I am glad to serve you" and similar sentences, and hitting the greeting is a rule.

[0086] Blocking business is that if the customer wants to buy a 100,000 yuan financial product, the customer service mistakenly buys 1000 yuan, and the amount comparison does not pass at this time, which is the starting point of blocking, prompting the business, at this time the business cannot purchase for the customer, only the corresponding monitoring personnel can release, then the transaction can continue, which can effectively prevent risks.

[0087] In the above embodiment, the content of telephone banking business handling is compared in real time in advance through voice transcription, element extraction, semantic understanding and other artificial intelligence technologies, key process blocking risks are blocked, compliance requirements of business handling are ensured, and consumer rights are ensured.

[0088] In one of the embodiments, the operation data is matched with the text features, including: matching the operation data with the corresponding intent and slot value in the text features.

[0089] Among them, the text features include intent and slot value, where the intent refers to what a sentence wants to express, and the slot value refers to what the key elements in this sentence are, for example, "I want to buy 10000 yuan of An Enjoy Win financial product for three years, the intent is to buy An Enjoy Win, and the slot value is 10000 and three years respectively, which can be identified by an intent recognition model and an element extraction model.

[0090] In one of the embodiments, the customer service quality evaluation method further includes: performing emotion recognition on the user's voice in the user's dialogue flow with the intelligent customer service to obtain emotion information; and when the emotion information does not meet the requirements, establishing a communication channel between the user and the human customer service.

[0091] Specifically, when the customer service emotion is relatively angry and has a complaint intention, the IVR is transferred to the human customer service. Therefore, the server also identifies the emotions of the customer service and the user, so as to timely replace the human customer service.

[0092] In one of the embodiments, the customer service quality evaluation method further includes: obtaining a corresponding customer service script according to the customer service quality of the human customer service; and optimizing the customer service script to obtain a script template, which is used to guide the intelligent customer service to provide customer service.

[0093] Specifically, the evaluation model can score the service quality of the customer in actual application, for new business, the customer service script with the highest score can be scored, and the script can be optimized, such as removing adverbs and the like, to obtain a script template, which is used to guide the intelligent customer service to provide customer service.

[0094] After the new designed dialogue template is online, the matching degree of the two is calculated by scoring and evaluating the model to predict the score. In addition, the customer scoring is similar to the standard answer, and the model is not accurate at the beginning. The model is matched with the customer score and the model score to optimize the model to make the score closer to the real situation.

[0095] Specifically, in combination with Figure 4 and Figure 5 shown, Figure 4 is the overall architecture diagram of the customer service quality evaluation method in an embodiment, Figure 5 is a framework diagram of the customer service quality evaluation method in an embodiment.

[0096] In this embodiment, the user has a dialogue with the customer service through the IVR, and generates a dialogue flow, wherein Figure 4 The customer in the embodiment is represented by a person, but it can include intelligent customer service and manual customer service. The server processes the dialogue flow to extract audio features, and inputs the dialogue flow into the ASR to obtain dialogue text. Then the dialogue text is processed to obtain text features. The audio features and the text features are input into the evaluation model to obtain the customer service quality. In combination with Figure 5 In this embodiment, only audio features and text features are illustrated, and in other embodiments, other audio features and text features can be introduced as needed.

[0097] The customer handles the business in the business system through communication. The business system outputs the handling content and outputs it to the semantic engine according to the handling. The semantic engine compares the user intent and the customer handling content, calculates the similarity between the two according to the semantic metric, or matches whether it meets the intent through the rule engine, scores the intent satisfaction degree; according to the intent matching score, it is decided whether to block the business handling.

[0098] In the above embodiment, the content of the telephone banking business is compared in real time in advance through voice transcription, element extraction, semantic understanding and other artificial intelligence technologies, the key process blocking risk is blocked, the compliance requirements of the business handling are ensured, and the consumer rights are ensured. Through the quantitative coding of the explicit feature matrix of the voice text of the customer service and the user, the service score is automatically collected through this means, and a closed-loop optimization of the customer service quality improvement is formed, instead of the incomplete coverage and subjective randomness of the user's subjective scoring, and the service quality of the customer service is effectively improved

[0099] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0100] Based on the same inventive concept, this application also provides a customer service quality assessment device for implementing the aforementioned customer service quality assessment method. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more customer service quality assessment device embodiments provided below can be found in the limitations of the customer service quality assessment method described above, and will not be repeated here.

[0101] In one embodiment, such as Figure 6 As shown, a customer service quality assessment device is provided, including: a dialogue flow acquisition module 601, a text feature recognition module 602, an audio feature recognition module 603, and an assessment module 604, wherein:

[0102] Dialogue Flow Acquisition Module 601 is used to acquire the dialogue flow between customer service and users;

[0103] The text feature recognition module 602 is used to perform text recognition on the dialogue stream to obtain the dialogue text, and to process the dialogue text to obtain text features.

[0104] The audio feature recognition module 603 is used to perform audio recognition on the dialogue stream to obtain audio features;

[0105] Evaluation module 604 is used to evaluate customer service quality based on text and audio features.

[0106] In one embodiment, the text feature recognition module 602 includes:

[0107] The word segmentation unit is used to segment the current sentence in the dialogue text to obtain word segments;

[0108] The word feature calculation unit is used to calculate and analyze the corresponding word features.

[0109] The sentence feature calculation unit is used to process word features to obtain sentence features;

[0110] a text feature calculation unit configured to calculate a text feature according to the sentence feature and a position of the corresponding current sentence in the dialogue text.

[0111] In one of the embodiments, the audio feature module is configured to extract audio features from the dialogue flow, the audio features including one or more of a volume variation matrix, a speech speed variation matrix, and a tone variation matrix.

[0112] In one of the embodiments, the evaluation module 604 is configured to input the text feature and the audio feature into a pre-trained evaluation model to calculate the customer service quality, wherein the evaluation model is trained according to historical service data and customer ratings.

[0113] In one of the embodiments, the customer service quality evaluation apparatus further includes:

[0114] an operation data acquisition module configured to acquire operation data of the customer on the business system according to the dialogue flow;

[0115] a matching module configured to match the operation data with the text feature;

[0116] a matching processing module configured to, when the operation data does not match the text feature, block an operation corresponding to the operation data, and when the operation data matches the text feature, continue to perform the operation according to the operation data.

[0117] In one of the embodiments, the matching module is configured to match the operation data with corresponding intents and slot values in the text feature.

[0118] In one of the embodiments, the customer service quality evaluation apparatus further includes:

[0119] an emotion recognition module configured to recognize emotions of the user in the dialogue flow between the user and the intelligent customer service to obtain emotion information;

[0120] a customer switching module configured to, when the emotion information does not meet a requirement, establish a communication channel between the user and a human customer service.

[0121] In one of the embodiments, the customer service quality evaluation apparatus further includes:

[0122] a script acquisition module configured to acquire a corresponding customer service script according to the customer service quality of the human customer service;

[0123] a template generation module configured to optimize the customer service script to obtain a script template, the script template being used to guide the intelligent customer service to provide customer service.

[0124] The modules in the customer service quality evaluation device can be implemented by software, hardware, or a combination thereof. The modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in the computer device in software form, so that the processor can call and execute the operations corresponding to the modules.

[0125] In one embodiment, a computer device, which can be a server, has an internal structure as shown in Figure 7 The computer device includes a processor, a memory, and a network interface connected by a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with external terminals through network connections. The computer program is executed by the processor to implement a customer service quality evaluation method.

[0126] Those skilled in the art can understand that Figure 7 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0127] In one embodiment, a computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the following steps are implemented: obtaining a dialogue flow between a customer service and a user; performing text recognition on the dialogue flow to obtain dialogue text, and processing the dialogue text to obtain text features; performing audio recognition on the dialogue flow to obtain audio features; and evaluating based on the text features and the audio features to obtain a customer service quality.

[0128] In one embodiment, when the processor executes the computer program, the processing of the dialogue text to obtain text features includes: performing word segmentation on a current sentence in the dialogue text to obtain segmented words; calculating word features corresponding to the segmented words; processing the word features to obtain sentence features; and processing the sentence features and a position of the current sentence in the dialogue text to obtain text features.

[0129] In one embodiment, when the processor executes the computer program, the audio recognition on the dialogue flow to obtain audio features includes: performing audio feature extraction on the dialogue flow, the audio features including one or more of a volume change matrix, a speech speed change matrix, and a tone change matrix.

[0130] In one embodiment, the processor, when executing the computer program, implements the evaluation of the customer service quality according to the text features and the audio features, including: inputting the text features and the audio features into a pre-trained evaluation model to calculate the customer service quality; wherein the evaluation model is trained according to historical service data and customer ratings.

[0131] In one embodiment, the processor, when executing the computer program, further implements the following steps: obtaining operation data of the customer service on the business system according to the dialogue flow; matching the operation data with the text features; when the operation data does not match the text features, blocking the operation corresponding to the operation data; when the operation data matches the text features, then continuing to operate according to the operation data.

[0132] In one embodiment, the processor, when executing the computer program, implements the matching of the operation data with the text features, including: matching the operation data with the corresponding intent and slot value in the text features.

[0133] In one embodiment, the processor, when executing the computer program, further implements the following steps: performing emotion recognition on the user's voice in the dialogue flow between the user and the intelligent customer service to obtain emotion information; when the emotion information does not meet the requirements, establishing a communication channel between the user and the human customer service.

[0134] In one embodiment, the processor, when executing the computer program, further implements the following steps: obtaining the corresponding customer service script according to the customer service quality of the human customer service; optimizing the customer service script to obtain a script template, which is used to guide the intelligent customer service to provide customer service.

[0135] In one embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the following steps: obtaining a dialogue flow between a customer service and a user; performing text recognition on the dialogue flow to obtain dialogue text, and processing the dialogue text to obtain text features; performing audio recognition on the dialogue flow to obtain audio features; and evaluating the customer service quality according to the text features and the audio features.

[0136] In one embodiment, the computer program is executed by the processor to implement the processing of the dialogue text to obtain the text features, including: performing word segmentation on a current sentence in the dialogue text to obtain segmented words; calculating corresponding word features; processing the word features to obtain sentence features; and processing the sentence features and the position of the corresponding current sentence in the dialogue text to obtain the text features.

[0137] In an embodiment, the computer program, when executed by the processor, implements the audio recognition of the dialogue flow to obtain the audio features, including: performing audio feature extraction on the dialogue flow, the audio features including one or more of a volume change matrix, a speech speed change matrix, and a tone change matrix.

[0138] In an embodiment, the computer program, when executed by the processor, implements the evaluation according to the text features and the audio features to obtain the customer service quality, including: inputting the text features and the audio features into a pre-trained evaluation model to perform calculation to obtain the customer service quality; wherein the evaluation model is trained according to historical service data and customer ratings.

[0139] In an embodiment, the computer program, when executed by the processor, further implements the following steps: obtaining operation data of the customer service on the business system according to the dialogue flow; matching the operation data with the text features; when the operation data does not match the text features, blocking the operation corresponding to the operation data; when the operation data matches the text features, then continuing to perform operation according to the operation data.

[0140] In an embodiment, the computer program, when executed by the processor, implements the matching of the operation data with the text features, including: matching the operation data with corresponding intents and slot values in the text features.

[0141] In an embodiment, the computer program, when executed by the processor, further implements the following steps: performing emotion recognition on the voice of the user in the dialogue flow between the user and the intelligent customer service to obtain emotion information; when the emotion information does not meet the requirements, establishing a communication channel between the user and the human customer service.

[0142] In an embodiment, the computer program, when executed by the processor, further implements the following steps: according to the customer service quality of the human customer service, obtaining corresponding customer service scripts; optimizing the customer service scripts to obtain a script template, the script template being used to guide the intelligent customer service to provide customer service.

[0143] In an embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, implements the following steps: obtaining a dialogue flow between a customer service and a user; performing text recognition on the dialogue flow to obtain dialogue text, and processing the dialogue text to obtain text features; performing audio recognition on the dialogue flow to obtain audio features; and evaluating according to the text features and the audio features to obtain a customer service quality.

[0144] In an embodiment, the computer program, when executed by the processor, implements processing of the dialogue text to obtain text features, including: performing word segmentation processing on a current sentence in the dialogue text to obtain segmented words; calculating word features corresponding to the segmented words; processing the word features to obtain sentence features; and processing the sentence features and a position of the current sentence in the dialogue text to obtain the text features.

[0145] In an embodiment, the computer program, when executed by the processor, implements audio recognition of the dialogue flow to obtain audio features, including: performing audio feature extraction on the dialogue flow, the audio features including one or more of a volume change matrix, a speech speed change matrix, and a tone change matrix.

[0146] In an embodiment, the computer program, when executed by the processor, implements evaluation according to the text features and the audio features to obtain a quality of service, including: inputting the text features and the audio features into a pre-trained evaluation model to calculate the quality of service; and wherein the evaluation model is trained according to historical service data and customer ratings.

[0147] In an embodiment, the computer program, when executed by the processor, further implements the following steps: obtaining operation data of a service system by a customer service according to the dialogue flow; matching the operation data with the text features; when the operation data and the text features do not match, blocking an operation corresponding to the operation data; and when the operation data and the text features match, continuing to operate according to the operation data.

[0148] In an embodiment, the computer program, when executed by the processor, implements matching of the operation data with the text features, including: matching the operation data with corresponding intents and slot values in the text features.

[0149] In an embodiment, the computer program, when executed by the processor, further implements the following steps: performing emotion recognition on a voice of a user in the dialogue flow between the user and the intelligent customer service to obtain emotion information; and when the emotion information does not meet a requirement, establishing a communication channel between the user and a human customer service.

[0150] In an embodiment, the computer program, when executed by the processor, further implements the following steps: obtaining corresponding customer service scripts according to a quality of service of the human customer service; and optimizing the customer service scripts to obtain a script template, the script template being used to guide the intelligent customer service to provide the customer service.

[0151] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties.

[0152] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0153] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0154] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for evaluating quality of service in a customer service, characterized by, The method comprises: obtaining a dialogue flow of a customer service and a user; text recognition is performed on the dialogue flow to obtain dialogue text, and the dialogue text is processed to obtain text features; audio recognition is performed on the dialogue flow to obtain audio features, including: inputting the dialogue flow into the feature extraction part of the pre-trained evaluation model to obtain the audio features; the text features and the audio features are input into the pre-trained evaluation model to calculate the customer service quality; wherein the text recognition on the dialogue flow to obtain dialogue text comprises converting the dialogue flow into dialogue text in the form of "user A:aaaa; user B:bbb……"; the processing of the dialogue text to obtain text features comprises: performing word segmentation processing on the current sentence in the dialogue text to obtain segmented words; word features corresponding to the segmented words are calculated; sentence features are obtained according to the word features; the text features are obtained according to the sentence features and the positions of the corresponding current sentences in the dialogue text, including: inputting the sentence features and the features of the historical sentences of the sentence into a chapter semantic extraction model to obtain the text features, and the text features include the features of the entire dialogue flow.

2. The method of claim 1, wherein, the audio recognition of the dialogue flow to obtain audio features comprises: audio feature extraction is performed on the dialogue flow, and the audio features include one or more of volume change matrix, speech rate change matrix and tone change matrix.

3. The method of claim 1, wherein, the evaluation according to the text features and the audio features to obtain the customer service quality comprises: the text features and the audio features are input into the pre-trained evaluation model to calculate the customer service quality; wherein the evaluation model is trained according to historical service data and customer ratings.

4. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: obtaining operation data of the customer service on a business system according to the dialogue flow; matching the operation data with the text features; when the operation data does not match the text features, blocking the operation corresponding to the operation data; when the operation data matches the text features, then continue to operate according to the operation data.

5. The method of claim 4, wherein, the matching of the operation data with the text features comprises: matching the operation data with the corresponding intent and slot value in the text features.

6. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: performing emotion recognition on the user's voice in the dialogue flow between the user and the intelligent customer service to obtain emotion information; when the emotion information does not meet the requirements, establishing a communication channel between the user and the artificial customer service.

7. The method of claim 6, wherein, The method further comprises: obtaining the corresponding customer service script according to the customer service quality of the artificial customer service; optimizing the script to obtain a script template, which is used to guide the intelligent customer service to provide customer service.

8. A device for evaluating quality of service, characterized by comprising: The device comprises: a dialogue flow acquisition module for obtaining a dialogue flow of a customer service and a user; a text feature recognition module for performing text recognition on the dialogue flow to obtain dialogue text, and processing the dialogue text to obtain text features; An audio feature recognition module is configured to perform audio recognition on the dialogue flow to obtain audio features, including: inputting the dialogue flow into a feature extraction part of a pre-trained evaluation model to obtain the audio features; An evaluation module is configured to evaluate the service quality based on the text features and the audio features, including: inputting the text features and the audio features into the pre-trained evaluation model to obtain the service quality; The text feature recognition module is further configured to convert the dialogue flow into dialogue text in the form of "user A: aaaa; user B: bbb……". The text feature recognition module is further configured to perform word segmentation on the current sentence in the dialogue text to obtain segmented words. The processor executes the computer program to implement the steps of the method in any one of claims 1 to 7. The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 7. The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 7. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. ​ 10. A computer-readable storage medium having stored thereon a computer program, characterized in that, ​ 11. A computer program product comprising a computer program, characterized in that, ​

Images