Service quality evaluation and model training method, electronic device and storage medium

Abstract

The embodiment of the present application provides a service quality assessment and model training method, an electronic device and a storage medium, wherein the service quality assessment method includes: obtaining a service conversation text to be evaluated for service quality, wherein the service conversation text includes multiple rounds of conversations between a customer and a customer service representative; extracting multiple rounds of customer conversations corresponding to the customer from the multiple rounds of conversations, and performing emotion recognition on the multiple rounds of customer conversations to obtain multiple corresponding emotion recognition results; performing representation aggregation on multiple emotion representations corresponding to the multiple emotion recognition results and semantic representations of the multiple rounds of customer conversations corresponding to the multiple emotion recognition results to obtain an aggregated representation; and performing service quality assessment on the service conversation text based on the aggregated representation. Through the embodiment of the present application, the service quality of the customer service can be evaluated more comprehensively and objectively.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of artificial intelligence technology, and in particular to a service quality assessment method, a service quality assessment model training method, and corresponding electronic equipment and computer storage media. Background Art

[0002] With the rise of various service industries, providing customers with satisfactory customer services (customer service for short) has become increasingly important and has become a key factor in supporting the sustainable development of the industry.

[0003] In various customer service scenarios, the evaluation of customer satisfaction is of great significance to improving customer service quality. However, since most existing service quality evaluations are based on manual evaluations of the customers being served, and the parameters used are also very limited, the evaluation results are not accurate enough.

[0004] Therefore, how to automatically evaluate the service quality of customer service with the assistance of algorithms to obtain more accurate evaluation results has become an urgent problem to be solved. Summary of the invention

[0005] In view of this, an embodiment of the present application provides a service quality evaluation solution to at least partially solve the above problems.

[0006] According to a first aspect of an embodiment of the present application, a service quality assessment method is provided, comprising: obtaining a service conversation text to be evaluated for service quality, wherein the service conversation text includes multiple rounds of conversations between a customer and a customer service representative; extracting multiple rounds of customer conversations corresponding to the customer from the multiple rounds of conversations, and performing emotion recognition on the multiple rounds of customer conversations respectively to obtain corresponding multiple emotion recognition results; performing representation aggregation on multiple emotion representations corresponding to the multiple emotion recognition results and semantic representations of the multiple rounds of customer conversations corresponding to the multiple emotion recognition results to obtain an aggregated representation; and performing service quality assessment on the service conversation text based on the aggregated representation.

[0007] According to a second aspect of an embodiment of the present application, a method for training a service quality assessment model is provided, comprising: obtaining a service conversation text sample, the service conversation text sample comprising a multi-round conversation sample between a customer and a customer service representative; generating a training sample based on a multi-round customer conversation sample corresponding to the customer extracted from the multi-round conversation sample and a service quality label corresponding to the multi-round customer conversation sample; performing emotion recognition on the multi-round customer conversation samples respectively through the service quality assessment model to be trained to obtain a corresponding plurality of emotion recognition results; performing representation aggregation on a plurality of emotion representations corresponding to the plurality of emotion recognition results and semantic representations of the multi-round customer conversation samples corresponding to the plurality of emotion recognition results to obtain an aggregated representation; performing service quality assessment prediction on the service conversation text sample based on the aggregated representation to obtain a prediction result; and training the service quality assessment model based on the difference between the prediction result and the service quality label.

[0008] According to the third aspect of an embodiment of the present application, there is provided an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method described in the first aspect or the second aspect.

[0009] According to a fourth aspect of an embodiment of the present application, a computer storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described in the first aspect or the second aspect is implemented.

[0010] According to a fifth aspect of an embodiment of the present application, a computer program product is provided, including computer instructions, wherein the computer instructions instruct a computing device to perform operations corresponding to the method described in the first aspect or the second aspect.

[0011] According to the solution provided in the embodiment of the present application, on the one hand, when conducting service quality evaluation, it is no longer based on manual evaluation or single-sentence service dialogue, but on multiple rounds of dialogue with customers. Through multiple rounds of dialogue, the customer's service situation in the service can be more comprehensively and accurately understood, and thus a more accurate service quality evaluation result can be obtained; on the other hand, when conducting service quality evaluation, the customer's emotional change information (through multiple emotional expressions) and semantic information (through semantic expressions) are combined. That is, the service quality of customer service is no longer determined solely by semantic information, but through the combination of emotional information and semantic information of multiple rounds of dialogue, the customer's emotional changes throughout the process can be understood, thereby evaluating the service quality more comprehensively and objectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0013] Figure 1 A schematic diagram of an exemplary system applicable to the embodiment of the present application;

[0014] Figure 2A A flowchart of the steps of a service quality assessment model training method according to Embodiment 1 of the present application;

[0015] Figure 2B for Figure 2A A structural diagram of a service quality assessment model training in the illustrated embodiment;

[0016] Figure 3A A flowchart of a method for evaluating quality of service according to Embodiment 2 of the present application;

[0017] Figure 3B for Figure 3A A schematic diagram of an example scenario in the illustrated embodiment;

[0018] Figure 4 This is a schematic diagram of the structure of an electronic device according to the third embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.

[0020] The specific implementation of the embodiment of the present application is further explained below in conjunction with the accompanying drawings of the embodiment of the present application.

[0021] Figure 1 An exemplary system applicable to the embodiment of the present application is shown. Figure 1 As shown, the system 100 may include a server 102, a communication network 104 and / or one or more user devices 106. Figure 1 The example in FIG. 1 is a plurality of user devices.

[0022] The server 102 may be any suitable server for storing information, data, programs, and / or any other suitable type of content. In some embodiments, the server 102 may perform any suitable function. For example, in some embodiments, the server 102 may be used to perform a service quality assessment in a customer service scenario. As an optional example, in some embodiments, the server 102 may be used to perform a service quality assessment for customer service based on a service conversation text including multiple rounds of customer and customer service. As another example, in some embodiments, the server 102 may be used to perform emotion recognition based on multiple rounds of customer conversations on the customer side of the service conversation text; and to perform a service quality assessment for customer service by combining the emotion recognition results and the semantic information of the multiple rounds of customer conversations. As another example, in some embodiments, the server 102 may be used to send a service quality assessment result to a user device.

[0023] In some embodiments, the communication network 104 can be any suitable combination of one or more wired and / or wireless networks. For example, the communication network 104 can include any one or more of the following: the Internet, an intranet, a wide area network (WAN), a local area network (LAN), a wireless network, a digital subscriber line (DSL) network, a frame relay network, an asynchronous transfer mode (ATM) network, a virtual private network (VPN) and / or any other suitable communication network. The user device 106 can be connected to the communication network 104 via one or more communication links (e.g., communication link 112), and the communication network 104 can be linked to the server 102 via one or more communication links (e.g., communication link 114). The communication link can be any communication link suitable for transmitting data between the user device 106 and the server 102, such as a network link, a dial-up link, a wireless link, a hard-wired link, any other suitable communication link or any suitable combination of such links.

[0024] The user device 106 may include any one or more user devices suitable for collecting conversation data between a customer and a customer service representative and sending the data to the server for evaluating the service quality of the customer service representative. In some embodiments, the user device 106 may include any suitable type of device. For example, in some embodiments, the user device 106 may include a mobile device, a tablet computer, a laptop computer, a desktop computer, a wearable computer, a game console, a media player, a vehicle entertainment system, and / or any other suitable type of user device.

[0025] Although server 102 is illustrated as one device, in some embodiments, any suitable number of devices may be used to perform the functions performed by server 102. For example, in some embodiments, multiple devices may be used to implement the functions performed by server 102. Alternatively, the functions of server 102 may be implemented using cloud services.

[0026] Based on the above system, the solution of the present application is described below through multiple embodiments. For ease of understanding, the present application embodiment first describes the training of the service quality evaluation model, and then describes the application of the trained service quality evaluation model.

[0027] Embodiment 1

[0028] Reference Figure 2A , shows a step flow chart of a service quality assessment model training method according to embodiment 1 of the present application.

[0029] The service quality assessment model training method of this embodiment includes the following steps:

[0030] Step S202: Obtain a service dialogue text sample.

[0031] In the embodiments of the present application, the service conversation text samples may be text samples of conversations between customers and customer service personnel in any service industry, such as service conversation text samples of e-commerce services, service conversation text samples of medical services, service conversation text samples of telecommunication operation services, and so on, and the embodiments of the present application are not limited to this.

[0032] The service conversation text sample includes multiple rounds of conversation samples between the customer and the customer service. It should be noted that in the embodiments of the present application, unless otherwise specified, "multiple rounds", "multiple", "multiple" and other numbers related to "multiple" all mean two or more.

[0033] For example, customer: "Hello, I received the clothes, but there is a problem"; customer service: "Hello, what is the problem?"; customer: "I ordered size M, but there is no size on this one, and it is very loose to wear"; customer service: "Dear, this one is one size ~"; customer: "Why is it not stated?!"; customer service: "It is stated on the details page of our products ~"; customer: "Who would look at the details page, when I ordered it, there were clearly S, M and L options"; customer service: "Yes, but this one is one size ~"; customer: "You gave me a number to choose from, but said it is one size, which is misleading consumers, I want to complain". It can be seen that in this example, the service conversation text sample includes 9 rounds of conversations between the customer and the customer service, of which 5 rounds are customer conversations and 4 rounds are customer conversations. Usually, a coherent sentence expression by the customer or the customer service can be considered as a round of conversation.

[0034] Step S204: Generate training samples according to the multi-round customer conversation samples corresponding to the customers extracted from the multi-round customer conversation samples and the service quality labels corresponding to the multi-round customer conversation samples.

[0035] Because the services provided by customer service are mainly for customers, and customer conversations can effectively feedback the service status of customer service, therefore, in an embodiment of the present application, the training samples for training the service quality assessment model are generated based on multiple rounds of customer conversation samples corresponding to the customer side in the conversation between the customer and the customer service and the service quality labels corresponding to these samples.

[0036] The service quality label can be set appropriately by those skilled in the art according to actual needs, and can be set to negative, positive and neutral, for example. Of course, a finer granularity division can also be performed, such as negative can be further divided into anger, surprise, etc., and positive can be further divided into happiness, gratitude, etc.

[0037] In order to improve the efficiency of model training, before inputting the training samples into the service quality evaluation model, the multi-round customer conversation samples can be preprocessed, such as data normalization (such as normalization of name, address and entity), removal of stop words and continuous conversation merging and other data preprocessing processes. The preprocessed multi-round customer conversation samples are composed of multi-round conversations on the customer side, which are arranged in chronological order. The associated service quality labels can be represented by numbers, such as 1 to 5, from 1 to 5 representing very dissatisfied, dissatisfied, average, satisfied and very satisfied respectively. Of course, it can also be in other forms, such as from 1 to 5 representing very bad, bad, average, good and very good, etc.

[0038] After the above processing, the final training samples can be generated and input into the service quality evaluation model.

[0039] Step S206: using the service quality evaluation model to be trained, respectively perform emotion recognition on multiple rounds of customer conversation samples to obtain corresponding multiple emotion recognition results.

[0040] To facilitate the description of the training process of the service quality evaluation model, the structure of an exemplary service quality evaluation model is first described here. Figure 2B As shown. Figure 2B As can be seen in the figure, the service quality evaluation model includes: emotion recognition part, semantic extraction part, representation aggregation part and prediction output part.

[0041] Among them, the emotion recognition part can be implemented by any machine learning model or deep learning model; the semantic extraction part can be implemented by an encoder, and the encoder can be implemented by any neural network model, including but not limited to convolutional neural network model, recurrent neural network model, pre-trained language model, etc.; the representation aggregation part can include a splicing layer and an encoder, the splicing layer is used to splice the emotion representation and the semantic representation, and the encoder is used to extract features from the spliced ​​representation vector, and the encoder can also be implemented by any neural network model, including but not limited to convolutional neural network model, recurrent neural network model, pre-trained language model, etc.; the prediction output part can be implemented by a multi-layer perceptron MLP.

[0042] The functions of each part of the above service quality evaluation model are described below in conjunction with the training process.

[0043] Among them, in this step, the emotion recognition part of the service quality evaluation model can be used to perform emotion recognition on multiple rounds of customer dialogue samples to obtain corresponding multiple emotion recognition results.

[0044] Exemplarily, multiple rounds of customer conversation samples can be disassembled into single-round conversation samples, and the single-round conversation samples can be input into a trained emotion recognition model one by one. The emotion recognition model can be any neural network model with emotion recognition function, such as a machine learning model, a deep learning model, etc. Through the emotion recognition model, the probability value of each emotion category corresponding to the single-round conversation sample can be output, also known as the emotion category distribution. If the emotion categories include negative, neutral and positive, then an exemplary emotion category distribution corresponding to the single-round conversation sample can be {0.8, 0.1, 0.1}, that is, the probability of the emotion being negative is 0.8, the probability of being neutral is 0.1, and the probability of being positive is 0.1.

[0045] Step S208: Aggregate the multiple emotion representations corresponding to the multiple emotion recognition results and the semantic representations of the multiple rounds of customer conversation samples corresponding to the multiple emotion recognition results to obtain an aggregated representation.

[0046] After obtaining the emotion recognition results, it is necessary to convert them into representation vectors suitable for calculation with semantic representation. Therefore, it is also necessary to generate corresponding emotion representations based on the emotion recognition results. In addition, it is also necessary to obtain the semantic representation of the customer conversation sample.

[0047] Based on this, in a feasible way, the emotion recognition part of the service quality evaluation model can be continued to perform weighted vectorization on multiple emotion recognition results to obtain corresponding multiple emotion representations.

[0048] For example, based on the emotion recognition results, an N-dimensional vector can be initialized for each emotion category, where N can be any integer in the set [128,512]. The probability value of each emotion category in the emotion recognition result is multiplied by the corresponding N-dimensional vector to obtain a weighted vector representation. The weighted vector representations associated with all emotion recognition results are summed to obtain the emotion representation corresponding to a single round of customer conversation samples.

[0049] The semantic representation corresponding to the customer conversation sample can be realized through the semantic extraction part of the service quality evaluation model. That is, through the semantic extraction part, semantic features are extracted for multiple rounds of customer conversation samples respectively to obtain corresponding multiple semantic representations. It should be noted that the acquisition of semantic representation can be performed at any time before representation aggregation, for example, it can be performed immediately after the customer conversation sample is input into the service quality evaluation model, or before or after the emotion recognition model performs emotion recognition, or it can be performed in parallel with the emotion recognition model performing emotion recognition, etc.

[0050] In one example, a single-round customer conversation sample can be fed into an encoder, and the encoder can obtain the semantic representation corresponding to the single-round conversation sample, and the dimension of the semantic representation is given by the encoder hidden layer dimension. The encoder can be implemented by any neural network model, including but not limited to convolutional neural networks, recurrent neural networks, pre-trained language models, etc.

[0051] After obtaining the emotional representation and semantic representation corresponding to each round of customer conversation samples, the two can be aggregated to integrate the emotional information into the semantic information of the customer conversation samples, so as to understand the customer's semantic expression in combination with the customer's emotions, and obtain accurate results for the subsequent customer-to-customer service quality evaluation.

[0052] The process of obtaining the aggregated representation can be implemented through the representation aggregation part of the service quality assessment model. For example, through the representation aggregation part of the service quality assessment model, multiple emotion representations and corresponding multiple semantic representations corresponding to multiple emotion recognition results are aggregated, and based on the aggregation results, the aggregated representation corresponding to the entire multi-round customer conversation is obtained. Specifically, through the representation aggregation part, multiple emotion representations and corresponding multiple semantic representations corresponding to multiple emotion recognition results can be aggregated to obtain an aggregated vector; feature extraction is performed on the aggregated vector to obtain the aggregated representation corresponding to the entire multi-round customer conversation sample. In a feasible manner, the aggregation can be implemented in the form of splicing. Of course, other appropriate forms such as fusion forms are also suitable for the solutions of the embodiments of the present application. Taking splicing as an example, the emotion representation and the semantic representation can be spliced ​​through the splicing layer of the aggregate representation part to obtain a spliced ​​vector; then, the encoder of the representation aggregation part performs feature extraction on the spliced ​​vector to obtain the final aggregated representation.

[0053] Exemplarily, the emotion representation and semantic representation associated with a single round of customer conversation samples can be concatenated through the above concatenation layer to obtain the aggregate vector of the single round of customer conversation samples. Furthermore, the aggregate vectors corresponding to all single round customer conversation samples are fed into an encoder (which is different from the above encoder) according to the time sequence of the conversation samples, and the corresponding hidden layer vectors are extracted to obtain the aggregate representation of all rounds of customer conversation samples. The encoder can be implemented by any neural network model, including but not limited to convolutional neural networks, recurrent neural networks, pre-trained language models, etc.

[0054] Step S210: performing service quality evaluation prediction on the service conversation text based on the aggregate representation to obtain a prediction result.

[0055] After obtaining an aggregate representation corresponding to the entire multi-round customer conversation sample, a service quality assessment prediction can be performed based on the aggregate representation. For example, the service quality assessment prediction can be performed on the service conversation text sample based on the aggregate representation through the prediction output part of the service quality assessment model to obtain a prediction result.

[0056] Although the above process has been processed based on customer conversation samples, since the conversation samples are customer interaction samples for customer service, service quality evaluation and prediction for customer service can be performed based on the customer conversation samples.

[0057] For example, after obtaining the aggregate representation of all rounds of customer conversation samples, the aggregate representation can be sent to a multi-layer perceptron MLP for service quality evaluation prediction to obtain prediction results, such as satisfaction prediction scores.

[0058] Step S212: training the service quality assessment model according to the difference between the prediction result and the service quality label.

[0059] When using Figure 2B In the case of the service quality assessment model shown in , this step can train the service quality assessment model through the prediction output part of the model according to the difference between the prediction result and the service quality label.

[0060] For example, the prediction result and the service quality label can be used to calculate the cross entropy loss, minimize the loss, and then optimize the model to achieve model training. The cross entropy loss can use a conventional cross entropy loss function, which will not be described in detail here.

[0061] Because the customer's emotional representation is introduced, the service quality evaluation model can only use a limited number of labeled training samples such as survey satisfaction data samples during the training process, calculate the cross entropy loss between the predicted results such as the predicted score and the survey satisfaction labels corresponding to the survey satisfaction data samples, minimize the loss, and then optimize the model for training. Therefore, by introducing the prediction information of emotion recognition, the label data required in the training stage of the service quality evaluation model is greatly reduced, which can effectively solve the problem of inaccurate results obtained due to the low participation rate in customer service scenario satisfaction surveys.

[0062] The training of the service quality evaluation model is iterated until the training termination condition is met, such as the number of training times reaches a predetermined number, or the loss meets a predetermined loss value, etc.

[0063] Through this embodiment, emotion recognition is first performed on each round in the customer conversation sample, and the emotion classification result obtained by the recognition is mapped to a high-dimensional vector, and the vector is aggregated with the result after encoding the customer conversation sample, and then the service quality evaluation prediction is performed based on a small number of service quality labels, and the loss is calculated, and the model is trained based on the loss. Therefore, it is possible to achieve better service quality evaluation (such as satisfaction) modeling and prediction effects by using a small number of training samples, such as a small number of survey satisfaction data samples. In addition, based on the customer's multi-round customer conversation samples as a benchmark, the customer's emotional information is introduced, and combined with semantic representation, the trained service quality model can be able to perform service quality evaluation in customer service scenarios more comprehensively and objectively.

[0064] Embodiment 2

[0065] This embodiment focuses on explaining the solution provided in the embodiment of the present application from the perspective of using the trained service quality assessment model.

[0066] Reference Figure 3A , shows a step flow chart of a service quality evaluation method according to embodiment 2 of the present application.

[0067] The service quality evaluation method of this embodiment includes the following steps:

[0068] Step S302: Obtain the service conversation text for which service quality evaluation is to be performed.

[0069] The service conversation text to be evaluated for service quality can be a text involving a conversation between a customer and a customer service representative in any service industry, such as a service conversation text for an e-commerce service, a service conversation text for a medical service, a service conversation text for a communication operation service, etc., which is not limited in the embodiments of the present application. In addition, the service conversation text includes multiple rounds of conversations between a customer and a customer service representative.

[0070] Step S304: extracting multiple rounds of customer conversations corresponding to the customer from the multiple rounds of conversations, and performing emotion recognition on the multiple rounds of customer conversations respectively to obtain corresponding multiple emotion recognition results.

[0071] When the solution of this embodiment is implemented based on the service quality evaluation model obtained by training in Embodiment 1, the emotion recognition part of the model can be used to perform emotion recognition on multiple rounds of customer conversations to obtain corresponding multiple emotion recognition results.

[0072] Optionally, before inputting the multiple rounds of customer conversations into the service quality evaluation model, the multiple rounds of customer conversations may be preprocessed, such as data normalization (such as normalization of name, address and entity), removal of stop words and merging of continuous conversations. The preprocessed multiple rounds of customer conversations consist of multiple rounds of conversations on the customer side, and the multiple rounds of conversations are arranged in chronological order. The multiple rounds of customer conversations are input into the service quality evaluation model, and emotion recognition is performed on them through the emotion recognition part thereof to obtain corresponding multiple emotion recognition results.

[0073] Furthermore, multiple emotion recognition results may be weighted vectorized to obtain multiple corresponding emotion representations. For example, multiple emotion recognition results may be weighted vectorized through the emotion recognition part of the service quality evaluation model to obtain multiple corresponding emotion representations.

[0074] Exemplarily, multiple rounds of customer conversations can be broken down into single rounds of conversations, and the single rounds of conversations can be input into the emotion recognition part of the service quality evaluation model one by one. Through the emotion recognition part, the probability value (also called emotion category distribution) of each emotion category corresponding to the single round of conversation is output, that is, the emotion recognition result. Based on the emotion recognition result, an N-dimensional vector is initialized for each emotion category, and N can be any integer in the set [128,512]. The probability value of each emotion category in the emotion recognition result is multiplied by the corresponding N-dimensional vector to obtain a weighted vector representation. The weighted vector representations associated with all emotion recognition results are added together to obtain the emotion representation corresponding to the single round of customer conversation.

[0075] Step S306: Aggregate the multiple emotion representations corresponding to the multiple emotion recognition results and the semantic representations of the multiple rounds of customer conversations corresponding to the multiple emotion recognition results to obtain an aggregated representation.

[0076] Because representation aggregation requires both emotional representation and semantic representation, the operation of obtaining the semantic representation of multiple rounds of customer conversations can be performed at any time after obtaining multiple rounds of customer conversations and before executing this step. In one feasible manner, semantic features can be extracted for each of the multiple rounds of customer conversations to obtain multiple corresponding semantic representations. When the solution of this embodiment is implemented based on the service quality evaluation model obtained by training in Example 1, semantic features can be extracted for each of the multiple rounds of customer conversations by the semantic extraction part of the model to obtain multiple corresponding semantic representations. For example, a single round of customer conversation can be fed into the encoder of the semantic extraction part, and the semantic representation corresponding to the single round of customer conversation can be obtained by the encoder, and the dimension of the semantic representation is given by the hidden dimension of the encoder.

[0077] After obtaining the emotional representation and semantic representation corresponding to each round of customer conversations, the two can be aggregated. In one feasible way, multiple emotional representations corresponding to multiple emotion recognition results and semantic representations of multiple rounds of customer conversations corresponding to multiple emotion recognition results can be aggregated to obtain an aggregate vector; feature extraction is performed on the aggregate vector to obtain an aggregate representation corresponding to the entire multiple rounds of customer conversations. In specific implementation, the emotional representation and semantic representation corresponding to each round of customer conversations are aggregated to obtain each aggregate vector corresponding to each round of customer conversations. Then, each aggregate vector is spliced ​​or superimposed together to form an aggregate vector corresponding to all rounds of customer conversations. Feature extraction is then performed on the aggregate vector to obtain an aggregate representation corresponding to all rounds of customer conversations.

[0078] For example, the emotion representation and semantic representation associated with a single round of customer conversation can be concatenated through the concatenation layer in the representation aggregation part of the service quality evaluation model to obtain the aggregation vector of the single round of customer conversation. Furthermore, the aggregation vectors corresponding to all single rounds of customer conversations are fed into the encoder according to the time sequence of the conversation samples, and the corresponding hidden layer vectors are extracted to obtain the aggregation representation of all rounds of customer conversations.

[0079] Step S308: Evaluate the service quality of the service conversation text based on the aggregated representation.

[0080] For example, the prediction output part of the service quality evaluation model can be used to perform service quality evaluation prediction based on the above-mentioned aggregate representation, and the service quality evaluation result of the service conversation text can be obtained according to the prediction result.

[0081] For example, after obtaining the aggregate representation of all rounds of customer conversations, the aggregate representation can be sent to the multi-layer perceptron MLP in the prediction output part to perform service quality evaluation prediction to obtain a prediction result, such as a satisfaction prediction score, etc. The prediction result can indicate the service quality corresponding to the service conversation text.

[0082] The following is an example of an e-commerce customer service scenario to illustrate the above process. Figure 3B shown.

[0083] exist Figure 3B In the e-commerce customer service scenario shown, it is assumed that the service dialogue text to be evaluated for service quality includes 9 rounds of dialogue, namely: Customer: "Hello, I received the clothes, but there is a problem"; Customer service: "Hello, what is the problem?"; Customer: "I ordered a size M, but there is no size indication on this one, and it is very loose when I wear it"; Customer service: "Dear, this one is one size~"; Customer: "Then why is it not stated?!"; Customer service: "It is stated on the details page of our product~"; Customer: "Who would look at the details page? When I ordered, there were clearly S, M and L options"; Customer service: "Yes, but this one is one size~"; Customer: "You gave the number for selection and said it was one size. This is misleading consumers. I want to complain."

[0084] Multiple rounds of customer dialogues corresponding to the customer are extracted from it, namely: "Hello, I received the clothes, but there are some problems" (for ease of description, referred to as customer dialogue 1 in the following text); "I ordered a size M, but the size is not written on it, and it is very loose to wear" (hereinafter referred to as customer dialogue 2); "Why is it not written?!" (hereinafter referred to as customer dialogue 3); "Who would go to the details page? When I ordered, there were clearly S, M and L options" (hereinafter referred to as customer dialogue 4); "You gave me a number to choose from, but said it was a one-size-fits-all size. This is misleading consumers. I want to complain" (hereinafter referred to as customer dialogue 5). The above 5 rounds of customer dialogues will be input into the service quality evaluation model.

[0085] First, the emotion recognition part of the service quality assessment model will perform emotion recognition on these five rounds of customer conversations to obtain emotion recognition results; and, it will be further vectorized into emotion representations, which are simply illustrated in this example as: emotion representation 1 corresponding to conversation with customer 1, emotion representation 2 corresponding to conversation with customer 2, emotion representation 3 corresponding to conversation with customer 3, emotion representation 4 corresponding to conversation with customer 4, and emotion representation 5 corresponding to conversation with customer 5.

[0086] In this example, while the emotion recognition part performs emotion recognition on the customer conversation, the semantic extraction part of the service quality assessment model performs semantic extraction on the five customer conversations respectively to obtain the corresponding five semantic representations. A simple illustration is: semantic representation 1 corresponding to conversation with customer 1, semantic representation 2 corresponding to conversation with customer 2, semantic representation 3 corresponding to conversation with customer 3, semantic representation 4 corresponding to conversation with customer 4, and semantic representation 5 corresponding to conversation with customer 5.

[0087] Next, the representation aggregation part of the service quality evaluation model aggregates the five representation vectors to obtain five aggregate vectors, namely: aggregate vector 1 after the aggregation of emotion representation 1 and semantic representation 1, aggregate vector 2 after the aggregation of emotion representation 2 and semantic representation 2, aggregate vector 3 after the aggregation of emotion representation 3 and semantic representation 3, aggregate vector 4 after the aggregation of emotion representation 4 and semantic representation 4, and aggregate vector 5 after the aggregation of emotion representation 5 and semantic representation 5. Furthermore, the representation aggregation part will aggregate these five aggregate vectors again according to the time sequence of multiple rounds of customer conversations to form aggregate vector 1+aggregate vector 2+aggregate vector 3+aggregate vector 4+aggregate vector 5, and generate the final aggregate vector, which is simply represented as aggregate vector C.

[0088] The aggregate vector C will be input into the prediction output part of the service quality evaluation model, such as the MLP of this part. The service quality evaluation prediction is performed through the MLP to obtain the prediction result and output it. In this example, the prediction result is that the service satisfaction is 3 points.

[0089] Thus, the quality evaluation of the service dialogue text in the above e-commerce customer service scenario is achieved, and this quality evaluation can also be regarded as an evaluation of the service quality or satisfaction of this customer service.

[0090] It can be seen that through this embodiment, on the one hand, when conducting service quality evaluation, it is no longer based on manual evaluation or single-sentence service dialogue, but on multiple rounds of dialogue with customers. Through multiple rounds of dialogue, the customer's service situation in this service can be more comprehensively and accurately understood, and thus a more accurate service quality evaluation result can be obtained; on the other hand, when conducting service quality evaluation, the customer's emotional change information (through multiple emotional expressions) and semantic information (through semantic expressions) are combined. That is, it is no longer dependent on semantic information alone to determine the service quality of customer service, but through the combination of emotional information and semantic information of multiple rounds of dialogue, the customer's emotional changes in the whole process can be understood, so as to evaluate the service quality more comprehensively and objectively.

[0091] Embodiment 3

[0092] Reference Figure 4 , shows a schematic diagram of the structure of an electronic device according to the third embodiment of the present application. The specific embodiment of the present application does not limit the specific implementation of the electronic device.

[0093] like Figure 4 As shown, the electronic device may include: a processor (processor) 402 , a communication interface (Communications Interface) 404 , a memory (memory) 406 , and a communication bus 408 .

[0094] in:

[0095] The processor 402 , the communication interface 404 , and the memory 406 communicate with each other via a communication bus 408 .

[0096] The communication interface 404 is used to communicate with other electronic devices or servers.

[0097] The processor 402 is used to execute the program 410, and specifically can execute the relevant steps in any of the above method embodiments.

[0098] Specifically, the program 410 may include program codes, which include computer operation instructions.

[0099] The processor 402 may be a CPU, or an application specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0100] The memory 406 is used to store the program 410. The memory 406 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0101] The program 410 may be specifically used to enable the processor 402 to execute operations corresponding to the method described in any of the foregoing method embodiments.

[0102] The specific implementation of each step in program 410 can refer to the corresponding description of the corresponding steps and units in the above method embodiment, and has corresponding beneficial effects, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the above method embodiment, which will not be repeated here.

[0103] An embodiment of the present application also provides a computer program product, including computer instructions, which instruct a computing device to execute operations corresponding to any one of the above-mentioned multiple method embodiments.

[0104] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0105] The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or implemented as software or computer code that can be stored in a recording medium (such as a CDROM, RAM, floppy disk, hard disk or magneto-optical disk), or implemented as a computer code originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded through a network and to be stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a special-purpose processor or programmable or special-purpose hardware (such as an ASIC or FPGA). It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown here, the execution of the code converts the general-purpose computer into a special-purpose computer for executing the method shown here.

[0106] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present application.

[0107] The above implementation methods are only used to illustrate the embodiments of the present application, and are not limitations on the embodiments of the present application. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present application. The scope of patent protection of the embodiments of the present application should be limited by the claims.

Claims

1. A service quality evaluation method, comprising: Obtaining a service conversation text for which service quality evaluation is to be performed, wherein the service conversation text includes multiple rounds of conversations between the customer and the customer service; Extracting multiple rounds of customer conversations corresponding to the customer from the multiple rounds of conversations, and performing emotion recognition on the multiple rounds of customer conversations respectively to obtain corresponding multiple emotion recognition results; Aggregating the multiple emotion representations corresponding to the multiple emotion recognition results and the semantic representations of multiple rounds of customer conversations corresponding to the multiple emotion recognition results to obtain an aggregated representation; Performing a service quality evaluation on the service conversation text based on the aggregate representation; Among them, the method also includes: disassembling the multiple rounds of customer conversations to obtain corresponding single-round conversations; determining the probability value of the emotion category corresponding to the single-round conversation, wherein the emotion recognition result includes the probability value; based on the probability value and the initialization vector of the emotion category, determining the weighted vector of the emotion recognition result; adding the weighted vectors to obtain the emotion representation corresponding to the emotion recognition result.

2. The method according to claim 1, wherein: Before aggregating the multiple emotion representations corresponding to the multiple emotion recognition results and the semantic representations of the multiple rounds of customer conversations corresponding to the multiple emotion recognition results, the method further includes: Semantic features are extracted from the multiple rounds of customer conversations to obtain corresponding multiple semantic representations.

3. The method according to claim 1, wherein: After obtaining the corresponding plurality of emotion recognition results, the method further includes: The multiple emotion recognition results are weighted vectorized to obtain corresponding multiple emotion representations.

4. The method according to any one of claims 1 to 3, wherein: The step of aggregating the multiple emotion representations corresponding to the multiple emotion recognition results and the semantic representations of multiple rounds of customer conversations corresponding to the multiple emotion recognition results to obtain an aggregated representation includes: Aggregating the multiple emotion representations corresponding to the multiple emotion recognition results and the semantic representations of multiple rounds of customer conversations corresponding to the multiple emotion recognition results to obtain an aggregated vector; Feature extraction is performed on the aggregated vector to obtain an aggregated representation corresponding to the entirety of multiple rounds of customer conversations.

5. The method according to any one of claims 1 to 3, wherein: The performing service quality evaluation on the service conversation text based on the aggregate representation includes: A service quality evaluation prediction is performed based on the aggregate representation, and a service quality evaluation result of the service conversation text is obtained according to the prediction result.

6. A service quality assessment model training method, comprising: Acquire a service conversation text sample, wherein the service conversation text sample includes a multi-round conversation sample between a customer and a customer service representative; Generate training samples according to the multiple rounds of customer conversation samples corresponding to the customer extracted from the multiple rounds of conversation samples and the service quality labels corresponding to the multiple rounds of customer conversation samples; Using the service quality evaluation model to be trained, respectively performing emotion recognition on the multiple rounds of customer conversation samples to obtain corresponding multiple emotion recognition results; Aggregating the multiple emotion representations corresponding to the multiple emotion recognition results and the semantic representations of the multiple rounds of customer conversation samples corresponding to the multiple emotion recognition results to obtain an aggregated representation; Performing a service quality evaluation prediction on the service conversation text sample based on the aggregate representation to obtain a prediction result; Training the service quality assessment model according to the difference between the prediction result and the service quality label; Among them, the method also includes: disassembling the multiple rounds of customer conversations to obtain corresponding single-round conversations; determining the probability value of the emotion category corresponding to the single-round conversation, wherein the emotion recognition result includes the probability value; based on the probability value and the initialization vector of the emotion category, determining the weighted vector of the emotion recognition result; adding the weighted vectors to obtain the emotion representation corresponding to the emotion recognition result.

7. The method according to claim 6, wherein: The service quality assessment model includes: an emotion recognition part, a semantic extraction part, a representation aggregation part and a prediction output part; in: Through the emotion recognition part, emotion recognition is performed on the multiple rounds of customer conversation samples respectively to obtain corresponding multiple emotion recognition results; By means of the semantic extraction part, semantic features are extracted from the multiple rounds of customer conversation samples respectively to obtain corresponding multiple semantic representations; Aggregating the multiple emotion representations and the corresponding multiple semantic representations corresponding to the multiple emotion recognition results through the representation aggregation part, and obtaining an aggregated representation corresponding to the entire multiple rounds of customer conversations based on the aggregation results; Through the prediction output part, the service quality evaluation prediction is performed on the service conversation text sample based on the aggregate representation to obtain a prediction result; and according to the difference between the prediction result and the service quality label, the service quality evaluation model is trained.

8. The method according to claim 7, wherein: The method further comprises: The emotion recognition part performs weighted vectorization on the multiple emotion recognition results to obtain corresponding multiple emotion representations.

9. The method according to claim 7 or 8, wherein: The step of aggregating the multiple emotion representations and the corresponding multiple semantic representations corresponding to the multiple emotion recognition results through the representation aggregation part, and obtaining the aggregated representation corresponding to the entirety of the multiple rounds of customer conversation samples based on the aggregation results, includes: Aggregating the plurality of emotion representations and the corresponding plurality of semantic representations corresponding to the plurality of emotion recognition results through the representation aggregation part to obtain an aggregated vector; Feature extraction is performed on the aggregated vector to obtain an aggregated representation corresponding to the multi-round customer conversation samples as a whole.

10. An electronic device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method according to any one of claims 1 to 9.

11. A computer storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

12. A computer program product, comprising computer instructions, wherein the computer instructions instruct a computing device to execute operations corresponding to the method according to any one of claims 1 to 9.

Images