Telephone response quality evaluation device

Abstract

To provide a telephone response quality evaluation device that can objectively evaluate the quality of telephone responses by operators. [Solution] The system includes an input unit 11 that inputs conversation data between a caller 3 and an operator 2 to be evaluated, an analysis unit 12 that analyzes the conversation data input to the input unit using an emotion analysis engine, and an evaluation unit 13 that evaluates the operator's response quality based on the analysis results of the conversation data analyzed by the analysis unit. The emotion analysis engine inputs the conversation data into an emotion learning model 121 to determine the detailed emotions of the caller and the operator, and the evaluation unit 13 evaluates the operator's response quality based on the determined detailed emotions.

Description

Technical Field

[0001] The present invention relates to a quality evaluation device for evaluating the quality of telephone responses by operators in a call center.

Background Art

[0002] In a call center, there is known a call center system that converts communication content from a customer stored in a telephone reception device into text data, stores this text data, and extracts communications containing a preset keyword from the stored text data (Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in conventional call centers, the evaluation of the response quality of operators who respond by phone has solely relied on the experience and intuition of skilled operators. Therefore, the development of a method for objectively evaluating the response quality of operators has been desired.

[0005] The problem to be solved by the present invention is to provide a quality evaluation device for telephone responses that can objectively evaluate the quality of an operator's telephone response.

Means for Solving the Problems

[0006] The present invention includes an input unit that inputs a conversation between a call partner and an operator to be evaluated, an analysis unit that analyzes the conversation input to the input unit by an emotion analysis engine, The system includes an evaluation unit that evaluates the operator's response quality based on the analysis results of the conversation analyzed by the analysis unit, The emotion analysis engine inputs the conversation into an emotion learning model and determines the detailed emotions of the caller and the operator. The evaluation unit solves the above problem with a telephone response quality evaluation device that evaluates the operator's response quality based on the determined detailed emotions.

[0007] In the present invention, it is more preferable to further include a text conversion unit that converts the content of the conversation into text data and outputs the converted text data of the conversation.

[0008] In the present invention, the items for evaluating the quality of the operator's response in inbound operations, including the operation of receiving inquiries from callers, are: Basic manners, including conversation opening, lack of verbal tics, and word choice, Listening skills include the ability to actively listen to the content of the conversation with the person on the other end of the line, understanding, repeating, and rephrasing questions or requests from the person on the other end of the line, and the ability to interrupt or pause in the conversation. Speaking ability, including speaking speed, speaking style that matches the emotions of the person on the other end of the call, and speaking style that adapts to the emotional patterns of the person on the other end of the call, Customer satisfaction, including the overall emotional satisfaction of the person on the other end of the call, It can include...

[0009] In the present invention, the items for evaluating the quality of the operator's response in outbound operations, including inside sales, are: Basic manners, including how to introduce yourself and how to start a conversation with positive emotions, Communication skills, including responding to questions or opinions from the caller, and the degree of emotional alignment between the caller and the operator, The presentation of necessary keywords, the closing statement including an introduction or words of thanks, Customer satisfaction, including the overall emotional satisfaction of the person on the other end of the call, It can include...

[0010] In the present invention, the evaluation unit can display the results of the operator's response quality evaluation using a radar chart. [Effects of the Invention]

[0011] According to the present invention, the quality of an operator's telephone handling can be objectively evaluated. [Brief explanation of the drawing]

[0012] [Figure 1] This is a block diagram showing a telephone response quality evaluation device according to one embodiment of the present invention. [Figure 2] This figure shows an example of detailed emotions in one embodiment of the present invention. [Figure 3] This figure shows an example of the configuration of an emotion learning model in one embodiment of the present invention. [Figure 4] This figure shows the evaluation items and evaluation criteria for inbound operations in one embodiment of the present invention. [Figure 5] This figure shows the evaluation items and evaluation criteria for outbound operations in one embodiment of the present invention. [Figure 6] This figure shows an example of the output of evaluation results for inbound operations in one embodiment of the present invention. [Figure 7] This figure shows an example of the output of the evaluation results for outbound operations in one embodiment of the present invention. [Modes for carrying out the invention]

[0013] Hereinafter, embodiments for implementing the present invention will be described while referring to the drawings. FIG. 1 is a block diagram showing a telephone response quality evaluation apparatus 1 according to an embodiment of the present invention. The telephone response quality evaluation apparatus 1 of the present embodiment inputs conversation data between an operator 2 and a call partner 3, analyzes the emotions of the operator 2 and the call partner 3 included in the conversation data, and objectively evaluates the response quality of the operator based on the analyzed emotions. Therefore, the telephone response quality evaluation apparatus 1 of the present embodiment includes an input unit 11, an analysis unit 12, an evaluation unit 13, and a text conversion unit 14. The telephone response quality evaluation apparatus 1 of the present embodiment is configured by a computer having a CPU or MPU, ROM, RAM, HDD or SDD, etc., and the functions of the analysis unit 12, the evaluation unit 13 and the text conversion unit 14 are realized by software installed in the ROM.

[0014] The input unit 11 receives, for example, data of a telephone conversation in which the operator 2 to be evaluated participates. The conversation data to be evaluated is voice data between the operator 2 and the call partner 3. As the operator 2 to be evaluated, in addition to those who perform the work of a telephone operator in a call center or the like, those who perform inbound or outbound operations as CSRs (Customer Service Representatives, those who serve customers on behalf of a company) are also included. The inbound operation includes response operations such as receiving inquiries from external customers. On the other hand, the outbound operation includes response operations mainly for an unspecified number of customers, such as inside sales (also called in-house sales or remote sales). The call partner 3 includes customers who have purchased products of a specific company or received services, as well as general consumers who may become future customers.

[0015] The analysis unit 12 analyzes the conversation data input to the input unit 11 using an emotion analysis engine. Specifically, the emotion analysis engine inputs the conversation data input to the input unit 11 into an emotion learning model 121 to determine the detailed emotions of the operator 2 and the call partner 3. As basic human emotions, there are emotions such as confusion, calm, surprise, fear, boredom, sadness, anger, and happiness. However, the detailed emotions in this embodiment are emotions classified more finely than the basic emotions. The detailed emotions in this embodiment are emotions corresponding to each level set for each of a plurality of emotion indicators. The analysis unit 12 determines the detailed emotions of the operator 2 and the call partner 3 using the levels of each emotion indicator output by the emotion learning model 121. Specifically, the levels include at least one type of positive level, a normal level, and at least one type of negative level. The analysis unit 12 determines the detailed emotions of the emotion indicators with levels other than the normal level as the detailed emotions of the operator 2 and the call partner 3.

[0016] Figure 2 is a diagram showing an example of detailed emotions. The illustrated detailed emotions indicate emotions corresponding to the number of levels excluding the normal level for each of a plurality of emotion indicators. An emotion indicator is an indicator for showing detailed emotions. In the illustrated example, 10 emotion indicators are shown, but the number of emotion indicators may be other than 10.

[0017] Each emotion indicator has a plurality of levels, and in the illustrated example, 5 levels are shown. That is, it has two types of positive levels, a normal level, and two types of negative levels. In Figure 2, the level with a large degree of positivity is designated as "positive ++", and the level with a small degree of positivity is designated as "positive +". Also, the level with a large degree of negativity is designated as "negative --", and the level with a small degree of positivity is designated as "negative -". Also, in each emotion indicator, when it is neither positive nor negative, the detailed emotion is normal.

[0018] For example, when the topmost emotion index in FIG. 2 is the "positive / negative" index, the detailed emotion corresponding to positive++ is "worship", the detailed emotion corresponding to positive+ is "liking", the detailed emotion corresponding to negative- is "suspicious", and the detailed emotion corresponding to negative-- is "contempt". In this example, with 10 emotion indexes and 4 levels set excluding the normal level, there are 40 (10×4) detailed emotions.

[0019] The emotion learning model 121 numerically outputs each level of positive and negative, as well as the normal level. For example, the emotion learning model 121 designates positive++ as n1, positive+ as n2, normal as n3, negative- as n4, and negative-- as n5. n1 to n5 are, for example, positive integers, and n1>n2>n3>n4>n5.

[0020] In FIG. 2, for the names of the emotion indexes, one Chinese character indicating a meaning close to each detailed positive and negative emotion, such as "positive" or "negative", is used for convenience. However, any name (for example, emotion index A, emotion index B, emotion index C, ···, etc.) can be used for the names of the emotion indexes. Also, the detailed emotions shown in the figure are merely examples, and by adjusting the number of emotion indexes and levels, the level of subdivision of the detailed emotions to be determined can be easily changed according to the purpose of use of the telephone call response quality evaluation device 1.

[0021] FIG. 3 shows a configuration example of the emotion learning model 121. The emotion learning model 121 includes an input layer, at least one intermediate layer, and an output layer. The input data D1 of the emotion learning model 121 shown in the figure is the conversation data between the operator 2 and the call partner 3 input to the input unit 11. The output data D2 is the level for each emotion index. The emotion learning model 121 numerically outputs the levels of each detailed emotion as described above.

[0022] The analysis unit 12 uses the output data D2 from the emotion learning model 121 to determine the detailed emotions of operator 2 and caller 3. For example, the analysis unit 12 determines that the detailed emotions corresponding to the levels of emotion indicators for which a value other than n3, which indicates the normal level, is the detailed emotion of operator 2 and caller 3. If the emotion learning model 121 outputs a value other than n3 for multiple detailed emotions, the analysis unit 12 determines that all of these detailed emotions are the user's detailed emotions.

[0023] The emotion learning model 121 is generated by applying machine learning (e.g., deep learning) to training data. This training data can be created, for example, as follows: With the cooperation of skilled operators 2 who are proficient in inbound and outbound operations, multiple (a predetermined number) of conversational data corresponding to each detailed emotion are collected, and training data is generated that includes each collected conversational data and the detailed emotion (label). Each detailed emotion (label) is assigned a numerical level corresponding to the emotion index of the correct detailed emotion, and other emotion indexes are assigned a numerical normal level (n3). The emotion learning model 121 is then generated by applying machine learning to the training data created in this way. As a result, when the emotion learning model 121 receives input data D1 (conversational data between operator 2 and caller 3), it outputs the numerical levels for each emotion index as output data D2.

[0024] Returning to Figure 1, the evaluation unit 13 evaluates the quality of operator 2's response, which is the subject of evaluation, based on the analysis results of the conversation data analyzed by the analysis unit 12. Specifically, the detailed emotions of operator 2 and caller 3, which are output data D2 of the emotion learning model 121, are applied to evaluation criteria that have been pre-associated with evaluation items, and this is used as the quality evaluation value for operator 2's telephone response.

[0025] Figures 4 and 5 are diagrams showing evaluation criteria tables that link evaluation items and evaluation criteria used by the evaluation unit 13. Figure 4 is an evaluation criteria table for inbound operations, and Figure 5 is an evaluation criteria table for outbound operations. Note that the evaluation criteria and sub-items shown in Figures 4 and 5 are excerpts of actual sub-items, mainly sub-items of evaluation criteria related to detailed emotions. In Figures 4 and 5, CSR refers to Operator 2.

[0026] As shown in Figure 4, the evaluation of Operator 2's response quality in inbound operations, including receiving inquiries from callers, consists of four items: basic manners, listening skills, speaking skills, and customer satisfaction, each evaluated on a four-point scale. Basic manners include the appropriateness of the conversation opening, the absence of verbal tics, and word choice. Listening skills include the ability to listen attentively to the caller's conversation, understanding, repeating, and rephrasing the caller's questions or requests, and the ability to avoid interrupting or using pauses appropriately. Speaking skills include the speed of conversation, speaking in a way that matches the caller's emotions, and speaking in a way that matches the caller's emotional patterns. Customer satisfaction includes the overall emotional satisfaction of the caller.

[0027] In contrast, the evaluation criteria for operator response quality in outbound operations, including inside sales, consist of four items, as shown in Figure 5: opening, communication skills, closing, and customer satisfaction, each evaluated on a four-point scale. Here, opening includes the wording used when introducing oneself and positive emotions. Communication skills include conversational ability, proposal skills, and the expression of positive emotions towards the caller. Closing includes the presentation of necessary keywords and closing with an introduction or words of thanks. Customer satisfaction includes the overall emotional satisfaction of the caller.

[0028] Furthermore, the basic manners for inbound operations shown in Figure 4 are evaluated based on whether the emotion of operator 2 in the detailed emotion output data D2 is positive, such as "joy," "happiness," or "fun," after the opening of the conversation has been completed. If the degree of positivity is 80% or higher, the evaluation score is 4 points; if the degree of positivity is 60% or higher but less than 80%, the evaluation score is 3 points; if the degree of positivity is 40% or higher but less than 60%, the evaluation score is 2 points; if the degree of positivity is 20% or higher but less than 40%, the evaluation score is 1 point; and if scoring is not possible, the evaluation score is 0 points.

[0029] Similarly, the listening skills of inbound callers, as shown in Figure 4, are evaluated based on whether appropriate interjections are used. If the interjections listed in the interjection words are used, and the percentage of interjections used that match the customer's (caller's) emotions in the total number of interjections, as shown in the detailed emotion output data D2 for operator 2 and caller 3, is 20% or less, then the evaluation score is 4 points; if it is 40% or less, the evaluation score is 3 points; if it is 60% or less, the evaluation score is 2 points; if it is 80% or more, the evaluation score is 1 point; and if scoring is impossible, the evaluation score is 0 points.

[0030] Similarly, the speaking ability of inbound callers, as shown in Figure 4, is evaluated based on the degree to which the positive emotions of caller 3 and operator 2 match from the start time to the end time of the call. Regarding the emotions of operator 2 and caller 3 in the detailed emotion output data D2, a score of 4 is assigned if the positive emotions of operator 2 and caller 3 match by 80% or more; 3 points if the match is between 60% and 80%; 2 points if the match is between 40% and 60%; 1 point if the match is less than 40%; and 0 points if scoring is impossible.

[0031] Similarly, customer satisfaction in inbound operations, as shown in Figure 4, is evaluated based on the percentage of time that Operator 2 responds with the appropriate emotional pattern in accordance with the emotional pattern of Caller 3. For the emotions of Operator 2 and Caller 3 in the detailed emotional output data D2, a score of 8 is assigned if there is a 100% match, 7 points if there is a 90% or higher match, 6 points if there is an 80% or higher match but less than 90%, 5 points if there is a 70% or higher match but less than 80%, 4 points if there is a 60% or higher match but less than 70%, 3 points if there is a 50% or higher match but less than 60%, 2 points if there is a 40% or higher match but less than 50%, 1 point if there is a less than 40% match, and 0 points if scoring is impossible. Furthermore, the maximum score of 8 points is compressed by half to arrive at the final score.

[0032] In contrast, the opening of outbound operations shown in Figure 5 is evaluated based on whether the introduction (self-introduction) at the start of the conversation is done with positive emotions. Regarding the emotions of operator 2 in the detailed emotion output data D2, if the positive emotion is 80% or more, the evaluation score is 4 points; if the positivity is 60% or more but less than 80%, the evaluation score is 3 points; if the positivity is 40% or more but less than 60%, the evaluation score is 2 points; if the positivity is 20% or more but less than 40%, the evaluation score is 1 point; and if scoring is not possible, the evaluation score is 0 points.

[0033] Similarly, the communication skills of outbound calls shown in Figure 5 are evaluated based on the degree to which the positive emotions of caller 3 and operator 2 match from the start time to the end time of the call. For the emotions of operator 2 and caller 3 in the detailed emotion output data D2, a score of 4 is given if there is an 80% or greater match, a score of 3 if there is a 60% or greater match but less than 80%, a score of 2 if there is a 40% or greater match but less than 60%, a score of 1 if there is a less than 40% match, and a score of 0 if scoring is not possible.

[0034] Similarly, customer satisfaction for outbound operations, as shown in Figure 5, is evaluated based on the percentage of time that Operator 2 responds with the appropriate emotional pattern in accordance with the emotional pattern of Caller 3. For the emotions of Operator 2 and Caller 3 in the detailed emotional output data D2, a score of 8 is assigned if there is a 100% match, 7 points if there is a 90% or higher match, 6 points if there is an 80% or higher match but less than 90%, 5 points if there is a 70% or higher match but less than 80%, 4 points if there is a 60% or higher match but less than 70%, 3 points if there is a 50% or higher match but less than 60%, 2 points if there is a 40% or higher match but less than 50%, 1 point if there is a less than 40% match, and 0 points if scoring is impossible. Furthermore, the maximum score of 8 points is compressed by half to arrive at the final score.

[0035] Figure 6 shows an example of the output of evaluation results for inbound operations in one embodiment of the present invention, and Figure 7 shows an example of the output of evaluation results for outbound operations in one embodiment of the present invention. By inputting conversation data between operator 2 and caller 3 into the input unit 11, the display 15 shown in Figure 1 displays the evaluation results shown in Figure 6 or Figure 7. The evaluation results shown in Figures 6 and 7 display the evaluation score for each of the four evaluation items and the overall average score numerically at the top. To the right of this, a radar chart plotting the four evaluation items on a four-point scale is displayed. These displays allow for a quick overview of the evaluation results. In addition, the evaluation criteria and sub-item evaluation results for each of the four evaluation items are displayed below. This allows for detailed investigation of the reasons for the evaluation.

[0036] The text conversion unit 14 converts the conversation data (voice data) between operator 2 and caller 3 into text data. In other words, the text conversion unit 14 is speech recognition software, and outputs the converted conversation text data to the display 15 or records it in memory (not shown). This conversation text data is used as log data for inbound and outbound operations. [Explanation of symbols]

[0037] 1… Telephone response quality evaluation device 11...Input section 12…Analysis Department 121… Emotion Learning Model 13…Evaluation Department 14...Text conversion section 15…Display 2… Operator (CSR) 3…The person on the other end of the call (the customer)

Claims

1. An input unit for inputting conversation data between the caller and the operator to be evaluated, The analysis unit analyzes the conversation data input to the input unit using an emotion analysis engine. The system includes an evaluation unit that evaluates the operator's response quality based on the analysis results of the conversation data analyzed by the analysis unit, The emotion analysis engine inputs the conversation data into an emotion learning model and determines the detailed emotions of the caller and the operator. The evaluation unit is a telephone response quality evaluation device that evaluates the operator's response quality based on the determined detailed emotions.

2. The telephone response quality evaluation device according to claim 1, further comprising a text conversion unit that converts the content of the conversation data into text data and outputs the converted text data of the conversation data.

3. The items for evaluating the quality of operator responses in inbound operations, including receiving inquiries from callers, are: Basic manners, including conversation opening, lack of verbal tics, and word choice, Listening skills include the ability to actively listen to the other party's conversation, understanding, repeating, and rephrasing questions or requests from the other party, and the ability to interrupt or pause in the conversation. Speaking ability, including speaking speed, speaking style that matches the emotions of the person on the other end of the call, and speaking style that adapts to the emotional patterns of the person on the other end of the call, Customer satisfaction, including the overall emotional satisfaction of the person on the other end of the call, A telephone response quality evaluation device according to claim 1, including the following:

4. The items for evaluating the quality of operator responses in outbound operations, including inside sales, are: The way they introduce themselves, the opening of a conversation that includes positive emotions, Communication skills, including conversational ability, proposal skills, and the expression of positive emotions towards the person on the other end of the call, The presentation of necessary keywords, the closing statement including an introduction or words of thanks, Customer satisfaction, including the overall emotional satisfaction of the person on the other end of the call, A telephone response quality evaluation device according to claim 1, including the following:

5. The telephone response quality evaluation device according to any one of claims 1 to 4, wherein the evaluation unit displays the results of the operator's response quality evaluation using a radar chart.

Images