Emotional information utilization device, emotional information utilization method, and program

Abstract

An emotion information utilization device according to one embodiment has: a database having stored therein conversation information that includes emotion information representing the emotion of a conversation participant for each of at least prescribed sections; and a search unit that searches the database for the conversation information using search conditions including at least the section and the emotion information.

Description

[Technical Field]

[0001] The present invention relates to an emotion information utilization device, an emotion information utilization method, and a program. [Background technology]

[0002] Techniques for estimating a speaker's emotions from voice or text have been known for some time (for example, Patent Document 1), and are used for evaluating operators and providing support for responses in contact centers (also called call centers). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-113542 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the past, emotion estimation results (hereinafter also referred to as emotion information) could not be fully utilized.

[0005] For example, while it was previously possible to estimate a speaker's emotional information for the entire call or for each utterance, it was not possible to estimate the speaker's emotional information for a certain segment, which is important when supporting operators in handling calls or analyzing and improving the quality of customer service. As a result, emotional information could not be fully utilized for supporting operators in handling calls or analyzing and improving the quality of customer service.

[0006] Furthermore, for example, in the past, when one wanted to evaluate a certain call, it was difficult to interpret the quality of the call from emotional information, and therefore emotional information could not be fully utilized for call evaluation.

[0007] An embodiment of the present invention has been made in view of the above points, and aims to utilize emotion information. [Means for solving the problem]

[0008] In order to achieve the above object, an emotional information utilization device according to one embodiment has a database in which call information including emotional information that expresses the emotions of a speaker at least for each predetermined segment, and a search unit that searches the database for the call information using search criteria that include at least the segment and the emotional information. [Effects of the Invention]

[0009] Emotional information can be utilized. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram illustrating an example of the overall configuration of a contact center system according to an embodiment of the present invention. [Figure 2] FIG. 1 is a diagram illustrating an example of the functional configuration of an emotion information utilization device according to an embodiment of the present invention. [Figure 3] 10 is a flowchart illustrating an example of a call search process according to the present embodiment. [Figure 4] FIG. 10 is a diagram showing an example of a call search screen (part 1). [Figure 5] FIG. 10 is a diagram showing an example of a search result screen (part 1). [Figure 6] FIG. 10 is a diagram showing an example of a call search screen (part 2). [Figure 7] FIG. 10 is a diagram showing an example of a search result screen (part 2). [Figure 8] 10 is a flowchart illustrating an example of a response support process according to the present embodiment. [Figure 9] FIG. 10 is a diagram illustrating an example of a response support screen. [Figure 10] FIG. 10 is a diagram illustrating an example of an operator monitoring screen. [Figure 11] 10 is a flowchart illustrating an example of a call evaluation process according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] An embodiment of the present invention will be described below. In this embodiment, a contact center system 1 will be described that is targeted at a contact center and that can estimate the emotions of two parties (an operator and a customer) during a call and utilize the emotion information resulting from the estimation for supporting the operator in handling the call, analyzing and improving the quality of the call, evaluating the call, etc.

[0012] Note that targeting a contact center is just one example, and the present invention can be similarly applied to other places as well, such as offices, where the emotions of employees working there during a call are estimated and the resulting emotion information is used to support telephone responses and to analyze and improve the quality of telephone responses.

[0013] The following describes a contact center system 1 that achieves the following (1) to (3).

[0014] (1) The emotional information of the speaker is estimated for each segment of a certain conversation, and this emotional information is used to analyze and improve the quality of customer service.

[0015] (2) The emotional information of the speaker is estimated for each segment of a conversation, and this emotional information is used to support the operator during the conversation.

[0016] (3) A call is modeled based on the emotional information of past calls, and the emotional information of the call is used to interpret the evaluation of a call.

[0017] The above (1) makes it possible to, for example, perform more accurate analysis of response quality and more effectively improve response quality. The above (2) makes it possible to more effectively support the response of operators. The above (3) makes it possible to easily interpret the evaluation results when a certain call is evaluated.

[0018] <Overall configuration of contact center system 1> An example of the overall configuration of a contact center system 1 according to this embodiment is shown in Fig. 1. As shown in Fig. 1, the contact center system 1 according to this embodiment includes an emotion information utilization device 10, one or more operator terminals 20, one or more supervisor terminals 30, one or more analyst terminals 40, a PBX (Private Branch Exchange) 50, and a customer terminal 60. Here, the emotion information utilization device 10, the operator terminal 20, the supervisor terminal 30, the analyst terminal 40, and the PBX 50 are installed in a contact center environment E, which is the system environment of a contact center. Note that the contact center environment E is not limited to a system environment within the same building, and may be, for example, a system environment within multiple geographically separated buildings.

[0019] The emotion information utilization device 10 converts voice calls between customers and operators into text in real time using voice recognition, estimates the emotions of the customer and the operator, and utilizes the emotion information resulting from this estimation for supporting the agent's response, analyzing and improving response quality, and evaluating the call. The emotion information utilization device 10 also provides the operator terminal 20, supervisor terminal 30, or analyst terminal 40 with various screens (for example, a call search screen, search result screen, response support screen, operator monitoring screen, etc., which will be described later) for performing such response support and analysis and improvement of response quality.

[0020] The operator terminal 20 is a terminal of any type, such as a PC (personal computer), used by an operator, and functions as an IP (Internet Protocol) telephone. For example, a response support screen is displayed on the operator terminal 20 during a call with a customer.

[0021] The supervisor terminal 30 is a terminal of any type, such as a PC (personal computer), used by a supervisor. The supervisor terminal 30 can search past calls on a call search screen and display the search results on a search result screen. The supervisor terminal 30 can also display an operator monitoring screen that allows an operator to monitor a call in the background while the operator is talking to a customer. A supervisor is someone who monitors the operator's calls and supports the operator's telephone answering duties when a problem is likely to occur or at the operator's request. Usually, one supervisor monitors the calls of several to a dozen operators.

[0022] The analyst terminal 40 is a terminal of any type, such as a PC (personal computer), used by an analyst who analyzes and improves response quality. The analyst terminal 40 can search past calls on a call search screen and display the search results on a search result screen. Note that the analyst may also be a supervisor, in which case the supervisor terminal 30 will also function as the analyst terminal 40.

[0023] The PBX 50 is a telephone exchange (IP-PBX) and is connected to a communication network 70 including a VoIP (Voice over Internet Protocol) network and a PSTN (Public Switched Telephone Network). When a call is received from a customer terminal 60, the PBX 50 calls one or more predetermined operator terminals 20, and connects the customer terminal 60 to any of the operator terminals 20 that respond to the call.

[0024] The customer terminal 60 is a variety of terminals used by customers, such as a smartphone, a mobile phone, or a landline phone.

[0025] It should be noted that the overall configuration of the contact center system 1 shown in Fig. 1 is one example, and other configurations are also possible. For example, in the example shown in Fig. 1, the emotion information utilization device 10 is included in the contact center environment E (that is, the emotion information utilization device 10 is an on-premise type), but all or part of the functions of the emotion information utilization device 10 may be implemented by a cloud service or the like. Similarly, in the example shown in Fig. 1, the PBX 50 is an on-premise type telephone exchange, but it may also be implemented by a cloud service.

[0026] <Functional configuration of emotion information utilization device 10> The functional configuration of an emotion information utilization device 10 according to this embodiment is shown in Fig. 2. As shown in Fig. 2, the emotion information utilization device 10 according to this embodiment has a speech recognition-to-text conversion unit 101, an emotion estimation unit 102, a UI provision unit 103, a search unit 104, and an evaluation unit 105. Each of these units is realized, for example, by processing executed by a processor such as a CPU (Central Processing Unit) by one or more programs installed in the emotion information utilization device 10.

[0027] Furthermore, the emotion information utilization device 10 according to this embodiment has a call information DB 106. This DB (database) is realized by, for example, an auxiliary storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive). Note that this DB may also be realized by, for example, a database server connected to the emotion information utilization device 10 via a communications network.

[0028] The speech recognition and text conversion unit 101 converts the voice conversation between the operator terminal 20 and the customer terminal 60 into text by speech recognition. At this time, the speech recognition and text conversion unit 101 also performs speech recognition for each speaker and converts it into text. As a result, the operator's speech and the customer's speech are each converted into text. Hereinafter, the text obtained by speech recognition will also be referred to as "speech recognition text."

[0029] The emotion estimation unit 102 estimates the speaker's emotion information at predetermined intervals during a voice call between the operator terminal 20 and the customer terminal 60. The emotion information is information that represents the result of estimating the speaker's emotion, and examples of emotions include "satisfied," "dissatisfied," "anger," "normal," "anxiety," "doubt," "understanding," and "joy." However, this is just one example, and emotions may be broader categories such as "positive" and "negative," or any other arbitrary category. Furthermore, the definition of emotion categories may be added, changed, or deleted, for example, by a user.

[0030] However, in addition to estimating the speaker's emotional information for each segment, the emotion estimation unit 102 may also estimate the speaker's emotional information for each utterance or for the entire call. The emotion estimation unit 102 may estimate the speaker's emotional information using, for example, an emotion estimation model that utilizes known deep learning technology. In this case, the emotion estimation unit 102 may estimate the emotional information from the voice during the voice call between the operator terminal 20 and the customer terminal 60, or may estimate the emotional information from the speech-recognized text obtained by the speech recognition and text conversion unit 101. When estimating the emotional information for each segment, the emotion estimation unit 102 may construct an emotion estimation model that estimates emotional information for each segment and estimate the emotional information for each segment using this emotion estimation model, or may construct an emotion estimation model that estimates emotional information for each utterance and estimate the emotional information for each utterance using this emotion estimation model, and then estimate the emotional information for each segment by, for example, averaging the emotional information of the utterances included in each segment.

[0031] Here, the emotion estimation unit 102 estimates the emotion information of the speaker for each segment shown in any one of the following (A) and (B), for example.

[0032] (A) Time division For example, the average duration of a typical call is calculated and then divided into three parts: the "beginning," "middle," and "end." The speaker's emotional information is then estimated for each of the "beginning," "middle," and "end." Specifically, for example, if the average duration of a typical call is "3 minutes," the period from the start of the call (0:00) to 1:00 is the "beginning," 1:01 to 2:00 is the "middle," and 2:01 to the end of the call is the "end." Therefore, the customer's emotional information and the operator's emotional information from the start of the call to 1:00, the customer's emotional information and the operator's emotional information from 1:01 to 2:00, and the customer's emotional information and the operator's emotional information from 2:01 to the end of the call are estimated, respectively.

[0033] However, the division into "early stage," "middle stage," and "late stage" is merely an example, and the divisions may be divided into two parts, "first half" and "second half," or into even smaller divisions of four or more parts.

[0034] (B) Scene (topic) division A scene is a topic of conversation between an operator and a customer. Examples of scenes include "opening," which represents an initial greeting, "business confirmation," which represents a confirmation of the customer's business, "product explanation," which represents a product explanation, "situation confirmation," which represents a confirmation of the customer's situation, "identity confirmation," which represents a confirmation of the customer's identity, and "closing," which represents a final greeting. Note that scenes can be identified, for example, from voice recognition text using known technology.

[0035] Specifically, for example, if the scenes of a certain call are "opening," "confirming the purpose," "explanation of product," and "closing," the emotional information of the customer and the operator in "opening," the emotional information of the customer and the operator in "confirming the purpose," the emotional information of the customer and the operator in "explanation of product," and the emotional information of the customer and the operator in "closing" will each be estimated.

[0036] (C) A separator separated by a call event A call event is, for example, an event such as being put on hold, being transferred, or the occurrence of a predetermined length of silence. Specifically, for example, if a call is transferred once, the period from the start of the call to the transfer is considered one segment, and the period from the transfer to the end of the call is considered another segment, and the customer's emotional information and the operator's emotional information are estimated for each segment. As another specific example, if a call is transferred once and then put on hold once, the period from the start of the call to the transfer is considered one segment, and the period from the transfer to the hold is considered another segment, and the period from the hold to the end of the call is considered another segment, and the customer's emotional information and the operator's emotional information are estimated for each segment.

[0037] By estimating emotional information for each of the above segments (A), it becomes possible to grasp, for example, the temporal changes and flow of emotions between customers and agents. By estimating emotional information for each of the above segments (B), it becomes possible to grasp, for example, the emotions of customers and agents for each scene. By estimating emotional information for each of the above segments (C), it becomes possible to grasp, for example, the changes in the emotions of customers and agents before and after the occurrence of events, such as before and after transferring a call to another operator or supervisor, before and after putting a call on hold to search FAQs or consult with a supervisor, and before and after a silence occurs. These make it possible, for example, to perform more accurate analysis of response quality, more effective improvement of response quality, and more effective support for agents in responding to calls.

[0038] The UI providing unit 103 transmits display information for displaying various screens (for example, a call search screen, a search result screen, a response support screen, an operator monitoring screen, etc.) to the operator terminal 20, the supervisor terminal 30, or the analyst terminal 40.

[0039] When the search unit 104 receives a search request including search conditions specified on the call search screen, it uses the search conditions to search for call information from the call information DB 106. Furthermore, the search unit 104 converts the search result including the call information searched from the call information DB 106 into the sender of the search request.

[0040] The evaluation unit 105 creates an evaluation model from emotional information of calls that have been manually evaluated in advance, and then uses this evaluation model to evaluate the call to be evaluated.

[0041] The call information DB 106 stores call information of past calls. Here, the call information includes, for example, a call ID that uniquely identifies the call, the call date and time of the call, the call duration, an operator ID that uniquely identifies the operator who answered the call, the operator's name, the operator's extension number, the customer's telephone number, the voice recognition text of the call, each segment of the call and emotional information for each segment, and other information. In addition to these, for example, emotional information of the speaker for each utterance may be included, or emotional information of the speaker for the entire call may be included. Information indicating the call reason may also be included. The call reason is also called the subject of the call and refers to the reason why the customer called. A single call may have multiple call reasons, in which case information indicating each of the multiple call reasons is included in the call information.

[0042] Note that call information is created for each call between a customer and an operator and stored in the call information DB 106 .

[0043] <Call search> A case will be described where emotion information is used to search for call information on past calls in order to analyze and improve the quality of customer service. The call search process will be described below with reference to FIG. 3.

[0044] The search unit 104 receives a search request from the supervisor terminal 30 or the analyst terminal 40 (step S101). The search request is sent to the emotion information utilization device 10 when a search condition is specified on a call search screen displayed on the supervisor terminal 30 or the analyst terminal 40 and a search button is pressed.

[0045] Next, the search unit 104 searches for call information from the call information DB 106 using the search conditions included in the search request received in step S101 (step S102). As will be described later, examples of the search conditions include a segment and emotion information at that segment. This makes it possible to search for calls in which a certain emotion is estimated at a certain segment.

[0046] Then, the search unit 104 transmits the search results including the call information searched in the above step S102 to the supervisor terminal 30 or analyst terminal 40 that sent the search request (step S103). Note that the search unit 104 may transmit search results including some of the information included in the call information (for example, the call ID, the call duration, the operator ID, the operator name, each segment and the emotion information for each segment, etc.) instead of the call information itself.

[0047] <Example of call search screen and search result screen (part 1)> 4 and 5 show examples of a call search screen and a search result screen when emotion information for the entire call and emotion information for each segment of the above (A) are estimated. These call search screens and search result screens are displayed on the supervisor terminal 30 or the analyst terminal 40 based on the display information (display information for the call search screen, display information for the search result screen) created and transmitted by the UI providing unit 103.

[0048] The call search screen 1000 shown in FIG. 4 includes a division specification field 1001, an emotion specification field 1002, and a search button 1003. In the division specification field 1001, a time division (in the example shown in FIG. 4, "beginning," "middle," or "end") can be selected and specified as a search condition. In the emotion specification field 1002, a customer's emotion at the division specified in the division specification field 1001 (in the example shown in FIG. 4, "satisfied," "dissatisfied," "anger," "neutral," "anxious," "questioning," or "satisfied") can be selected and specified as a search condition. The search button 1003 is a button for transmitting a search request. The supervisor or analyst specifies the division and emotion in the division specification field 1001 and the emotion specification field 1002, respectively, and then presses the search button 1003. As a result, a search request including the divisions and emotions specified in the division specification field 1001 and emotion specification field 1002, respectively, as search conditions is transmitted from the supervisor terminal 30 or the analyst terminal 40 to the emotion information utilization device 10.

[0049] In the example shown in FIG. 4, the emotion specification field 1002 specifies the emotion of the customer. However, for example, a separate field for specifying a speaker (operator or customer) may be provided, and the emotion of the speaker may be specified in the emotion specification field 1002. Furthermore, the emotion specification field 1002 may allow specification of, for example, "positive" or "negative." Furthermore, multiple pairs of time divisions and emotions at those divisions may be specified (for example, search conditions such as ("beginning", "anger") and ("end", "satisfied") may be specified). Furthermore, search conditions such as changes in emotion at multiple time divisions (for example, changes in emotion between the "middle" and the "end"), or the continuation of the same emotion at multiple time divisions (for example, the continuation of the same emotion from the "beginning" to the "end") may be specified.

[0050] When search results for the above search request are received from the emotion information utilization device 10, the supervisor terminal 30 or the analyst terminal 40 displays, for example, a search result screen 1100 shown in Fig. 5. The search result screen 1100 shown in Fig. 5 includes search result fields 1110 and 1120 that display the contents of the call information included in the search results.

[0051] The search result fields 1110 and 1120 each display the call date and time, call duration, operator name, operator extension number, and customer phone number. The search result fields 1110 and 1120 also include emotion estimation result fields 1111 and 1121, respectively.

[0052] For example, in the emotion estimation result field 1111 of the search result field 1110, an icon representing the customer's emotion information for the entire call is displayed, along with icons representing the customer's emotion information for each of three sections ("beginning," "middle," and "end") in parentheses. In the example shown in Fig. 5, the customer's emotion information for the entire call is "anger," the customer's emotion information at the "beginning" is "satisfied," the customer's emotion information at the "middle" is "neutral," and the customer's emotion information at the "end" is "anger," indicating that the customer's emotion has changed from "satisfied" to "neutral" to "anger."

[0053] Similarly, for example, the emotion estimation result column 1121 in the search result column 1120 displays an icon representing the customer's emotion information for the entire call, as well as icons representing the customer's emotion information for each of three sections ("beginning," "middle," and "end") in parentheses. In the example shown in Fig. 5, the customer's emotion information for the entire call is "satisfied," the customer's emotion information at the "beginning" is "normal," the customer's emotion information at the "middle" is "satisfied," and the customer's emotion information at the "end" is "satisfied," indicating that the customer's emotion has changed from "normal" to "satisfied" to "satisfied."

[0054] When the search result field 1110 or the search result field 1120 is selected, more detailed content of the call information corresponding to the selected search result field is displayed.

[0055] In this way, supervisors and analysts can search for past calls that match a time period and the emotions (especially the customer's emotions) at that time period as search criteria. This makes it possible to extract calls in which the customer was angry at a specific time period (for example, towards the end of the call), which can be used to plan measures to improve such calls and to educate operators.

[0056] In addition, multiple sets of divisions and emotions, such as ("early stage", "anger") and ("late stage", "satisfied"), can be specified as search conditions, making it possible to extract, for example, calls in which the customer was angry at the beginning but satisfied at the end. This makes it possible to evaluate such calls as good calls or to use the information to educate operators, for example.

[0057] For example, in the search result field 1110, emotion information for the entire call or a segment prior to the latest segment may be displayed only if the customer's emotion information for the entire call or the latest segment meets certain specific conditions. Examples of specific conditions include the customer's emotion information at the latest segment being "negative" or a specific emotion, a change from emotion information other than negative to negative, a negative or specific emotion continuing for a certain number of segments, or a specific emotion at a segment representing a specific scene. In this case, the specific scene or specific emotion may be specified by a supervisor, analyst, or the like.

[0058] As a result, if certain conditions are not met, the call is deemed to be problem-free and the display of the transition in emotional information is omitted, reducing the amount of information and the burden on supervisors and analysts to check. In other words, since part of the display is omitted for calls that do not require attention, supervisors and analysts can more easily check calls that require attention.

[0059] The above-mentioned specific conditions are conditions for omitting part of the display for calls that are not problematic. However, conversely, if it is desired to omit part of the display for calls other than good calls, the specific conditions may be, for example, a change from non-positive emotional information to positive, or a positive or specific emotion continuing for a certain number of intervals.

[0060] <Example of call search screen and search result screen (part 2)> 6 and 7 show examples of a call search screen and a search result screen when emotion information is estimated for each of the divisions in (B) above. These call search screens and search result screens are displayed on the supervisor terminal 30 or the analyst terminal 40 based on the display information (display information for the call search screen, display information for the search result screen) created and transmitted by the UI providing unit 103.

[0061] The call search screen 2000 shown in FIG. 6 includes a division specification field 2001, an emotion specification field 2002, and a search button 2003. In the division specification field 2001, a scene (in the example shown in FIG. 6, "opening," "confirmation of business," "product description," "status hearing," "identity verification," and "closing") can be selected and specified as a search condition. In the emotion specification field 2002, a customer's emotion (in the example shown in FIG. 6, "satisfied," "dissatisfied," "anger," "neutral," "anxiety," "doubt," and "agreement") at the scene (division) specified in the division specification field 2001 can be selected and specified as a search condition. The search button 2003 is a button for transmitting a search request. The supervisor or analyst specifies a scene (division) and an emotion in the division specification field 2001 and the emotion specification field 2002, respectively, and then presses the search button 2003. As a result, a search request including as search conditions the scenes (departments) and emotions specified in the departement specification field 2001 and emotion specification field 2002, respectively, is transmitted from the supervisor terminal 30 or the analyst terminal 40 to the emotion information utilization device 10.

[0062] 6, the emotion of the customer is specified in the emotion specification field 2002, but for example, a separate field for specifying a speaker (operator or customer) may be provided, and the emotion of the speaker may be specified in the emotion specification field 2002. Also, it may be possible to specify multiple pairs of scenes (divisions) and emotions at those divisions (for example, it may be possible to specify search conditions such as ("opening", "anger") and ("closing", "satisfaction")).

[0063] When search results for the above search request are received from the emotion information utilization device 10, the supervisor terminal 30 or the analyst terminal 40 displays, for example, a search result screen 2100 shown in Fig. 7. The search result screen 2100 shown in Fig. 7 includes search result fields 2110 and 2120 that display the contents of the call information included in the search results.

[0064] The search result fields 2110 and 2120 each display the call date and time, call duration, operator name, operator extension number, and customer phone number. The search result fields 2110 and 2120 also include emotion estimation result fields 2111 and 2121, respectively.

[0065] For example, icons representing the emotional information of the customer in each scene of the call are displayed in the emotion estimation result column 2111 of the search result column 2110. In the example shown in Fig. 7, the emotional information of the customer in "Opening" is "Normal," the emotional information of the customer in "Understanding the Business Purpose" is "Dissatisfied," the emotional information of the customer in "Identity Verification" is "Normal," the emotional information of the customer in "Product Description" is "Questioning," and the emotional information of the customer in "Closing" is "Normal."

[0066] Similarly, for example, icons representing the customer's emotional information in each scene of the call are displayed in the emotion estimation result column 2121 of the search result column 2120. In the example shown in Fig. 7, the customer's emotional information in "Opening" is "Normal", the customer's emotional information in "Understanding the Business Purpose" is "Normal", the customer's emotional information in "Hearing About the Situation" is "Satisfied", and the customer's emotional information in "Closing" is "Satisfied".

[0067] When the search result field 2110 or the search result field 2120 is selected, more detailed content of the call information corresponding to the selected search result field is displayed.

[0068] In this way, supervisors and analysts can search for past calls that match search criteria, using segmentations that represent scenes and the emotions (especially customer emotions) at those segments. This makes it possible to extract calls in which customers are angry about a particular scene, which can be useful, for example, for planning measures to improve such calls, educating operators, and improving talk scripts. A talk script is, for example, a manual (or script) that describes what an operator should say for each scene.

[0069] In addition, it is possible to specify multiple pairs of divisions and emotions, such as ("opening", "anger") and ("closing", "satisfied"), as search conditions. This makes it possible to extract calls in which the customer was angry at the opening but satisfied at the closing. This makes it possible to evaluate such calls as good calls or use the information to educate operators.

[0070] <Response support> In order to support an operator's response during a call with a customer, a case will be described in which emotion information is used to display the customer's emotion in real time on the operator terminal 20, or in real time on the supervisor terminal 30 of the supervisor monitoring the operator. The response support process will be described below with reference to Fig. 8. The following response support process is repeatedly executed at predetermined time intervals (for example, every few seconds) during a call between the customer and the operator.

[0071] The UI providing unit 103 receives the speech-recognition text obtained by the speech recognition and text conversion unit 101 from a predetermined time ago to the present and the emotion information estimated by the emotion estimation unit 102 (step S201). It is assumed that the emotion estimation unit 102 estimates the speaker's emotion information for each utterance and for each segment.

[0072] Next, the UI providing unit 103 creates display information (display information for the response assistance screen or display information for the operator monitoring screen, or both) including the speech-recognized text and emotion information received in step S201 (step S202). Note that this display information may be the display information for the response assistance screen or the operator monitoring screen itself, or, if the response assistance screen or the operator monitoring screen is already displayed on the operator terminal 20 or the supervisor terminal 30, it may be display information showing the difference between them. In addition, the display information may include information to be displayed in an advice display field on the response assistance screen (described later), information for notifying an alert on the operator monitoring screen, etc.

[0073] Then, the UI providing unit 103 transmits the display information created in the above step S202 to the operator terminal 20 and the supervisor terminal 30 (step S203).

[0074] <<Response support screen>> An example of a response assistance screen displayed on the operator terminal 20 of a certain operator is shown in Fig. 9. Note that this response assistance screen is displayed on the operator terminal 20 based on display information (display information of the response assistance screen) created and transmitted by the UI providing unit 103.

[0075] The response support screen 3000 shown in FIG. 9 includes a current emotion display field 3010, a history display field 3020, and an advice display field 3030.

[0076] The current emotion display field 3010 displays the customer's current emotion information ("questioning" in the example shown in FIG. 9). Note that the current emotion display field 3010 may also display the customer's emotion information in the scene immediately before the current scene (i.e., the immediately preceding scene), for example.

[0077] The history display field 3020 displays the timing of customer and operator utterances from the start of the call to the present, as well as emotional information for each scene (division). In the example shown in FIG. 9, the customer's emotion in "Opening" is "Anger," the customer's environment in "Understanding the Business Purpose" is "Anger," and the current scene is "Product Explained." In addition, a display component indicating "Speech Occurred" is displayed at the timing when an utterance was made. Note that, for example, by hovering the mouse cursor or the like over the display component, speech recognition text or the like corresponding to the display component may be displayed.

[0078] The advice display field 3030 displays the current scene, information to support the operator, etc. In the example shown in Fig. 9, it displays that the current scene is "product explanation," that there are many "questions" in the customer's emotions in the current scene, and that an easy-to-understand explanation is needed.

[0079] In the example shown in FIG. 9, scenes are used as delimiters, but the delimiters shown in (A) and (B) above may also be used.

[0080] In this way, the operator can check the current customer's emotions and the customer's emotions at each interval in real time. Also, information to support the operator according to the customer's emotions can be checked in real time. This allows the operator to respond to the customer appropriately.

[0081] Furthermore, because the customer's emotions for each scenario are displayed in real time, the operator can, for example, detect false statements made by the customer. For example, in an "identity verification" scenario, when an operator confirms a customer's personal information, if the customer's emotion is "anxiety," the operator can detect a high possibility of false statements. The emotion information utilization device 10 may detect or estimate such a high possibility of false statements, and information representing the detection or estimation result may be transmitted to the operator terminal 20 and displayed in the advice display field of the customer service assistance screen. Additionally, advice may be selected and displayed in the advice display field of the customer service assistance screen based on the customer's or operator's emotional information at the current or previous point in time. The advice displayed in the advice display field may be selected from pre-prepared advice based on, for example, changes in the scenario or emotional information, or advice based on best practices from other excellent operators in the same situation may be selected, or advice matching certain conditions, such as the scenario or emotional information, may be selected.

[0082] For example, similar to the search result field 1110 on the search result screen 1100 shown in FIG. 5, the history display field 3020 may display information (utterance timing and emotional information) for the entire call or a segment prior to the latest segment only if the emotional information of the customer for the entire call or the latest segment meets certain conditions.

[0083] <Operator monitoring screen> 10 shows an example of an operator monitoring screen displayed on the supervisor terminal 30 of a certain supervisor. Note that this operator monitoring screen is displayed on the supervisor terminal 30 based on display information (display information of the operator monitoring screen) created and transmitted by the UI providing unit 103.

[0084] 10 includes call content fields 3110-3130 that display the call content of the operator being monitored by the supervisor. Each of the call content fields 3110-3130 displays the extension number, call duration, operator name, etc. Each of the call content fields 3110-3130 also includes current scene fields 3111-3131 and current emotion fields 3112-3132, respectively.

[0085] For example, the call content column 3110 includes a current scene column 3111 and a current emotion column 3112, with an icon representing "understanding the matter" displayed in the current scene column 3111 and an icon representing "satisfied" displayed in the current emotion column 3112. This indicates that the current scene of the call being monitored in the call content column 3110 is "understanding the matter" and the customer's current emotion is "satisfied."

[0086] Similarly, for example, the call content column 3120 includes a current scene column 3121 and a current emotion column 3122, with an icon representing "identity verification" displayed in the current scene column 3121 and an icon representing "question" displayed in the current emotion column 3122. This indicates that the current scene of the call being monitored in the call content column 3120 is "identity verification" and the customer's current emotion is "question."

[0087] Similarly, for example, the call content field 3130 includes a current scene field 3131 and a current emotion field 3132, with an icon representing "opening" displayed in the current scene field 3131 and an icon representing "anger" displayed in the current emotion field 3132. This indicates that the current scene of the call being monitored in the call content field 3130 is "opening" and the customer's current emotion is "anger."

[0088] When any of the call content fields 3110 to 3130 is selected, more detailed content of the call corresponding to the selected call content field (for example, the voice recognition text of the call) is displayed.

[0089] In this way, the supervisor can monitor in real time the current scene (break) in the call of the operator he / she is monitoring, the current emotion of the customer, etc. Therefore, the supervisor can identify a call that is likely to lead to a complaint based on the emotion of the customer, for example, and can intervene in the call to support the operator of that call.

[0090] Furthermore, for example, the supervisor may be notified of some information by any method based on the transition of the customer's emotional information. To cite a specific example, an alert (e.g., an alert by flashing or sound output) may be displayed on the operator monitoring screen. As an example, when the emotional information is estimated for each scene, an alert may be issued if the customer's emotion transitions from a state other than "anger" to "anger." Furthermore, for example, an alert may be issued if the customer's emotion is "anger" in a specific scene (e.g., "identity verification"). Furthermore, in order to exclude calls in which the customer is angry from the start of the call, an alert may not be issued if the customer's emotion is "anger" even in the "opening." The emotion information utilization device 10 may determine whether or not such an alert is required, and information indicating the determination result may be transmitted to the supervisor terminal 30.

[0091] <Call evaluation> In order to facilitate interpretation when evaluating a certain call, a case will be described in which past calls are modeled using emotional information, and the call to be evaluated is evaluated using this model. The call evaluation process will be described below with reference to Fig. 11. Note that the following steps S301 to S302 are processes that are performed in advance, and step S303 is a process that is performed for each call to be evaluated.

[0092] The evaluation unit 105 acquires evaluated call information from the call information DB 106 (step S301). Here, evaluated call information refers to call information that has been manually evaluated in advance from among the call information stored in the call information DB 106. Hereinafter, evaluated call information of a call that has been manually evaluated as an excellent call will be referred to as "excellent call information." There are various possible perspectives for evaluating whether a call is excellent or not. For example, it is possible to evaluate a call as excellent if it is "a call in which the customer was explained very well and the customer was satisfied" or "a call in which the product or service was recommended very well and led to a contract." However, these are merely examples and are not limited to these. A call that serves as a model for other operators in some respect can be evaluated as excellent.

[0093] Next, the evaluation unit 105 uses the evaluated call information acquired in step S301 to create an evaluation model by a known clustering method, a known machine learning method, or the like (step S302).

[0094] Then, the evaluation unit 105 evaluates the call information of the call to be evaluated using the evaluation model created in step S303 (step S303).

[0095] <<How to create an evaluation model and how to evaluate calls to be evaluated>> Hereinafter, the number of evaluated call information acquired in step S301 above is set to N, and each evaluated call information is set to x n (n=1, ,N). Also, assuming that emotion information is estimated for each utterance, the evaluated call information x n The emotional information of the kth utterance contained in e nk The speaker who made the kth utterance is p nk The time when the kth utterance was made is t nk In addition, the evaluated call information x n Call Reasons n Let's say.

[0096] At this time, the nth evaluated call information is x n ={rn ,{(e nk ,p nk ,t nk )|k=1, ,K n}}. K n is the nth evaluated call information x n The number of utterances included in the emotional information e nk may be a categorical value representing an emotion such as "anger," "satisfaction," or "dissatisfaction," or may be a vector or array whose elements are the probability or likelihood of these emotions.

[0097] If one evaluated call information contains multiple call reasons, the evaluated call information is divided for each call reason and expressed as above. For example, the nth evaluated call information x n Two Call Reasons n and r' n If it contains, evaluated call information x n {r n ,{(e nk ,p nk ,t nk )|k=1,···,K' n}} and {r' n ,{(e nk ,p nk ,t nk )|k=K' n +1,···,K n}} and then divide the former into x n ={r n ,{(e nk ,p nk ,t nk )|k=1, ,K n}}, and the latter is renumbered to the N+1th evaluated call information x N+1 ={r N+1 ,{(e N+1,k ,p N+1,k ,t N+1,k )|k=1, ,K N+1}} would be fine.

[0098] In the following, the number of evaluated call information items containing multiple call reasons will be represented as N, assuming that the above division has been performed.

[0099] (Method for creating an evaluation model and evaluation method for target calls, part 1) Hereinafter, it is assumed that excellent call information is acquired as evaluated call information in step S302 of FIG.

[0100] Learning data used to create evaluation models Any of the following (a) to (d) will be used as training data.

[0101] (a)x n ={e nk |k=1, ,K n}(n=1, ,N) is used as the learning data. In other words, only the sequence of emotional information contained in each piece of good call information is used as the learning data.

[0102] (b)x n ={(e nk ,p nk )|k=1, ,K n}(n=1, ,N) is used as training data. In other words, only the emotional information and speaker sequences contained in each good call information are used as training data.

[0103] (c)x n ={(e nk ,p nk ,t nk )|k=1, ,K n}(n=1, ,N) is used as training data. In other words, the emotional information contained in each piece of good call information and the sequence of the speaker and the speech time are used as training data.

[0104] (d)x n ={r n ,{(e nk ,p nk ,t nk )|k=1, ,K n}}(n=1, ,N) is used as training data. In other words, in addition to the emotional information and the sequence of speaker and speech time included in each piece of good call information, the call reasons included in that piece of good call information are also used as training data.

[0105] Modeling method For example, a clustering technique for variable-length sequences is used, which constructs clusters from the training data and serves as the evaluation model.

[0106] Evaluation data The call information to be evaluated is x={r,{(e k ,p k ,t k )|k=1,···,K}} (where K is the number of utterances), the evaluation data will have the same format as the training data. In other words, if the above (a) is used as the training data, then x={e k |k=1,···,K}, and if the above (b) is used as the training data, then x={(e k ,p k )|k=1, ,K}, and if the above (c) is used as the training data, then x={(e k ,p k ,t k )|k=1,···,K}, and if the above (d) is used as the training data, then x={r,{(e k ,p k ,t k )|k=1,···,K}} is used as the evaluation data.

[0107] Evaluation method If the distance to the center of gravity of any cluster is small (for example, if the distance is below a predetermined threshold), the call to be evaluated is evaluated as a good call, and if not, the call to be evaluated is evaluated as not a good call. Also, depending on which cluster's center of gravity the distance is small, the type of good call the call to be evaluated is evaluated as (for example, whether it is "a call where the customer was explained very well and the customer was satisfied" or "a call where the product or service was recommended very well and led to a contract")

[0108] (How to create an evaluation model and how to evaluate calls to be evaluated, part 2) 10, it is assumed that the evaluated call information acquired in step S302 includes good call information, call information of calls evaluated as not bad calls (hereinafter referred to as normal call information), and call information of calls evaluated as calls requiring improvement (hereinafter referred to as call information requiring improvement). However, it is not necessarily necessary to acquire three types of information: good call information, normal call information, and call information requiring improvement. For example, if you want to evaluate whether the call to be evaluated is a good call or not, it is sufficient to acquire only good call information and normal call information, and on the other hand, if you want to evaluate whether the call to be evaluated is a call requiring improvement or not, it is sufficient to acquire only normal call information and call information requiring improvement.

[0109] Learning data used to create evaluation models Any of the above (a) to (d) will be used as training data. Note that this training data will be provided with information indicating whether it was evaluated as a "good call," "average call," or "call requiring improvement" as training data.

[0110] Modeling method For example, a classification model that classifies calls into three classes, "good calls," "average calls," and "calls requiring improvement," is constructed as an evaluation model by supervised learning using machine learning techniques. For example, if it is desired to evaluate whether a call to be evaluated is a good call or not, a classification model that classifies calls into two classes, "good calls" or "average calls" (other than good calls), can be constructed as an evaluation model, and on the other hand, if it is desired to evaluate whether a call to be evaluated is a call requiring improvement or not, a classification model that classifies calls into two classes, "average calls" or "calls requiring improvement" can be constructed as an evaluation model.

[0111] Evaluation data The call information to be evaluated is x={r,{(e k ,p k ,t k )|k=1,···,K}} (where K is the number of utterances), the evaluation data has the same format as the training data.

[0112] Evaluation method The call to be evaluated is evaluated as either a "good call," a "normal call," or a "call requiring improvement" based on the output when the evaluation data is input into the evaluation model.

[0113] As a result, when evaluating a call, the evaluation results can be easily interpreted, and the interpretation results can be used for various analyses (e.g., analysis to improve response quality) or for evaluating operators (e.g., awarding excellent operators).

[0114] The present invention is not limited to the above-described specifically disclosed embodiments, and various modifications, changes, and combinations with known technologies are possible without departing from the scope of the claims. [Explanation of symbols]

[0115] 1. Contact Center System 10 Emotional Information Utilization Device 20 Operator terminal 30 Supervisor Terminal 40 Analyst terminal 50 PBX 60 Customer terminals 70 Communication Network 101 Speech recognition and text conversion unit 102 Emotion estimation part 103 UI provision department 104 Search Section 105 Evaluation Department 106 Call Information DB

Claims

1. a database storing call information including emotion information that expresses the emotion of a speaker at least for each predetermined interval; a search unit that searches the database for the call information using search conditions that include at least the delimiter and the emotion information; An emotional information utilization device having the above.

2. 2. The emotion information utilization device according to claim 1, wherein the division is either a time division, a scene division representing a topic scene in the call corresponding to the call information, or a division in units in which the call is divided by a predetermined call event.

3. 3. The emotion information utilization device according to claim 1, further comprising a UI providing unit that displays emotion information included in the call information searched by the search unit on a first display unit.

4. The UI providing unit 4. The emotion information utilization device according to claim 3, wherein the emotion information for each segment included in the call information searched by the search section is displayed on the first display section.

5. The UI providing unit 5. The emotion information utilization device according to claim 4, wherein, when emotion information of a latest segment among the emotion information for each segment included in the call information searched by the search unit is emotion information indicating a negative emotion, the emotion information of the latest segment and emotion information of segments preceding the latest segment are displayed on the first display unit.

6. The UI providing unit 6. The emotion information utilization device according to claim 3, wherein the emotion information is displayed on the first display unit provided in a first terminal connected to the emotion information utilization device via a communications network.

7. an emotion estimation unit that estimates emotion information for each segment and emotion information for each utterance from utterances related to a call between a first speaker and a second speaker; The UI providing unit 7. The emotion information utilization device according to claim 3, wherein the emotion information for each segment and the emotion information for each utterance are displayed on a second display unit while the call is being made.

8. The UI providing unit 8. The emotion information utilization device according to claim 7, wherein when specific emotion information is estimated at a specific segment, the specific information is notified to a specific notification destination.

9. The UI providing unit 9. The emotion information utilization device according to claim 8, wherein even if the specific emotion information is estimated at the specific division, when the specific emotion information is estimated at a first division of the call, the specific information is not notified to the predetermined notification destination.

10. The UI providing unit 10. The emotion information utilization device according to claim 8 or 9, wherein, with regard to the emotion information for each segment, when there is a change from emotion information other than that indicating a negative emotion to emotion information that indicates a negative emotion, when there is a change from emotion information other than that indicating a positive emotion to emotion information that indicates a positive emotion, when emotion information that indicates a negative emotion continues for a certain segment, or when emotion information that indicates a positive emotion continues for a certain segment, the predetermined information is notified to the predetermined notification destination.

11. The UI providing unit 11. The emotion information utilization device according to claim 8, wherein the predetermined information is notified to a second terminal connected to the emotion information utilization device via a communication network.

12. The emotion estimation unit 12. The emotion information utilization device according to claim 7, further comprising an evaluation unit that creates an evaluation model that models emotion information included in call information that has been manually evaluated in advance from among the call information stored in the database, and evaluates the call information of a call to be evaluated using the evaluation model.

13. The evaluation unit The emotion information utilization device according to claim 12, wherein an evaluation model is created by modeling the sequence of emotion information for each utterance using clustering or machine learning technology, and the call information of the call to be evaluated is evaluated using the evaluation model.

14. a storage step of storing call information including emotion information representing the emotion of a speaker at least for each predetermined segment in a database; a search step of searching the database for the call information using search criteria that include at least the delimiter and the emotion information; A method for utilizing emotional information using a computer.

15. a storage step of storing call information including emotion information representing the emotion of a speaker at least for each predetermined segment in a database; a search step of searching the database for the call information using search criteria that include at least the delimiter and the emotion information; A program that causes a computer to execute the following.

Images