Conversation support device, method and program

Abstract

To perform appropriate correspondence by considering a type of a topic in conversation and emotion of an opposite speaker.SOLUTION: Information indicating a type of a topic in conversation is obtained and a feature amount of an emotion sequence is extracted from uttered voice of an opposite speaker in conversation when supporting utterance of an object speaker when the object speaker has conversation with the opposite speaker. When the type of the topic in conversation and the feature amount of emotion when the opposite speaker speaks are inputted, recommendation emotion recommended for the object speaker is estimated and information indicating estimated recommendation emotion is outputted based on relevance information in which relevance is defined between the feature amount of the emotion sequence when the opposite speaker speaks and emotion of the object speaker for each type of the topic of conversation.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] One aspect of the present invention relates to a conversation support device, method, and program used to support conversations between operators and customers in, for example, a call center. [Background technology]

[0002] In recent years, the number of ways for companies to communicate with their customers has increased, and customer satisfaction and corporate image are becoming more and more important. In order to improve customer satisfaction, it is necessary to improve the quality of customer service at call centers and storefronts where customers' requests and opinions can be heard directly.

[0003] The likeability rating of call center operators and store clerks is one of the important factors for improving customer satisfaction, etc. If the likeability rating can be estimated, it can be used to evaluate the service quality of operators and store clerks, search for good examples, and educate operators and store clerks, and can be applied to improve the quality of service at call centers and store clerks.

[0004] Therefore, a technology has been proposed in which features are generated from a speech voice, and the likeability rating of the speaker of the speech voice is estimated from the features based on the correlation between the features of the generated speech voice and the speaker's likeability rating (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6859283 Summary of the Invention [Problem to be solved by the invention]

[0006] The technology described in Patent Document 1 is useful as a means of advance speech training because it can estimate the likeability of a speaker who has uttered an utterance by taking into account not only the acoustic features of the utterance but also features such as the way in which interjections are used, the use of honorific language, and the clarity of sentence endings.

[0007] However, a conversation is conducted by a back and forth of words between the other speaker, and the appropriate content of the speech changes depending on the emotions of the other speaker. In particular, in a call center or other on-site environment, the emotions of the other speaker vary depending on the type of topic in the conversation and the situation at the time, and emotions often change over time during the conversation. For this reason, simply applying the technology described in the cited document 1 may result in cases where the operator is unable to respond appropriately to the topic of the conversation or the emotions of the other speaker at the time.

[0008] The present invention has been made in light of the above circumstances, and aims to provide a technique for taking into consideration the type of subject matter in a conversation and the feelings of the other speaker, and enabling an appropriate response. [Means for solving the problem]

[0009] In order to solve the above problem, one aspect of the conversation support device or support method according to the present invention, when supporting the speech of a target speaker when the target speaker converses with a partner speaker using an information processing device, acquires information representing a subject of the conversation and extracts features of an emotion sequence from the speech of the partner speaker in the conversation. Then, for each subject of the conversation, relevance information is provided that defines a relevance between the features of the emotion sequence at the time of the partner speaker's speech and the emotion of the target speaker when the target speaker responds following the speech of the partner speaker, and when the information representing the subject of the conversation and the features of the emotion at the time of the partner speaker's speech are input, a recommended emotion to be recommended to the target speaker is estimated based on the relevance information, and information representing the estimated recommended emotion is output.

[0010] According to one aspect of the present invention, the recommended emotion for the target speaker to respond to the other speaker's utterance is estimated and output, taking into consideration the type of conversation topic and the other speaker's emotion at the time of utterance. Therefore, during the conversation, the emotion that the target speaker should keep in mind in response to the other speaker's emotion at each moment is presented to the target speaker for each conversation topic. As a result, it is possible to smoothly proceed with the conversation with the other speaker, taking into consideration the other speaker's emotion for each topic, and to increase the success rate of the conversation. Effect of the Invention

[0011] That is, according to one aspect of the present invention, it is possible to provide a technique that enables an appropriate response to be made in consideration of the type of subject matter in a conversation and the emotions of the other speaker. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram showing an example of a system including a conversation support device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing an example of a hardware configuration of a conversation support device according to an embodiment of the present invention. [Diagram 3] FIG. 3 is a block diagram showing an example of a software configuration of the conversation support device according to an embodiment of the present invention. [Figure 4] FIG. 4 is a flowchart showing an example of a process procedure and process contents in a learning phase executed by the control unit of the conversation support device shown in FIG. [Diagram 5] FIG. 5 is a flowchart showing an example of a process procedure and process contents in the estimation phase executed by the control unit of the conversation support device shown in FIG. [Figure 6] FIG. 6 is a diagram showing an example of the association between the operator's emotion and the customer's emotion. [Figure 7] FIG. 7 is a diagram showing an example of a pattern of time-series changes in customer sentiment as a feature quantity of customer sentiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0014] [One embodiment] (Configuration example) (1) System FIG. 1 is a diagram showing an example of the configuration of a system including a conversation support device according to an embodiment of the present invention.

[0015] The system according to one embodiment of the present invention is equipped with a conversation support device SV, which monitors conversations taking place via a network NW between customer terminals UT1 to UTn used by customers and operator terminals MT1 to MTk used by call center operators, and supports the operator's speech based on the results of the conversation support device SV.

[0016] The network NW includes a wide area network with the Internet at its core, and an access network for accessing the wide area network. The access network may be, for example, a public data communication network using wired or wireless communication, a local area network (LAN) using wired or wireless communication, or a cable television (CATV) network.

[0017] The customer terminals UT1 to UTn are, for example, smartphones, and have a function of performing voice communication with other terminals including the operator terminals MT1 to MTk using a communication protocol defined in the network NW. Note that the customer terminals UT1 to UTn are not limited to smartphones, and any terminals having a similar voice communication function can be used.

[0018] The operator terminals MT1 to MTk are, for example, personal computers to which headsets HS1 to HSk are connected. By using the headsets HS1 to HSk, the operators can carry out voice communication with the customers using the customer terminals UT1 to UTn.

[0019] (2) Conversation Support Device SV The conversation support device SV is configured, for example, by a server computer located on the Web or on the cloud.

[0020] 2 and 3 are block diagrams showing an example of a hardware configuration and a software configuration, respectively, of the conversation support device SV.

[0021] The conversation support device SV includes a control unit 1 using a hardware processor such as a central processing unit (CPU). A storage unit having a program storage unit 2 and a data storage unit 3, and a communication interface (hereinafter, the interface will be abbreviated as I / F) unit 4 are connected to the control unit 1 via a bus 5.

[0022] The communication I / F unit 4 operates in accordance with a communication protocol defined by the network NW under the control of the control unit 1. The communication I / F unit 4 acquires voice communication data transmitted and received between the customer terminals UT1 to UTn and the operator terminals MT1 to MTk, and transmits speech support information for the operators to the operator terminals MT1 to MTk.

[0023] The program storage unit 2 is configured by combining a non-volatile memory such as a solid state drive (SSD) that can be written and read at any time as a storage medium, and a non-volatile memory such as a read only memory (ROM), and stores application programs required for executing various control processes according to an embodiment, in addition to middleware such as an operating system (OS). Hereinafter, the OS and each application program will be collectively referred to as a program.

[0024] The data storage unit 3 is, for example, a combination of a non-volatile memory such as an SSD that can be written to and read from at any time as a storage medium, and a volatile memory such as a RAM (Random Access Memory), and the storage area includes a learning data storage unit 31, an emotion estimation model storage unit 32, an emotion data set storage unit 33, and an utterance information storage unit 34.

[0025] The learning data storage unit 31 stores learning data. The learning data is used to train the emotion estimation model, and an example of the learning data will be described later.

[0026] The emotion estimation model storage unit 32 stores learned data including parameters of the emotion estimation model. The emotion estimation model receives the type of conversation subject and the feature amount of the customer's emotion sequence as input, estimates the emotion to be recommended to the operator in response to the input, and outputs the estimation result, and is configured using a machine learning model such as a support vector machine (SVM), a random forest, or a neural network.

[0027] The emotion data set storage unit 33 stores a set of predefined emotion data in association with a speech feature vector.

[0028] The speech information storage unit 34 stores a set of information indicating the contents of speech of an agent recommended for a customer's speech for each business matter in association with a recommended emotion.

[0029] The control unit 1 includes, as processing functions necessary for implementing one embodiment of the present invention, a learning data acquisition processing unit 11, a learning processing unit 12, a conversation subject type acquisition processing unit 13, a partner speech voice acquisition processing unit 14, a feature extraction processing unit 15, a feeling estimation processing unit 16, and a support information output processing unit 17.

[0030] Each of the processing units 11 to 17 of the control unit 1 is realized by causing a hardware processor of the control unit 1 to execute an application program stored in the program storage unit 2. Note that a part or all of the processing units 11 to 17 may be realized using hardware such as an LSI (Large Scale Integration) or an ASIC (Application Specific Integrated Circuit).

[0031] In the learning phase, the learning data acquisition processing unit 11 acquires learning data reflecting successful examples of past conversations, for example, from the operator terminals MT1 to MTk, and stores the acquired learning data in the learning data storage unit 31.

[0032] In the learning phase, the learning processing unit 12 causes the emotion estimation model to learn parameters using the learning data, and stores learned data including the learned parameters in the emotion estimation model storage unit 32. An example of the learning process will be described in an operation example.

[0033] In the estimation phase, the conversation subject type acquisition processing unit 13 receives, via the communication I / F unit 4, information indicating the type of conversation subject input by the operator at, for example, the operator terminal MT1 to MTk, prior to or immediately after the start of a conversation with a customer.

[0034] The other party's speech voice acquisition processing unit 14 receives speech voice data transmitted from the customer terminals UT1 to UTn during a conversation via the communication I / F unit 4, and passes the received speech voice data to the feature amount extraction processing unit 15.

[0035] The feature extraction processing unit 15 calculates a voice feature vector from the speech voice data passed from the other party's speech acquisition processing unit 14, and reads out corresponding emotion data from the emotion data set storage unit 33 based on the calculated voice feature vector. Then, it outputs the emotion transition represented by the read emotion data as the feature of the customer's emotion series.

[0036] The emotion estimation processing unit 16 inputs the information indicating the type of conversation subject acquired by the conversation subject type acquisition processing unit 13 and the feature quantity of the customer's emotion sequence output from the feature quantity extraction processing unit 15 to the trained emotion estimation model, thereby acquiring a recommended emotion for the agent that is estimated and output by the emotion estimation model.

[0037] The support information output processing unit 17 transmits the recommended emotion for the operator, estimated and output by the emotion estimation model, to the operator terminals MT1 to MTk in conversation from the communication I / F unit 4. At the same time, the support information output processing unit 17 selects a candidate utterance content corresponding to the recommended emotion from the utterance information storage unit 34, and transmits the selected candidate utterance content from the communication I / F unit 4 to the operator terminals MT1 to MTk.

[0038] (Example of operation) Next, an example of the operation of the conversation support device SV configured as above will be described.

[0039] In this operation example, an inbound conversation is conducted in a call center where an operator responds to a request for a conversation on the subject of an "inquiry" or "complaint" from a customer, and an outbound conversation is conducted where an operator speaks with a customer on the subject of a "request" or "sales promotion."

[0040] (1) Learning Phase FIG. 4 is a flowchart showing an example of a processing procedure and processing content of a process related to learning of a feeling estimation model, which is executed by the control unit 1 of the conversation support device SV in the learning phase.

[0041] (1-1) Acquiring learning data Prior to the learning process, the control unit 1 of the conversation support device SV, under the control of the learning data acquisition processing unit 11, acquires learning data in step S10.

[0042] For example, the learning data acquisition processing unit 11 acquires learning data reflecting successful examples of past conversations for each of the above-mentioned subjects, i.e., for “inquiry” and “complaint” in inbound conversations and “urging” and “sales promotion” in outbound conversations, from the operator terminals MT1 to MTk via the communication I / F unit 4. Then, the acquired learning data is stored in the learning data storage unit 31.

[0043] The learning data is a set of information that is created for each of the above-mentioned items and represents a combination of features of the emotion sequence when the customer speaks and a correct answer value of the emotion of the operator when the operator responds following the customer's utterance.

[0044] Generally, in a conversation between a customer and an operator, if the operator's emotions are "positive" towards what the customer has said, the customer's emotions immediately afterwards tend to become "positive." Figure 6 shows an example of the changes in the emotions of an operator and a customer in this case.

[0045] In other words, when a customer's emotional sequence tends to include "anger" and "sadness," if the operator responds with a "positive" emotion, the customer's emotional sequence will then transition to "positivity," resulting in a successful conversation in many cases.

[0046] Therefore, the conversation support device SV acquires learning data created based on examples of emotional transitions between customers and agents when conversations are successful according to the subject matter as described above, and stores the data in the learning data storage unit 31.

[0047] In the above example, the learning data is obtained from the operator terminals MT1 to MTk. However, the learning data may be created by the learning data acquisition processing unit 11 of the conversation support device SV based on past conversation record data stored in the data storage unit 3.

[0048] In addition, there are two possible methods for setting the correct value of the operator's emotion. One is to obtain information indicating whether the conversation was successful or not input by the operator at the operator terminal MT1-MTk after the conversation is finished, and set a correct value for the emotion of the operator when the conversation is successful based on the obtained information. The other is to monitor in the control unit 1 of the conversation support device SV whether or not another conversation is held with the same customer within a predetermined period after the conversation is finished, and if not, to determine that the conversation was successful and set a correct value for the emotion of the operator at that time.

[0049] (1-2) Learning the emotion estimation model Next, the control unit 1 of the conversation support device SV, under the control of the learning processing unit 12, causes the emotion estimation model to perform learning as follows.

[0050] That is, the learning processing unit 12 first selects one of the sets of learning data from the learning data storage unit 31 in step S11. Next, in step S12, using the selected learning data, information indicating the type of subject of the conversation and feature amounts of the emotion sequence at the time of the customer's speech, which are included in the learning data, are input to estimate the emotion of the operator using the emotion estimation model. Then, parameters of the emotion estimation model are updated so that the error between the estimated emotion of the operator and the correct value defined in the learning data is reduced.

[0051] The learning processing unit 12 determines whether the error has decreased below the threshold value, and if the error has converged below the threshold value, in step S15, stores the updated parameters as learned data in the emotion estimation model storage unit 32. On the other hand, if the word difference has not converged below the threshold value, the learning processing unit 12 returns to step S11 to select the next learning data and repeats the learning process. Then, when learning using all the learning data is completed, the final parameters are stored as learned data in the emotion estimation model storage unit 32, and the learning process is terminated.

[0052] (2) Estimation Phase After the learning phase is completed, when the estimation phase is set, the control unit 1 of the conversation support device SV executes a process for supporting the operator's emotional state and speech content when responding during a conversation according to the subject matter, as follows.

[0053] FIG. 5 is a flowchart showing an example of the procedure and contents of the operator assistance process executed by the control unit 1 of the conversation assistance device SV in the estimation phase.

[0054] (2-1) Inbound Conversation When the start of a conversation is detected in step S20, the control unit 1 of the conversation support device SV first acquires information indicating the type of subject in the inbound conversation under the control of the conversation subject type acquisition processing unit 13 in step S21.

[0055] For example, assume that an inbound conversation has started between the customer terminal UT1 and the operator terminal MT1. In this case, the operator determines the customer's subject matter from the contents of the conversation, i.e., whether it is an "inquiry" or a "complaint," and inputs the determination result to the operator terminal MT1. As a result, information indicating the type of subject matter is notified from the operator terminal MT1 to the conversation support device SV. The control unit 1 of the conversation support device SV receives the information indicating the type of subject matter notified from the operator terminal MT1 via the communication I / F unit 4 under the control of the conversation subject type acquisition processing unit 13.

[0056] Next, in step S22, the control unit 1 of the conversation support device SV receives the customer's speech voice data transmitted from the customer terminal UT1 via the communication I / F unit 4 under the control of the other party's speech voice acquisition processing unit .

[0057] Next, in step S23, under the control of the feature extraction processing unit 15, the control unit 1 of the conversation support device SV extracts features representing the emotion sequence of the customer from the received speech voice data of the customer.

[0058] For example, the feature extraction processing unit 15 first calculates a speech feature vector from the customer's speech data. The speech feature vector is calculated by, for example, cepstrum analysis, which is often used as a speech analysis method. More specifically, the speech feature vector is calculated as a vector of about 10 to 100 dimensions that combines MFCC (Mel Frequency Cepstral Coefficient), ΔMFCC, ΔΔMFCC, logarithmic power, Δlogarithmic power, etc.

[0059] The feature extraction processing unit 15 compares the calculated voice feature utilization vector with the emotion data set stored in the emotion data set storage unit 33, and outputs the emotion data showing the highest likelihood as the emotion of the customer.

[0060] Emotion data may be defined as, for example, "anger," "sadness (anxiety)," and "positive," and the feature extraction processing unit 15 selects and outputs the emotion data with the highest likelihood from among these emotion data. Figure 7 shows an example of the change over time in the intensity of a customer's emotion. That is, in this example, EM1 shows the change over time in the intensity of "anger," EM2 shows the change over time in the intensity of "sadness," and EM3 shows the change over time in the intensity of "positive."

[0061] The calculation of the likelihood is described in detail in, for example, the following document. References: Kiyohiro Shikano, Katsunori Ito, Tatsuya Kawahara, Kazuya Takeda, Mikio Yamamoto, "IT Text Speech Recognition System", pp.1-51, 2001, Ohmsha.

[0062] Next, under the control of the emotion estimation processing unit 16, the control unit 1 of the conversation support device SV first inputs, in step S24, the feature quantities of the emotion sequence of the customer extracted by the feature quantity extraction processing unit 15 to the emotion estimation model together with information indicating the type of the subject of the conversation previously acquired by the conversation subject type acquisition processing unit 13. As a result, the emotion estimation model estimates and outputs a recommended emotion for the operator based on the input type of the subject of the conversation, i.e., "inquiry" or "complaint", and the feature quantities of the emotion sequence of the customer.

[0063] In step S25, the emotion estimation processing unit 16 receives the recommended emotion for the operator estimated by the emotion estimation model, and passes it to the support information output processing unit 17.

[0064] In step S26, the support information output processing unit 17 transmits information, such as text data, representing the recommended emotion estimated by the emotion estimation processing unit 16 from the communication I / F unit 4 to the operator terminal MT1.

[0065] At the same time, in step S27, the support information output processing unit 17 accesses the utterance information storage unit 34 based on the estimated recommended emotion, thereby selecting a candidate utterance content corresponding to the recommended emotion for the operator. Then, in step S28, the support information output processing unit 17 transmits information, such as text data, representing the selected candidate utterance content from the communication I / F unit 4 to the operator terminal MT1.

[0066] As a result, the operator terminal MT1 displays the recommended emotion for the operator and text data of candidate speech contents on, for example, a display device. Therefore, the operator controls his / her own emotion so that it matches the displayed recommended emotion, and then talks with the customer while referring to the candidate speech contents. This enables the operator to appropriately control emotions and talk with appropriate content in response to "inquiries" or "complaints," which are matters unique to inbound conversations.

[0067] (2-2) Outbound conversations When the start of a conversation is detected in step S20, the control unit 1 of the conversation support device SV first acquires information indicating the type of subject in the outbound conversation under the control of the conversation subject type acquisition processing unit 13 in step S21.

[0068] For example, prior to a conversation, an operator inputs the subject of the conversation, i.e., either "urging" or "sales promotion," into the operator terminal MT1. As a result, information indicating the type of subject is notified from the operator terminal MT1 to the conversation support device SV. The control unit 1 of the conversation support device SV receives the information indicating the type of subject notified from the operator terminal MT1 via the communication I / F unit 4 under the control of the conversation subject type acquisition processing unit 13, and passes it to the emotion estimation processing unit 16.

[0069] Next, in step S22, the control unit 1 of the conversation support device SV receives the customer's speech voice data transmitted from the customer terminal UT1 via the communication I / F unit 4 under the control of the other party's speech voice acquisition processing unit 14. Then, in step S23, the control unit 1 of the conversation support device SV extracts features of the customer's emotion sequence from the received customer's speech voice data under the control of the feature extraction processing unit 15. The extraction of the features of the customer's emotion sequence is similar to that in the case of inbound conversation, so a description thereof will be omitted here.

[0070] Next, under the control of the emotion estimation processing unit 16, the control unit 1 of the conversation support device SV first inputs, in step S24, the feature quantities of the emotion sequence of the customer extracted by the feature quantity extraction processing unit 15 to the emotion estimation model together with information indicating the type of subject of the conversation previously acquired by the conversation subject type acquisition processing unit 13. As a result, the emotion estimation model estimates and outputs a recommended emotion for the agent based on the input subject of the conversation, that is, "urging" or "sales promotion" in this case, and the feature quantities of the emotion sequence of the customer.

[0071] In step S25, the emotion estimation processing unit 16 acquires a recommended emotion for the operator from the emotion estimation model, and passes it to the support information output processing unit 17.

[0072] In step S26, the support information output processing unit 17 transmits the text data representing the recommended emotion output from the emotion estimation processing unit 16 from the communication I / F unit 4 to the operator terminal MT1.

[0073] At the same time, in step S27, the support information output processing unit 17 accesses the utterance information storage unit 34 based on the estimated recommended emotion, thereby selecting a candidate utterance content corresponding to the recommended emotion for the operator. Then, in step S28, the support information output processing unit 17 transmits text data representing the selected candidate utterance content from the communication I / F unit 4 to the operator terminal MT1.

[0074] As a result, the operator terminal MT1 displays the recommended emotion for the operator and text data showing candidate speech contents on the display device. Therefore, the operator controls his / her own emotion so that it matches the displayed recommended emotion, and then talks with the customer while referring to the candidate speech contents. This enables the operator to appropriately control emotions and talk with appropriate content for "urging" or "sales promotion," which are matters specific to outbound conversations.

[0075] (3) Conversation termination control Finally, in step S29, the control unit 1 of the conversation support device SV judges whether the conversation has ended. If the conversation is still ongoing, the process returns to step S22 and repeats the series of processes in the estimation phase from step S22 to S29 described above. On the other hand, if the conversation has ended, the process ends and the process returns to the standby state.

[0076] (effect) As described above, in one embodiment, in the learning phase, a set of learning data of successful cases is acquired for each of the inbound conversation and outbound conversation between the customer and the operator, and the emotion estimation model is trained using this set of learning data. Then, in the estimation phase, when the operator converses with the customer, information indicating the type of the conversation subject is acquired, and features of the customer's emotion sequence are extracted from the customer's speech voice data, and these are input to the emotion estimation model, and an estimation result of a recommended emotion for the operator is acquired from the emotion estimation model. Then, the acquired estimation result of the recommended emotion is notified to the operator, and candidates of the speech content corresponding to the recommended emotion are selected and presented to the operator.

[0077] Therefore, in both inbound and outbound conversations, the operator can control his / her own emotions to be the recommended emotions notified by the conversation support device SV for each subject, and then can converse with the customer by referring to the candidate speech contents presented by the conversation support device SV. As a result, the operator can have a conversation with appropriate contents based on the optimal emotion according to the transition of the customer's emotion series for each subject in each scene of inbound and outbound conversations, which makes it possible to increase the success rate of the conversation.

[0078] [Other embodiments] (1) The emotion estimation operation by the emotion estimation model is not limited to the operation example described in the above embodiment, and other operation examples are also possible. For example, the type of conversation subject and the feature amount of the customer's emotion sequence extracted from the customer's speech voice data are input to the emotion estimation model, and the emotion estimation model clusters the feature amount of the input customer's emotion sequence. Then, if the cluster classified by the above clustering is a cluster with a high possibility of conversation failure, the emotion estimation model transitions to a cluster with a successful conversation, and estimates and outputs a recommended emotion for the operator from the feature amount of the emotion sequence defined in the transition destination cluster.

[0079] As described above, by using clustering to roughly classify customer emotion sequences and training the emotion estimation model to estimate recommended emotions for an agent based on the results, the amount of processing required for the emotion estimation model to estimate recommended emotions for an agent is reduced, making it possible to reduce the processing load on the conversation support device SV and shorten the time required for estimation processing.

[0080] (2) In one embodiment, the recommended emotion for the operator estimated using the emotion estimation model is presented to the human operator, but for example, the operator may be a virtual operator using an avatar or the like, and the facial expression and tone of speech of the virtual operator may be controlled based on the recommended emotion for the operator estimated using the emotion estimation model. Also, a speech voice may be synthesized based on candidates for speech content, and the synthesized speech voice may be output as a voice uttered by the virtual operator.

[0081] (3) In one embodiment, an example is given of the conversation support device SV being installed on a server computer on the Web or in the cloud, but the functions of the conversation support device SV may also be provided in a terminal such as operator terminals MT1-MTk or an administrator terminal used by a system administrator.

[0082] (4) The present invention can be applied not only to call centers but also to question and answer sessions between lecturers and participants in various seminars, exchanges of opinions in conferences, etc. In addition, the functional configuration of the conversation support device, its processing procedures, processing contents, etc. can be modified in various ways without departing from the gist of the present invention.

[0083] Although the embodiment of the present invention has been described in detail above, the above description is merely an example of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. In other words, in implementing the present invention, specific configurations according to the embodiment may be appropriately adopted.

[0084] In short, this invention is not limited to the above-mentioned embodiment as it is, and in the implementation stage, the components can be modified and embodied without departing from the gist of the invention. In addition, various inventions can be formed by appropriately combining multiple components disclosed in the above-mentioned embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, components from different embodiments may be appropriately combined. [Explanation of symbols]

[0085] SV: Conversation support device UT1~UTn…Customer terminal MT1~MTk…Operator terminal HS1~HSk…Headset NW…Network 1...Control section 2...Program memory section 3. Data storage section 4. Communication I / F section 5. Bus 11...Learning data acquisition processing unit 12...Learning processing unit 13...Conversation subject type acquisition processing unit 14...Other party's speech acquisition processing unit 15...Feature extraction processing section 16...Emotion estimation processing unit 17...Support information output processing unit 31...Learning data storage unit 32…Emotion estimation model memory section 33...Emotion data set storage unit 34... Speech information storage unit

Claims

1. A conversation support device that supports a target speaker's speech when the target speaker has a conversation with a partner speaker, A first processing unit that acquires information indicating a type of the subject of the conversation; a second processing unit that extracts emotion sequence features from the speech of the other speaker in the conversation; a third processing unit which has, for each type of subject of the conversation, relevance information defining a relevance between a feature of an emotion sequence at the time of the other speaker's utterance and a recommended emotion of the target speaker when the target speaker responds following the utterance of the other speaker, and which estimates the recommended emotion to be recommended to the target speaker based on the relevance information when information expressing the type of subject of the conversation and the feature of the emotion sequence at the time of the other speaker's utterance are input; a fourth processing unit that outputs information representing the estimated recommended emotion to the target speaker; A conversation support device comprising:

2. In the case of an inbound conversation in which the target speaker responds to a conversation request from the other speaker, 2. The conversation support device according to claim 1, wherein, when information indicating a type of a first matter specific to an inbound conversation and a feature of an emotion sequence extracted from the speech of the other speaker are input in the inbound conversation, the third processing unit estimates the recommended emotion to be recommended to the target speaker in the inbound conversation based on first relevance information defined corresponding to the first matter.

3. When an outbound conversation is conducted in which the other speaker speaks in response to a conversation request from the target speaker, 2. The conversation support device according to claim 1, wherein, when information indicating a type of a second matter specific to an outbound conversation and a feature of an emotion sequence extracted from the speech of the other speaker are input in the outbound conversation, the third processing unit estimates the recommended emotion to be recommended to the target speaker in the outbound conversation based on second relevance information defined corresponding to the second matter.

4. 2. The conversation support device according to claim 1, wherein the fourth processing unit selects a candidate utterance content corresponding to the information expressing the recommended emotion in addition to or instead of the information expressing the recommended emotion estimated by the third processing unit, and outputs the selected candidate utterance content to the target speaker.

5. When the third processing unit includes an emotion estimation model that estimates the recommended emotion to be recommended to the target speaker based on the relevance information, A fifth processing unit that trains the emotion estimation model, The fifth processing unit is an acquisition processing unit that acquires, as learning data, a set of combinations of feature amounts of emotion sequences at the time of the utterance of the other speaker and correct answer values ​​of the emotion of the target speaker when the target speaker responds following the utterance of the other speaker, the combinations being prepared for each type of the conversation subject; a learning processing unit that uses information indicating a type of the subject of the conversation and a feature amount of an emotion sequence at the time of the other speaker's utterance, which are included in the learning data, as input, to estimate the emotion of the target speaker using the emotion estimation model, and updates parameters of the emotion estimation model so as to reduce an error between the estimated emotion of the target speaker and the ground truth value; a storage processing unit that stores learned data including the parameters when the error is reduced to a preset value or less; The conversation support device according to claim 1 .

6. 6. The conversation support device according to claim 5, wherein the acquisition processing unit sets the correct answer value of the target speaker's emotion based on at least one of information indicating whether the conversation was successful or not input after the conversation ended and information indicating whether another conversation was held between the same other speaker and the target speaker within a predetermined period after the conversation ended.

7. A conversation support method for supporting a target speaker's speech when the target speaker has a conversation with a partner speaker by using an information processing device, comprising: obtaining information indicative of a type of subject of the conversation; extracting emotion sequence features from the speech of the other speaker in the conversation; a step of having relevance information that defines a relevance between a feature of an emotion sequence when the other speaker speaks and the emotion of the target speaker when the target speaker responds following the utterance of the other speaker for each type of subject of the conversation, and estimating a recommended emotion to be recommended to the target speaker based on the relevance information when the information representing the type of subject of the conversation and the feature of the emotion sequence when the other speaker speaks are input; an output step of outputting information representing the estimated recommended emotion to the target speaker; A conversation support method comprising:

8. The conversation support method according to claim 7, wherein the output process selects a candidate utterance content corresponding to the information expressing the recommended emotion in addition to or instead of the estimated information expressing the recommended emotion, and outputs the selected candidate utterance content to the target speaker.

9. 7. A program for causing a processor included in the conversation support device to execute at least one of the processes performed by first to fifth processing units included in the conversation support device according to claim 1.

Images