Information processing system, information processing device, information processing method, and program
The information processing system addresses the challenge of emotional analysis in call centers by extracting emotional values and word frequencies from voice data to understand conversation content and context, enhancing operator performance.
Patent Information
- Application Number
- JP2023091546
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2023-06-02
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2043-06-02
AI Technical Summary
Existing call center technologies fail to accurately perform emotional analysis by understanding conversation content and context from text data based on voice data, necessitating operators to adjust intonation and speaking style without proper emotional analysis.
An information processing system that extracts emotional values and frequently occurring words from voice data, performs voice recognition, and estimates conversation patterns using emotional values and word frequencies to analyze emotions and context.
Enables accurate emotional analysis by understanding conversation content and context from text data, improving operator skills and reducing stress by providing feedback on appropriate intonation and speaking style.
Smart Images

Figure 0007584567000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing system, an information processing device, an information processing method, and a program. [Background technology]
[0002] In call center operations, in addition to improving customer satisfaction, it is desirable to improve the skills of operators and reduce operator stress. For example, Patent Document 1 discloses an information provision system that includes an analysis unit that uses voice data of a call between a customer and an operator that has been quantified based on emotions to classify the call into one of the evaluation indices of the customer's emotions, and a provision unit that provides the results of the classification. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2022-154230 A Summary of the Invention [Problem to be solved by the invention]
[0004] In call center operations, operators usually need to change their intonation, words, and other speaking styles appropriately depending on the content of the conversation and the customer. For this reason, it is necessary to understand the content and context of the conversation from text data based on the audio data of the conversation, and then perform sentiment analysis. However, the technology described in Patent Document 1 is based on the premise that text data based on voice data is not used to correctly evaluate customer emotions. Therefore, the operator needs to appropriately change the way of speaking, such as intonation and words, depending on the content of the conversation and the customer. For this reason, there was a problem that it was not possible to analyze emotions by understanding the content and context of the conversation from text data based on the voice data of the conversation.
[0005] One aspect of the present invention has been made in consideration of the above points, and has an objective of providing an information processing system, an information processing device, an information processing method, and a program that can understand the content and context of a conversation from text data based on voice data and then perform emotion analysis. [Means for solving the problem]
[0006] The present invention has been made to solve the above-mentioned problems, and one aspect of the present invention is an information processing system comprising: a voice data acquisition unit that acquires voice data; an extraction unit that extracts, for each speaker, from the voice data using voice recognition, a plurality of emotional values that represent the emotions of the speaker in the voice data and text data that represents the content of the speech by the speaker; a frequent word extraction unit that extracts one or more frequent words included in the text data; an estimation unit that estimates which of a plurality of types the speaker belongs to based on the frequent words and the emotional values; and an output unit that outputs a score of the estimated pattern or the frequent words.
[0007] Another aspect of the present invention is an information processing device that includes: a voice data acquisition unit that acquires voice data; an extraction unit that extracts, for each speaker, from the voice data using voice recognition, a plurality of emotional values that represent the emotions of the speaker in the voice data and text data that represents the content of the speech by the speaker; a frequent word extraction unit that extracts one or more frequent words included in the text data; an estimation unit that estimates which of a plurality of types of pattern the speaker belongs to based on the frequent words and the emotional values; and an output unit that outputs a score of the estimated pattern or the frequent words.
[0008] Moreover, one aspect of the present invention is an information processing method to be executed by a computer of an information processing device, the information processing method having: a voice data acquisition step of acquiring voice data; an extraction step of extracting, for each speaker, from the voice data by voice recognition, a plurality of emotional values representing the emotions of the speaker in the voice data and text data representing the content of the speech by the speaker; a frequent word extraction step of extracting one or more frequent words included in the text data; an estimation step of estimating which of a plurality of types of patterns the speaker belongs to based on the frequent words and the emotional values; and an output step of outputting a score of the estimated pattern or the frequent words.
[0009] Another aspect of the present invention is a program for causing a computer of an information processing device to execute the following steps: a voice data acquisition step of acquiring voice data; an extraction step of extracting, for each speaker, from the voice data using voice recognition, a plurality of emotional values that represent the emotions of the speaker in the voice data and text data that represents the content of the speech by the speaker; a frequent word extraction step of extracting one or more frequent words included in the text data; an estimation step of estimating which of a plurality of types of patterns the speaker belongs to based on the frequent words and the emotional values; and an output step of outputting a score of the estimated pattern or the frequent words. Effect of the Invention
[0010] According to the present invention, it is possible to perform emotion analysis after understanding the conversation content and context from text data based on voice data. [Brief description of the drawings]
[0011] [Figure 1] 1 is a system configuration diagram showing an example of a configuration of an information processing system according to a first embodiment of the present invention. [Diagram 2] 1 is a block diagram showing an example of a hardware configuration of an information processing device according to an embodiment of the present invention. [Diagram 3]2 is a block diagram showing an example of a functional configuration of the information processing device according to the present embodiment. FIG. [Figure 4] 4A to 4C are explanatory diagrams showing an example of an emotion-word matrix, a weight matrix, and a pattern matrix according to the embodiment. [Diagram 5] 4 is a diagram showing an example of text data stored in a storage unit of the information processing device according to the embodiment; FIG. [Figure 6] 5 is a diagram showing an example of integrated data stored in a storage unit of the information processing device according to the embodiment; FIG. [Figure 7] FIG. 4 is a diagram showing an example of an output display in the information processing device according to the embodiment. [Figure 8] 5 is a flowchart illustrating an example of information processing in the information processing device according to the embodiment. [Figure 9] FIG. 11 is a block diagram showing an example of a functional configuration of an information processing device according to a second embodiment of the present invention. [Figure 10] 5 is a diagram showing an example of learning data used by a pattern learning unit in the information processing device according to the embodiment; FIG. [Figure 11] FIG. 4 is a diagram showing an example of an output display in the information processing device according to the embodiment. [Figure 12] 5 is a flowchart illustrating an example of information processing in the information processing device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] [First embodiment] Hereinafter, embodiments of the present invention will be described with reference to the drawings. <Configuration of information processing system SYS> First, the configuration of the information processing system SYS will be described.
[0013] FIG. 1 is a system configuration diagram showing an example of the configuration of an information processing system SYS according to the first embodiment of the present invention. The information processing system SYS includes an information processing device 100 and a call terminal device 200. The information processing device 100 and the call terminal device 200 are connected to each other via a network NW so as to be able to communicate with each other.
[0014] The information processing system SYS will now be described in more detail.
[0015] The information processing system SYS is a system that extracts multiple emotion values representing multiple emotions of multiple people (e.g., customers and operators) from voice data in a call between a customer and an operator at a call center, analyzes the conversation content (call content) and context by extracting frequently occurring words and the like from text data obtained by voice recognition of the voice data, and infers which of multiple types of patterns the conversation and speaker (e.g., operator) fall into based on the emotion values and frequently occurring words, and outputs the inferred result. Here, the frequent words include single words and phrases consisting of multiple words, but the frequent words may be either single words or phrases consisting of multiple words.
[0016] More specifically, the information processing system SYS acquires voice data, and extracts from the voice data, for each speaker, a plurality of emotion values that represent the emotion of the speaker in the voice data, and text data that represents the contents of the speaker's speech, by voice recognition. The information processing system SYS also extracts one or more frequently occurring words included in the text data, estimates which of a plurality of types the speaker belongs to based on the frequently occurring words and the emotion values, and outputs the estimated pattern.
[0017] With this configuration, the information processing system SYS can grasp the context of a conversation from frequently occurring words extracted from text data based on voice data and estimate patterns according to the frequently occurring words and emotional values, thereby enabling emotion analysis according to the conversation.
[0018] The information processing device 100 acquires voice data of a call and extracts from the voice data a number of emotion values representing the emotions of each speaker (e.g., the operator and the customer) during a predetermined period, e.g., one minute from the start of the conversation. The emotions are, for example, joy, anger, sadness, happiness, empathy, kindness, sincerity, and other emotions felt by the customer or the operator during the call. The emotion values are numerical values indicating the degree of the above-mentioned emotions.
[0019] The emotions are not limited to these, and any emotion may be used as long as it is an emotion felt by a customer or an operator. The predetermined period for extracting the emotion value of a conversation from the voice data may be a period going back a certain period from the end of the conversation, or may be a period starting from when a specific keyword is uttered in the conversation.
[0020] Furthermore, the information processing device 100 extracts text data of the conversation from the voice data by voice recognition. The text data is text data corresponding to the voice data from which the emotion value was extracted. The information processing device 100 grasps the content and context of a conversation by extracting frequently occurring words from text data. The information processing device 100 estimates which of a plurality of patterns the conversation falls into by using the frequently occurring words and emotion values. The information processing device 100 outputs the estimation result.
[0021] The call terminal device 200 is a terminal device used for calling. The call terminal device 200 may be a telephone, a smartphone, a calling application, a chat tool, or a recorder or a camera capable of recording voice data. In this embodiment, the call terminal device 200 is a terminal device used by an operator for calling a customer. The call terminal device 200 has a function of calling a customer and a function of recording the call.
[0022] In this embodiment, a call in a call center is taken as an example, but the present invention can be applied to any conversation between multiple people, such as face-to-face or online.
[0023] Next, the hardware configuration of the information processing device 100 will be described.
[0024] <Hardware configuration> FIG. 2 is a block diagram showing an example of a hardware configuration of the information processing device according to the present embodiment. The information processing device 100 includes a CPU 101, a storage medium interface unit 102, a storage medium 103, an input device 104, an output device 105, a ROM 106 (Read Only Memory), a RAM 107 (Random Access Memory), an auxiliary storage unit 108, and a network interface unit 109. The CPU 101, the storage medium interface unit 102, the input device 104, the output device 105, the ROM 106, the RAM 107, the auxiliary storage unit 108, and the network interface unit 109 are connected to each other via a bus.
[0025] Note that the CPU 101 referred to here refers to a processor in general, and includes not only a device called a CPU in the narrow sense, but also, for example, a GPU, a DSP, etc. Also, the CPU 101 referred to here is not limited to being realized by one processor, and may be realized by combining multiple processors of the same or different types.
[0026] <cpu101> The CPU 101 controls the information processing device 100 by reading and executing programs stored in the auxiliary storage unit 108, ROM 106, and RAM 107, reading various data stored in the auxiliary storage unit 108, ROM 106, and RAM 107, and writing the various data to the auxiliary storage unit 108 and RAM 107. The CPU 101 also reads various data stored in the storage medium 103 via the storage medium interface unit 102, and writes the various data to the storage medium 103.
[0027] <Storage medium 103> The storage medium 103 is a portable storage medium such as a magneto-optical disk, a flexible disk, or a flash memory, and stores various data.
[0028] <Storage medium interface unit 102> The storage medium interface unit 102 is an interface for reading and writing data from and to the storage medium 103 .
[0029] <Input device 104> The input device 104 is an input device such as a mouse, a keyboard, a touch panel, a microphone, a volume control button, a power button, a setting button, and an infrared receiver.
[0030] <Output Device 105> The output device 105 is an output device such as a display unit, a speaker, or the like.
[0031] <ROM106、RAM107> The ROM 106 and the RAM 107 store programs for operating each functional unit of the information processing device 100 and various data.
[0032] <Auxiliary storage section 108> The auxiliary storage unit 108 is a hard disk drive, a flash memory, or the like, and stores programs for operating each functional unit of the information processing device 100 and various data.
[0033] <Network Interface Unit 109> The network interface unit 109 has a communication interface and is connected to the network NW via wireless communication.
[0034] For example, the CPU 101 of the information processing device 100 corresponds to the control unit 15 in the functional configuration shown in Fig. 3. The ROM 106, RAM 107, auxiliary storage unit 108, or any combination thereof of the information processing device 100 corresponds to the storage unit 12 in the functional configuration shown in Fig. 3. The input device 104 and the output device 105 of the information processing device 100 correspond to the input unit 13 and the output unit 14 in the functional configuration shown in Fig. 3.
[0035] Although illustration and description of the hardware configuration of the call terminal device 200 will be omitted, the call terminal device 200 has the same hardware configuration as the information processing device 100 shown in FIG.
[0036] Next, the functional configuration of the information processing device 100 will be described.
[0037] <Functional configuration of information processing device 100> FIG. 3 is a block diagram showing an example of a functional configuration of the information processing device 100 according to this embodiment. The information processing device 100 includes a communication unit 11, a storage unit 12, an input unit 13, an output unit 14, and a control unit 15. The communication unit 11, the storage unit 12, the input unit 13, the output unit 14, and the control unit 15 are connected to each other via a bus.
[0038] <Communications Division 11> The communication unit 11 has a function of communicating with the communication terminal device 200. The communication unit 11 outputs various information received from the communication terminal device 200 to the control unit 15. In addition, the communication unit 11 transmits information input from the control unit 15 to the communication terminal device 200.
[0039] <Storage section 12> The memory unit 12 is configured with a storage medium, for example, a hard disk drive (HDD), a flash memory, an electrically erasable programmable read only memory (EEPROM), a random access read / write memory (RAM), a read only memory (ROM), or any combination of these storage media. The storage unit 12 may be, for example, a non-volatile memory.
[0040] The storage unit 12 stores text data 121 , integrated data 122 , and pattern data 123 . The text data 121 is text data generated by speech recognition of the voice data. Here, the text data 121 is data associated with an emotion value extracted from the voice data. The integrated data 122 is data in which frequently occurring words extracted from text data, emotion values of conversations in voice data corresponding to the text data, and call types are associated with each other. The call type is attribute information that identifies the type of call, such as an inbound call (received to a call center), an outbound call (made from a call center), or the content of an inquiry. The pattern data 123 is data of a plurality of model patterns in which frequently occurring words are associated with emotion values.
[0041] <Input section 13> The input unit 13 is an input device such as a mouse, a keyboard, a microphone, etc. that is connected to the information processing device 100. The input unit 13 accepts an operation input that is input from the outside. The input unit 13 outputs an operation signal to the control unit 15 according to the operation input.
[0042] <Output section 14> The output unit 14 is an output device such as a display device, etc. The output unit 14 outputs the presentation information output from the control unit 15 to the output device or another device such as the communication terminal device 200.
[0043] <Control unit 15> The control unit 15 has a function of controlling the information processing device 100. The control unit 15 reads out various data, applications, programs, etc. stored in the storage unit 12 and controls the information processing device 100.
[0044] The processing of the control unit 15 will now be described in more detail. The control unit 15 includes a voice data acquisition unit 151 , a text data extraction unit 152 , an emotion / text integration unit 153 , a pattern estimation unit 154 , and a pattern output unit 155 .
[0045] <Audio data acquisition unit 151> The voice data acquisition unit 151 acquires voice data from the communication terminal device 200 via the network NW and the communication unit 11. The voice data acquisition unit 151 outputs the voice data to the text data extraction unit 152.
[0046] <Text data extraction unit 152> When voice data is input from voice data acquisition unit 151, text data extraction unit 152 extracts text data by voice recognition of the voice data. The text data is text data corresponding to voice data for a predetermined period of time, for example, one minute, from the start of the call. The text data is also text data in which speakers and the contents of each speaker's speech are converted into text by speaker recognition. Text data extraction unit 152 stores the extracted text data in storage unit 12 in association with identification information that identifies the call. Furthermore, the text data extraction unit 152 extracts, for each utterance, a plurality of emotion values of the speaker, for example, the customer and the operator, during a predetermined period from the start of the call from the voice data, for example, by using a known technique. The text data extraction unit 152 stores the extracted emotion values in the storage unit 12 in association with the text data. Here, speaker recognition is performed by recognizing the voices of multiple speakers using existing voice separation technology or the like.
[0047] The emotion value may be a statistical quantity such as a mean value, a median value, or a variance value obtained by averaging each emotion value over a predetermined period of time. The predetermined period may be a period until the end of a conversation, a period going back a certain period from the end of a conversation, or a period of a certain amount of time that has elapsed since a specific keyword was uttered in a conversation.
[0048] <Emotion and text integration section 153> The emotion / text integrator 153 reads out the text data 121 from the storage unit 12, and integrates the emotion value and the text data for each utterance. Specifically, the emotion / text integrator 153 extracts words (frequent words) and phrases for each utterance of the text data, and quantifies the importance of each of the extracted words and phrases using, for example, TF-IDF (Term Frequency - Inverse Document Frequency). The emotion / text integrator 153 associates the emotion value, the frequent words, and a score such as the importance of each word or phrase for each utterance, and stores them in the storage unit 12 as integrated data 122.
[0049] <Pattern Estimation Unit 154> The pattern estimation unit 154 reads the integrated data 122 from the storage unit 12 and extracts patterns of emotions and words. Specifically, the pattern estimation unit 154 extracts emotion values and word scores of the integrated data 122 and defines them as an emotion-word matrix. The pattern estimation unit 154 decomposes the emotion-word matrix into a weight matrix and a pattern matrix. For the decomposition into the weight matrix and the pattern matrix, for example, singular value decomposition, non-negative matrix factorization, neural network, etc. are used. The pattern estimation unit 154 associates the emotion-word matrix, weight matrix, and pattern matrix and stores them in the storage unit 12 as pattern estimation results.
[0050] In this embodiment, for example, non-negative matrix factorization is used for decomposition into a weight matrix and a pattern matrix. In this way, all the values of the matrices decomposed into the weight matrix and the pattern matrix are non-negative, so that the pattern matrix and the weight matrix obtained as the decomposition results can be intuitively understood.
[0051] In addition, the pattern estimation unit 154 may extract emotion and word patterns using not only a linear algorithm such as matrix decomposition, but also a nonlinear algorithm such as a k-nearest neighbor method, UMAP (Uniform Manifold Approximation and Projection), or t-SNE (t-distributed stochastic neighbor embedding), or a combination thereof.
[0052] The following description will be given with reference to FIG.
[0053] FIG. 4 is an explanatory diagram of an emotion-word matrix, a weight matrix, and a pattern matrix according to this embodiment. The illustrated example is an example in which there is one type each of operator emotion value (OP emotion value), customer emotion value (CU emotion value), and word, and the number of patterns is three. If the number of calls is N, then the (N × 3) emotion-word matrix is decomposed into an (N × 3) weight matrix and a (3 × 3) pattern matrix. The weight matrix indicates the weight of each pattern in each call, and also indicates which emotions and words have a strong relationship in each conversation. Each column of the pattern matrix indicates the relative strength of the relationship between emotions and words through its weight, and each row value of the pattern matrix represents a common pattern of words and emotions in N conversations.
[0054] The size of the matrix is not limited to this, and can be changed according to the number of emotion values that can be obtained, and the types and number of words. Also, numerical values other than the word scores and emotion values, such as call types, may be used.
[0055] <Pattern output section 155> The pattern output unit 155 reads out the pattern estimation result from the storage unit 12, and generates a display image including emotions corresponding to the emotion values using the emotion-word matrix, words corresponding to the word scores, weights in the weight matrix, patterns in the pattern matrix, and pattern data based on past cases. The pattern output unit 155 outputs the generated display image to an output device or the call terminal device 200.
[0056] The pattern output unit 155 may output both each pattern of the pattern matrix and the word corresponding to the word score, that is, the score of the frequent word, or may output either one of them.
[0057] Next, various data stored in the storage unit 12 of the information processing device 100 will be described.
[0058] FIG. 5 is a diagram showing an example of the text data 121 stored in the storage unit 12 of the information processing device 100 according to this embodiment. The text data shown in the figure is data in which a call identification number, a start time, an end time, a speaker, the content of the speech, an emotion value (A), an emotion value (B), and an emotion value (C) are associated with each other. The call identification number is identification information (number) that identifies a call. The start time and end time are time information indicating the start time and end time of an utterance, and are obtained, for example, from time information included in the voice data. The speaker represents a speaker whose voice has been separated by speaker recognition. Note that if the speaker can be distinguished, such as speaker A, speaker B, etc., it is not necessary to distinguish the type of person, such as "operator" or "customer." The speech content is text data representing the content of an utterance by a speaker. The text data is generated by speech recognition. Emotion value (A), emotion value (B), and emotion value (C) are numerical values that represent each of the emotions that have been set in advance. For example, emotion value (A) is a numerical value that represents joy, emotion value (B) is a numerical value that represents anger, and emotion values corresponding to a plurality of emotions are output from the voice data for each utterance by known technology. As shown in the figure, the text data associates the content of an utterance for each speaker with a plurality of emotion values for each content of an utterance.
[0059] FIG. 6 is a diagram showing an example of the integrated data 122 stored in the storage unit 12 of the information processing device 100 according to this embodiment. The illustrated integrated data is data in which a call identification number, an operator word importance, a customer word importance, an operator emotion value, and a customer emotion value are associated with each other. The integrated data may be associated with a call type. The call identification number is identification information for identifying a call. The operator word importance indicates the score of a word extracted from the speech content of the operator. For example, the operator word importance indicates a predetermined number of words in descending order of word importance as a numerical value associated with the importance of each of the words. The customer word importance indicates the score of a word extracted from the customer's speech. For example, the customer word importance indicates a predetermined number of words (frequent words) in descending order of word importance, as a numerical value associated with the importance of each word. The operator emotion value and the customer emotion value are emotion values that indicate emotions extracted from the contents of the utterances of the operator and the customer.
[0060] Although an example of generating the integrated data by converting the emotion values into numerical values has been described, the integrated data may be generated by converting the emotion values into text.
[0061] Next, an example of output from the output unit 14 of the information processing device 100 will be described.
[0062] FIG. 7 is a diagram showing an example of an output display in the information processing device 100 according to this embodiment. The output display shown in the figure is an example of a case where there are three types of patterns and the display image generated by the pattern output section 155 is output to the output device. For example, assume that emotion values (A) of emotion A and emotion values (B) of emotion B of the operator and the customer are extracted from the voice data, the words UUU, VVV, and WWW are extracted as frequent words based on the voice data, and the word importance (word score) of the frequent words is extracted. The information processing device 100 generates an emotion-word matrix according to the emotion values and word scores, and decomposes the emotion-word matrix into a weight matrix and a pattern matrix. The displayed image includes an area for displaying the weights of emotion / word patterns, an area for listing emotion / word patterns, an area for pattern examples, and an area for overall evaluation.
[0063] The emotion / word pattern weight display area is an area that reads each column of the weight matrix and displays the weight of the pattern for each call. The characteristics of the call content are expressed by the strength (large or small) of the weight of each pattern. By using the pattern weights in this way, it is possible to distinguish and compare the tendencies of each call. In the example shown, it is displayed as "Pattern 1 weight 1.5," "Pattern 2 weight 0.1," and "Pattern 3 weight 0.5."
[0064] The emotion / word pattern list area is an area that reads the values of each row of the pattern matrix and displays the patterns in various graphs. The emotion / word pattern list area is also an area that displays a certain number of highly important words (words with large values in the weight matrix) as "frequent words". Instead of words, important phrases or phrases that are not allowed for the call center may be displayed.
[0065] The pattern example area is a surface area that displays the pattern data stored as pattern data in the memory unit 12. For example, the pattern data may hold the emotion and word patterns of highly skilled operators and the emotion and word patterns of less experienced and less skilled operators. The pattern output unit 155 searches the pattern data stored in the memory unit 12 and displays good patterns to be followed and bad patterns to be used as counterexamples. This allows operators to refer to customer emotion values from past cases to check how they should speak and what words they should use, for example. It is also possible to identify the contents of a call from words and compare them with the emotions of operators with high skills at that time.
[0066] Pattern data may also hold emotion and word patterns linked to other indicators related to call center and sales conversations. Indicators include, for example, operator turnover rate and customer satisfaction surveyed separately on social media, the web, or through questionnaires. By linking these indicators to emotion and word patterns, it is possible to grasp trends such as which emotion and word patterns improve the indicators.
[0067] The comprehensive evaluation area is an area that displays advice based on a comparison between the emotion and word patterns in a certain call and the patterns stored as pattern data. In the example shown, messages such as "Evaluation: In the current conversation content, Pattern 1 is closest. In the content of this call, it is close to an undesirable pattern," "Emotional advice: Try to lower the tone of your voice," and "Word advice: Talking to XXX will improve customer satisfaction" are displayed. The conversation content is estimated based on frequently used words and emotional values, and an evaluation of whether the conversation is conducted with appropriate emotions and whether appropriate words are being used, advice for improving emotions, and advice such as words that should be used to improve customer impressions such as satisfaction.
[0068] Next, the flow of information processing according to this embodiment will be described.
[0069] FIG. 8 is a flowchart showing an example of information processing in the information processing device 100 according to this embodiment. In step S101, the information processing device 100 acquires voice data. Then, the information processing device 100 executes the process of step S102. In step S102, the information processing device 100 extracts text data from the voice data by voice recognition. Then, the information processing device 100 executes the process of step S103.
[0070] In step S103, information processing device 100 extracts a plurality of emotion values for each conversation from the voice data. After that, information processing device 100 executes the process of step S104. In step S104, the information processing device 100 generates integrated data by integrating the text data and the emotion value. Specifically, the information processing device 100 calculates the word importance for each conversation, and generates the integrated data by associating a predetermined number of words with high word importance with the word importance and the emotion value extracted from the voice data of the conversation. After that, the information processing device 100 executes the process of step S105.
[0071] In step S105, the information processing device 100 reads the integrated data and generates an emotion-word matrix. The information processing device 100 also decomposes the emotion-word matrix into a weight matrix and a pattern matrix to estimate a conversation pattern. After that, the information processing device 100 executes the process of step S106.
[0072] Here, estimating a conversation pattern includes indicating weights for multiple patterns, such as "weight 1.5 for pattern 1," "weight 0.1 for pattern 2," and "weight 0.5 for pattern 3," as described above. Estimating a conversation pattern also includes selecting the weight with the largest weight value from the weights of multiple patterns, and estimating, for example, that it is pattern 1 or that pattern 1 is strong based on the selected weight. Furthermore, estimating a conversation pattern also includes setting a threshold value for the weights of multiple patterns, and estimating a tendency from patterns with weights equal to or greater than the threshold value, such as when the threshold value is 0.3 in the above example, pattern 1 is strong, followed by pattern 3.
[0073] In step S106, the information processing device 100 outputs the estimation result. Then, the information processing device 100 ends the information processing related to FIG.
[0074] As described above, the information processing system SYS according to this embodiment includes a voice data acquisition unit 151 that acquires voice data, an extraction unit (text data extraction unit 152) that extracts, for each speaker, from the voice data by voice recognition a plurality of emotion values that represent the emotions of the speaker in the voice data and text data that represents the content of the utterance by the speaker, a frequent word extraction unit (emotion / text integrating unit 153) that extracts one or more frequent words included in the text data and further integrates the text data and the emotion values into a numerical value or text format to generate integrated data, an estimation unit (pattern estimation unit 154) that estimates which of a plurality of types of pattern the utterance belongs to based on the frequent words and the emotion values, and an output unit (pattern output unit 155) that outputs the estimated pattern.
[0075] In this way, emotion analysis can be performed after grasping the conversation content and context from text data based on voice data. In addition, by extracting and displaying patterns simultaneously with words and phrases, in addition to emotion values, the speaker's skill according to the content of the utterance can be visualized. In addition, since it is possible to provide feedback on model emotions and speaking tendencies, it is possible to improve the speaker's skill according to the content of the utterance.
[0076] In the above information processing system SYS, the emotion value is a statistical quantity such as the average value of each emotion value over a predetermined period from the start of a conversation, and the estimation unit (pattern estimation unit 154) estimates which of multiple patterns the speaker belongs to using the emotion value, a numerical value representing frequently occurring words, and weights for each of the multiple types of patterns.
[0077] In this way, it is possible to use statistics such as emotion values and average emotion values over a predetermined period from the start of a conversation, which are likely to reveal characteristics of the conversation content, thereby improving the accuracy of estimating a pattern suited to the conversation content.
[0078] In the above information processing system SYS, the output section (pattern output section 155) outputs patterns and frequent words.
[0079] By doing this, you can check what words and emotions you should use in your conversation.
[0080] In the above information processing system SYS, the voice data is telephone call voice data, and the output unit (pattern output unit 155) matches a speaker with a telephone call partner of the speaker according to emotions and word patterns. Specifically, for an operator, if the operator is similar (approximate) to, for example, "Pattern 1 weight 1.5," "Pattern 2 weight 0.1," and "Pattern 3 weight 0.5," the pattern is considered to be friendly, and if the operator is similar to, for example, "Pattern 1 weight 1.0," "Pattern 2 weight 1.0," and "Pattern 3 weight 1.0," the pattern is considered to be calm. In addition, with regard to customers, if the pattern is similar to, for example, "Pattern 1 weight 1.0," "Pattern 2 weight 1.0," or "Pattern 3 weight 1.0," it is considered to be a sociable pattern, and if the pattern is similar to, for example, "Pattern 1 weight 1.5," "Pattern 2 weight 1.0," or "Pattern 3 weight 1.0," it is considered to be a critical pattern.
[0081] In this case, matching patterns are previously defined in the information processing system SYS such that a friendly pattern operator is matched to a sociable pattern customer, and a calm pattern operator is matched to a critical pattern customer. As a result, the output unit (pattern output unit 155) determines a pattern for the personality of the customer and the agent, and matches the customer with the agent according to the matching pattern.
[0082] The information processing device 100 also includes a voice data acquisition unit 151 that acquires voice data, an extraction unit (text data extraction unit 152) that extracts, for each speaker, from the voice data by voice recognition, a plurality of emotion values that represent the emotions of the speaker in the voice data and text data that represents the content of the speaker's speech, a frequent word extraction unit (emotion / text integration unit 153) that extracts one or more frequent words included in the text data, an estimation unit (pattern estimation unit 154) that estimates which of a plurality of types of patterns the speaker belongs to based on the frequent words and the emotion values, and an output unit (pattern output unit 155) that outputs the estimated pattern.
[0083] In this way, emotion analysis can be performed after grasping the conversation content and context from text data based on voice data. In addition, by extracting and displaying patterns simultaneously with words and phrases, in addition to emotion values, the speaker's skill according to the content of the utterance can be visualized. In addition, since it is possible to provide feedback on model emotions and speaking tendencies, it is possible to improve the speaker's skill according to the content of the utterance.
[0084] [Second embodiment] In the second embodiment, an example of learning an index related to a conversation by supervised learning will be described. Here, in the second embodiment, the differences from the first embodiment will be mainly described, and the other parts will be referred to the first embodiment and the description thereof will be omitted.
[0085] FIG. 9 is a block diagram showing an example of a functional configuration of an information processing device 100 according to the second embodiment of the present invention. The information processing device 100 includes a communication unit 11, a storage unit 12, an input unit 13, an output unit 14, and a control unit 15. The communication unit 11, the storage unit 12, the input unit 13, the output unit 14, and the control unit 15 are connected to each other via a bus.
[0086] <Storage section 12> The storage unit 12 stores text data 121, integrated data 122, pattern data 123, index data 124, and a learning model 125. The index data 124 is, for example, data on indexes to be checked by a QA (Quality Administrator) of a call center, index data related to management, index data to be improved by analyzing emotions and words, and related data thereof.
[0087] Examples of indicator data that QA should check include the following: · Response Time: The time it takes for an operator to respond to a customer inquiry. · Average Handling Time: The average time spent on a call with a customer. First Call Resolution Rate: The percentage of customers who reach a resolution on their first contact. ·Customer Satisfaction: An indicator that measures how satisfied a customer is with the operator's response. Operator Compliance: An indicator that evaluates whether an operator complies with corporate and industry rules, legal compliance, and privacy protection. ·Operator Skills and Knowledge: An indicator that evaluates whether the operator has appropriate knowledge of products and services and response skills. Call Quality: Indicators for evaluating the quality of a call, such as how easy it is to hear the operator's voice, how clear it is, what expressions and phrases should be spoken, what expressions and phrases should not be spoken, and the timing of speech. Escalation Rate: The percentage of cases where an operator escalates a case to a higher level person or supervisor.
[0088] In addition, the management indicators include, for example, the following indicator data: ·Cost Performance: An indicator of the balance between operational costs and customer satisfaction. Operator Efficiency: An indicator to evaluate whether operators are performing their duties efficiently, including call time, waiting time, and time to resolution. ·Operator Retention Rate: An indicator of the percentage of operators who work at a call center for a long period of time.
[0089] Also, for example, it may be an indicator related to digital marketing collected from the web, social networking sites, etc. · Conversion Rate: An indicator that shows the percentage of visitors who actually make a purchase or make an inquiry as a result of marketing activities. Bounce Rate: An indicator of the percentage of visitors who leave the site shortly after accessing it. Click-Through Rate: An indicator that shows the percentage of people who clicked on content such as an advertisement or email newsletter. · Social Share Rate: An indicator showing the percentage of people who shared content on social media etc. · Repeat Rate: An indicator showing the percentage of people who have previously visited a site who return. Net Promoter Score (NPS): A metric that measures how likely a customer is to recommend a brand or service to others.
[0090] The learning model 125 is a learning model generated by the pattern learning unit 156 .
[0091] <Control unit 15> The control unit 15 includes a voice data acquisition unit 151 , a text data extraction unit 152 , an emotion / text integration unit 153 , a pattern estimation unit 154 , a pattern output unit 155 , and a pattern learning unit 156 .
[0092] <Pattern Learning Section 156> Pattern learning unit 156 sets the index data stored in storage unit 12 as a target variable, sets the text data including emotion values stored in storage unit 12 as an explanatory variable, and generates a learning model by learning the correspondence relationship by machine learning including deep learning, etc. In this way, by inputting the text data including emotion values into the learning model, various indices are output.
[0093] The pattern learning unit 156 may generate a learning model according to the format of the learning data. For example, in the case of quantified learning data, a machine learning model that is strong for table data such as Lightgbm may be used, and in the case of textual learning data, a language model such as BERT may be used.
[0094] <Pattern Estimation Unit 154> The pattern estimation unit 154 utilizes the learning model stored in the memory unit 12, inputs text data including emotion values as explanatory variables to the learning model, and obtains the index set as the objective variable as output, thereby estimating (predicting) the index.
[0095] <Pattern output section 155> The pattern output unit 155 generates a display image showing the relationship between the predicted index, the content of the conversation, and the emotion value.
[0096] Fig. 10 is a diagram showing an example of learning data used by the pattern learning unit 156 in the information processing device 100 according to this embodiment. Also, Fig. 11 is a diagram showing an example of an output display in the information processing device 100 according to this embodiment. The learning data in this embodiment is, for example, text data including emotion values stored in the storage unit 12. The learning data is data in which a call identification number, text data, an operator emotion value, and a customer emotion value are associated with each other. Furthermore, as the learning data, text data associated with the emotion values shown in FIG. 10 and index data corresponding to the text data are used. Note that, instead of the text data including the emotion values shown in Fig. 10, the text data including the emotion values shown in Fig. 5 may be used as the learning data. For example, the emotion values are converted into text, such as "emotion A=10, emotion B=20, emotion C=0", and combined with the text data of the speech recognition result to be used as the learning data.
[0097] In addition, in the learning data shown in FIG. 10, learning may be performed using learning data in which index data stored in the storage unit 12 is associated with the operator emotion value and the customer emotion value instead of or in addition to the operator emotion value and the customer emotion value.
[0098] For example, when text data including the emotion values shown in FIG. 10 are input as explanatory variables to the learning model trained in this way, indicators A and B corresponding to the call identification number, text data, operator emotion value, and customer emotion value in FIG. 10 are output as objective variables, as shown in FIG. 11. The learning model may also output which words or emotion values were focused on when predicting the index. For example, a deep learning model with an attention mechanism is used as the learning model, and the output value of the intermediate layer of the learning model can be used to output which words or emotion values were focused on when predicting the index. Here, the output display shown in FIG. 11 in the above description is an example of a display image generated by the pattern output section 155 based on the index obtained as an output from the learning model. Note that which words or emotion values were focused on when estimating the index may be displayed in an easily understandable manner to the user by changing the display mode such as text color or background color. In this way, it is possible to easily visualize which words or emotion values contributed to the index prediction.
[0099] Next, information processing according to this embodiment will be described.
[0100] FIG. 12 is a flowchart showing an example of information processing in the information processing device 100 according to this embodiment. In step S201, the information processing device 100 acquires voice data. Next, the information processing device 100 executes the process of step S202. In step S202, the information processing device 100 extracts text data from the voice data by voice recognition. Then, the information processing device 100 executes the process of step S203.
[0101] In step S203, information processing device 100 extracts a plurality of emotion values for each conversation from the voice data. After that, information processing device 100 executes the process of step S204. In step S204, the information processing device 100 generates integrated data by integrating the text data and the emotion value. Specifically, the information processing device 100 calculates word importance for each conversation, and generates integrated data by associating a predetermined number of words with high word importance, the word importance, and the emotion value extracted from the voice data of the conversation. Alternatively, the information processing device 100 may generate integrated data by converting the emotion value into text and combining the emotion value converted into text with the text data of the voice recognition result. After that, the information processing device 100 executes the process of step S205.
[0102] In step S205, when information processing device 100 receives text data including emotion values, it generates a learning model that outputs an index (index value). After that, information processing device 100 executes the process of step S206. In step S206, the information processing device 100 reads out the learning model 125 stored in the storage unit 12, and inputs text data including emotion values to the learning model 125 to obtain an index as an output. The information processing device 100 estimates the output index as a pattern. After that, the information processing device 100 executes the process of step S207.
[0103] In step S207, the information processing device 100 outputs the estimation result. Then, the information processing device 100 ends the information processing related to FIG.
[0104] As described above, the information processing system SYS according to this embodiment includes a voice data acquisition unit 151 that acquires voice data, an extraction unit (text data extraction unit 152) that extracts, for each speaker, from the voice data by voice recognition a plurality of emotion values that represent the emotions of the speaker in the voice data and text data that represents the content of the speech by the speaker, a frequent word extraction unit (emotion / text integrating unit 153) that extracts one or more frequent words included in the text data and further integrates the text data and the emotion values into a numerical value or text format to generate integrated data, a pattern learning unit 156 that machine-learns a learning model based on the learning data, an estimation unit (pattern estimation unit 154) that predicts an index using the learning model with text and emotion values as input, and an output unit (pattern output unit 155) that outputs the estimated index.
[0105] In this way, emotion analysis can be performed after grasping the conversation content and context from text data based on voice data. In addition, by simultaneously learning not only emotion values but also words and phrases through machine learning and predicting and displaying various indices, the speaker's skill according to the content of the utterance can be visualized. In addition, by displaying the emotion values and words that the learning model focused on when predicting the indices, it is possible to provide feedback on the emotions and speaking tendencies that should be modeled, thereby improving the speaker's skill according to the content of the utterance.
[0106] In the above information processing system SYS, the output unit (pattern output unit 155) displays the predicted index. Also, by displaying the emotion values and words that were focused on when predicting the index, it is possible to provide feedback on the emotion and speech tendency to be modeled, thereby improving the speaker's skill according to the content of the speech.
[0107] In the above information processing system SYS, the voice data is telephone call voice data, and the output unit (pattern output unit 155) matches a speaker with a call partner of the speaker according to the pattern of the predicted index. Specifically, for example, when an operator is similar to "index 1 value 1.5" and "index 2 value 0.1", the pattern is friendly, and when it is similar to "index 1 value 1.0" and "index 2 value 1.0", the pattern is calm. For example, when a customer is similar to "index 3 value 1.0" and "index 4 value 1.0", the pattern is sociable, and when it is similar to "index 3 value 1.5" and "index 4 value 1.0", the pattern is critical. A matching pattern is previously determined in the information processing system SYS such that a friendly pattern operator is matched to a sociable pattern customer, and a calm pattern operator is matched to a critical pattern customer. The output unit (pattern output unit 155) determines the pattern for the personality of the customer and the operator, and matches the customer and the operator according to the matching pattern.
[0108] In this way, it is possible to match a speaker suitable for the call partner according to the pattern.
[0109] The information processing system SYS further includes a prediction unit (not shown) that predicts an evaluation value by inputting the emotional value and text data into a learning model generated by learning the correspondence between text data, an emotional value corresponding to the text data, and an evaluation value for a conversation corresponding to the text data.
[0110] Since the system can output an evaluation value along with the pattern, it is possible to visualize the speaker's skill according to the content of the utterance. In addition, it is possible to provide feedback on the emotions and speaking tendencies that should be modeled, which makes it possible to improve the speaker's skill according to the content of the utterance.
[0111] The information processing device 100 also includes a voice data acquisition unit 151 that acquires voice data, an extraction unit (text data extraction unit 152) that extracts, for each speaker, from the voice data by voice recognition, a plurality of emotion values that indicate the emotions of the speaker in the voice data and text data that indicates the content of the speech by the speaker, an integration unit that integrates the text data and the emotion values into a numerical value or text format, a frequent word extraction unit (emotion / text integration unit 153) that integrates the text data and the emotion values into a numerical value or text format to generate integrated data in the process of extracting one or more frequent words included in the text data, a pattern learning unit 156 that machine-learns a learning model based on the learning data, an estimation unit (pattern estimation unit 154) that predicts an index using the learning model with text and emotion values as input, and an output unit (pattern output unit 155) that outputs the estimated index.
[0112] In this way, sentiment analysis can be performed after understanding the content and context of the conversation from text data based on the voice data. In addition, by simultaneously learning not only emotional values but also words and phrases through machine learning and predicting and displaying various indicators, it is possible to visualize the speaker's skill according to the content of the utterance. In addition, by displaying the emotional values and words that the learning model focused on when predicting the indicators, it is possible to provide feedback on the emotions and speaking tendencies that should be modeled, thereby improving the speaker's skill according to the content of the utterance.
[0113] Although each embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design changes, etc. are possible within the scope that does not deviate from the gist of the present invention.
[0114] For example, in each of the above-described embodiments, an example has been described in which the information processing device 100 and the call terminal device 200 are used as respective devices. However, one aspect of the present invention may also be realized by a device that combines some or all of these devices, or a device in which some of these devices are rearranged.
[0115] In addition, the program that operates in the information processing device 100 and the call terminal device 200 in one aspect of the present invention may be a program that controls one or more processors such as a CPU (Central Processing Unit) so as to realize the functions shown in each of the above-mentioned embodiments and modifications related to one aspect of the present invention (a program that makes a computer function). The computer here also includes a quantum computer. The information handled by each of these devices is temporarily stored in a RAM (Random Access Memory) during processing, and then stored in various storage devices such as a flash memory and an HDD (Hard Disk Drive), and may be read, modified, and written by the CPU or the like as necessary.
[0116] In addition, a part or the whole of the information processing device 100 and the communication terminal device 200 in each of the above-mentioned embodiments and modifications may be realized by a computer having one or more processors. In that case, the control function may be realized by recording a program for realizing the control function in a computer-readable recording medium, reading the program recorded in the recording medium into a computer system, and executing the program.
[0117] The "computer system" here refers to a computer system built into the information processing device 100 and the communication terminal device 200, and includes hardware such as an OS and peripheral devices. Also, the "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into the computer system.
[0118] Furthermore, the term "computer-readable recording medium" may include a medium that dynamically stores a program for a short period of time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, and a medium that stores a program for a certain period of time, such as a volatile memory inside a computer system that serves as a server or client in such a case. The above program may be one that realizes part of the above-mentioned functions, or may be one that can realize the above-mentioned functions in combination with a program already recorded in the computer system.
[0119] In addition, a part or the whole of the information processing device 100 and the call terminal device 200 in each of the above-mentioned embodiments and modifications may be realized as an LSI, which is typically an integrated circuit, or may be realized as a chip set. In addition, each functional block of the information processing device 100 and the call terminal device 200 in each of the above-mentioned embodiments and modifications may be individually chipped, or may be integrated into a chip in part or in whole. The integrated circuit method is not limited to LSI, and may be realized by a dedicated circuit and / or a general-purpose processor. In addition, when an integrated circuit technology that replaces LSI appears due to the progress of semiconductor technology, it is also possible to use an integrated circuit based on that technology.
[0120] Although the embodiments and modifications have been described above in detail with reference to the drawings as one aspect of the present invention, the specific configuration is not limited to the embodiments and modifications, and includes design changes within the scope of the gist of the present invention. In addition, one aspect of the present invention can be modified in various ways within the scope of the claims, and embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. In addition, configurations in which elements described in the above embodiments and modifications are replaced with elements that have the same effect are also included. [Explanation of symbols]
[0121] SYS Information Processing System 100 Information processing device 101 CPU 102 Storage medium interface unit 103 Storage medium 104 Input Devices 105 Output Device 106 ROM 107 RAM 108 Auxiliary storage 109 Network Interface Unit 11 Communications Department 12 Storage section 121 Text Data 122 Integrated Data 123 Pattern Data 124 Index Data 125 Learning Model 13 Input section 14 Output section 15 Control section 151 Voice data acquisition unit 152 Text Data Extraction Unit 153 Emotion and Text Integration Department 154 Pattern Estimation Unit 155 Pattern Output Section 156 Pattern Learning Section 200 Telephone terminal device
Claims
1. a voice data acquisition unit that acquires voice data; an extracting unit that extracts, for each speaker, a plurality of emotion values that represent emotions of the speakers in the voice data and text data that represent the contents of the speech by the speakers from the voice data by voice recognition; a prediction unit that predicts the evaluation value by inputting the emotional value and the text data into a learning model that is generated based on a correspondence relationship between the text data, the emotional value corresponding to the text data, and an evaluation value for a conversation corresponding to the text data; An information processing system comprising:
2. a voice data acquisition unit that acquires voice data; an extracting unit that extracts, for each speaker, a plurality of emotion values that represent emotions of the speakers in the voice data and text data that represent the contents of the speech by the speakers from the voice data by voice recognition; a frequent word extraction unit that extracts one or more frequent words included in the text data; an estimation unit that estimates which of a plurality of types of patterns the speaker belongs to based on the frequent words and the emotion value; Equipped with the emotion value is any one of a statistic of evaluation values representing each emotion during a predetermined period from the start of a conversation, a statistic of evaluation values representing each emotion during a predetermined period going back a certain period from the end of the conversation, or a statistic of evaluation values representing each emotion during a predetermined period elapsed from the timing when a specific keyword was uttered in the conversation, the estimation unit estimates which of a plurality of patterns the speaker belongs to by using the emotion value, a numerical value representing the frequently occurring word, and a weight for each of a plurality of patterns; Information processing system.
3. a voice data acquisition unit that acquires voice data; an extracting unit that extracts, for each speaker, a plurality of emotion values that represent emotions of the speakers in the voice data and text data that represent the contents of the speech by the speakers from the voice data by voice recognition; a frequent word extraction unit that extracts one or more frequent words included in the text data; an estimation unit that estimates which of a plurality of types of patterns the speaker belongs to based on the frequent words and the emotion value; Equipped with the frequent word extraction unit integrates the text data and the emotion value corresponding to the text data as all numerical values or all text; Information processing system.
4. The voice data is call voice data, a matching unit that matches the speaker with a call partner of the speaker according to the pattern; Further comprising:
4. The information processing system according to claim 2 or 3.
5. a voice data acquisition unit that acquires voice data; an extracting unit that extracts, for each speaker, a plurality of emotion values that represent emotions of the speakers in the voice data and text data that represent the contents of the speech by the speakers from the voice data by voice recognition; a prediction unit that predicts the evaluation value by inputting the emotional value and the text data into a learning model that is generated based on a correspondence relationship between the text data, the emotional value corresponding to the text data, and an evaluation value for a conversation corresponding to the text data; An information processing device comprising:
6. a voice data acquisition unit that acquires voice data; an extracting unit that extracts, for each speaker, a plurality of emotion values that represent emotions of the speakers in the voice data and text data that represent the contents of the speech by the speakers from the voice data by voice recognition; a frequent word extraction unit that extracts one or more frequent words included in the text data; an estimation unit that estimates which of a plurality of types of patterns the speaker belongs to based on the frequent words and the emotion value; Equipped with the emotion value is any one of a statistic of evaluation values representing each emotion during a predetermined period from the start of a conversation, a statistic of evaluation values representing each emotion during a predetermined period going back a certain period from the end of the conversation, or a statistic of evaluation values representing each emotion during a predetermined period elapsed from the timing when a specific keyword was uttered in the conversation, the estimation unit estimates which of a plurality of patterns the speaker belongs to by using the emotion value, a numerical value representing the frequently occurring word, and a weight for each of a plurality of patterns; Information processing device.
7. a voice data acquisition unit that acquires voice data; an extracting unit that extracts, for each speaker, a plurality of emotion values that represent emotions of the speakers in the voice data and text data that represent the contents of the speech by the speakers from the voice data by voice recognition; a frequent word extraction unit that extracts one or more frequent words included in the text data; an estimation unit that estimates which of a plurality of types of patterns the speaker belongs to based on the frequent words and the emotion value; Equipped with the frequent word extraction unit integrates the text data and the emotion value corresponding to the text data as all numerical values or all text; Information processing device.
8. An information processing method to be executed by a computer of an information processing device, comprising: A voice data acquisition step of acquiring voice data; an extraction step of extracting, from the voice data by voice recognition, a plurality of emotion values representing emotions of speakers in the voice data and text data representing contents of utterances by the speakers; a prediction step of predicting the evaluation value by inputting the emotional value and the text data into a learning model generated based on a correspondence relationship between the text data, the emotional value corresponding to the text data, and an evaluation value for a conversation corresponding to the text data; An information processing method comprising the steps of:
9. An information processing method to be executed by a computer of an information processing device, comprising: A voice data acquisition step of acquiring voice data; an extraction step of extracting, from the voice data by voice recognition, a plurality of emotion values representing emotions of speakers in the voice data and text data representing contents of utterances by the speakers; a frequent word extraction step of extracting one or more frequent words included in the text data; an estimation step of estimating which of a plurality of types of patterns the speaker belongs to based on the frequent words and the emotion value; having the emotion value is any one of a statistic of evaluation values representing each emotion during a predetermined period from the start of a conversation, a statistic of evaluation values representing each emotion during a predetermined period going back a certain period from the end of the conversation, or a statistic of evaluation values representing each emotion during a predetermined period elapsed from the timing when a specific keyword was uttered in the conversation, In the estimation step, it is estimated which of a plurality of patterns the speaker belongs to, using the emotion value, the numerical value representing the frequently occurring word, and a weight for each of a plurality of patterns. Information processing methods.
10. An information processing method to be executed by a computer of an information processing device, comprising: A voice data acquisition step of acquiring voice data; an extraction step of extracting, from the voice data by voice recognition, a plurality of emotion values representing emotions of speakers in the voice data and text data representing contents of utterances by the speakers; a frequent word extraction step of extracting one or more frequent words included in the text data; an estimation step of estimating which of a plurality of types of patterns the speaker belongs to based on the frequent words and the emotion value; having In the frequent word extraction step, the text data and the emotion value corresponding to the text data are integrated as all numerical values or as all text. Information processing methods.
11. The computer of the information processing device A voice data acquisition step of acquiring voice data; an extraction step of extracting, from the voice data by voice recognition, a plurality of emotion values representing emotions of speakers in the voice data and text data representing contents of utterances by the speakers; a prediction step of predicting the evaluation value by inputting the emotional value and the text data into a learning model generated based on a correspondence relationship between the text data, the emotional value corresponding to the text data, and an evaluation value for a conversation corresponding to the text data; A program for executing the above.
12. The computer of the information processing device A voice data acquisition step of acquiring voice data; an extraction step of extracting, from the voice data by voice recognition, a plurality of emotion values representing emotions of speakers in the voice data and text data representing contents of utterances by the speakers; a frequent word extraction step of extracting one or more frequent words included in the text data; an estimation step of estimating which of a plurality of types of patterns the speaker belongs to based on the frequent words and the emotion value; Run the command, the emotion value is any one of a statistic of evaluation values representing each emotion during a predetermined period from the start of a conversation, a statistic of evaluation values representing each emotion during a predetermined period going back a certain period from the end of the conversation, or a statistic of evaluation values representing each emotion during a predetermined period elapsed from the timing when a specific keyword was uttered in the conversation, In the estimation step, it is estimated which of a plurality of patterns the speaker belongs to, using the emotion value, the numerical value representing the frequently occurring word, and a weight for each of a plurality of patterns. program.
13. The computer of the information processing device A voice data acquisition step of acquiring voice data; an extraction step of extracting, from the voice data by voice recognition, a plurality of emotion values representing emotions of speakers in the voice data and text data representing contents of utterances by the speakers; a frequent word extraction step of extracting one or more frequent words included in the text data; an estimation step of estimating which of a plurality of types of patterns the speaker belongs to based on the frequent words and the emotion value; Run the command, In the frequent word extraction step, the text data and the emotion value corresponding to the text data are integrated as all numerical values or all text. program.
Citation Information
Patent Citations
Telephone voice monitoring evaluation system
JP2017135642A
Speaker estimation device
JP2019124835A
Information providing system, information providing method, and computer program
JP2022154230A
Customer service support system, customer service support device, customer service support method, and customer service support program
JP2022187611A
Dialogue support server, dialogue support system, dialogue support method, and program
JP2023005289A
