Speech section classification device, speech section classification method, and speech section classification program

Abstract

This utterance segment classification device comprises: an utterance segment estimation unit for estimating an utterance segment from utterance text data including utterances by two or more people; an utterance type estimation unit for estimating the utterance type with respect to each piece of utterance included in the utterance segment estimated by the utterance segment estimation unit; and an utterance segment classification unit for classifying the utterance segment estimated by the utterance segment estimation unit using the utterance type of each piece of utterance estimated by the utterance type estimation unit and an utterance segment classification rule defined in advance as a rule for classifying an utterance segment on the basis of utterance type.

Description

[Technical Field]

[0001] The disclosed technology relates to a speech segment classification device, a speech segment classification method, and a speech segment classification program. [Background technology]

[0002] There are techniques for classifying speech segments included in a dialogue between two or more speakers, such as a dialogue between an operator and a customer at a contact center, or a dialogue between a sales representative and a customer in face-to-face sales.

[0003] Contact centers are currently working to record conversations between operators and customers, analyze the content, and use it to improve services. For example, there is a need to understand and collect so-called "customer feedback" by extracting and analyzing sections of the conversation in which customers express their dissatisfaction or requests regarding the services provided. As another example, there is a need to classify and analyze the content and type of sales talks that are made by operators during the conversation, thereby gaining insight into what kind of sales operators are excellent, and to use this knowledge for training new operators.

[0004] A conventional technique for classifying a speech segment consisting of a single utterance or multiple utterances, or more generally, a text of a certain length, according to its topic or content, is, for example, a method that uses training data in which information on the classification destination is attached to the utterance or the text (see, for example, Non-Patent Document 1). In this method, machine learning is performed using the training data in which information on the classification destination is attached, and a model that determines the classification destination is generated. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin. LIBLINEAR: A library for large linear classification Journal of Machine Learning Research 9(2008), 1871-1874. Summary of the Invention [Problem to be solved by the invention]

[0006] The above-described conventional techniques have the following problems. In a method of assigning a label to each utterance, performing machine learning, and training a classification model using the assigned labels, many utterances in natural conversation are very short, making it difficult to assign a label to each one. Furthermore, even if it were possible to assign a label to each utterance, many utterances that do not contribute to the classification of the speech section are often included, making it difficult to perform classification using a classifier simply based on the assigned labels. In other words, a method of running the entire utterance included in an utterance section through a classifier cannot accurately classify a section that contains many utterances that do not contribute to the classification of the speech section.

[0007] The disclosed technology has been made in consideration of the above points, and aims to provide a speech section classification device, a speech section classification method, and a speech section classification program that can accurately classify speech sections even if the speech section includes utterances that do not contribute to classification. [Means for solving the problem]

[0008] A first aspect of the present disclosure is a speech section classification device comprising: a speech section estimation unit that estimates a speech section from speech text data including utterances from two or more people; an utterance type estimation unit that estimates an utterance type for each utterance included in the speech section estimated by the speech section estimation unit; and a speech section classification unit that classifies the speech section estimated by the speech section estimation unit using the utterance type of each utterance estimated by the utterance type estimation unit and a speech section classification rule that is predetermined as a rule for classifying the speech section based on the utterance type.

[0009] A second aspect of the present disclosure is a speech section classification method that estimates a speech section from speech text data including utterances from two or more people, estimates an utterance type for each utterance included in the estimated speech section, and classifies the estimated speech section using the utterance type of each estimated utterance and a speech section classification rule that is predetermined as a rule for classifying speech sections based on the utterance type.

[0010] A third aspect of the present disclosure is a speech section classification program that causes a computer to estimate a speech section from speech text data including utterances from two or more people, estimate an utterance type for each utterance included in the estimated speech section, and classify the estimated speech section using the utterance type of each estimated utterance and a speech section classification rule that is predetermined as a rule for classifying speech sections based on the utterance type. [Effects of the Invention]

[0011] The disclosed technology has the advantage that it is possible to accurately classify an utterance section even when the utterance in the utterance section includes an utterance that does not contribute to classification. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 2 is a block diagram illustrating an example of a hardware configuration of the speech segment classification device according to the embodiment. [Figure 2] 1 is a block diagram illustrating an example of a functional configuration of an utterance segment classification device according to an embodiment. [Figure 3] 3 is a diagram illustrating an example of the configuration of a sentence input unit illustrated in FIG. 2. FIG. [Figure 4] 3 is a diagram illustrating an example of the configuration of a speech section estimation unit illustrated in FIG. 2. FIG. [Figure 5] 3 is a diagram illustrating an example of the configuration of an utterance type estimation unit illustrated in FIG. 2. FIG. [Figure 6] 3 is a diagram illustrating an example of the configuration of an utterance period classification unit and an output unit illustrated in FIG. 2. FIG. [Figure 7] 5 is a flowchart showing an example of a processing flow by the speech segment classification program according to the first embodiment. [Figure 8] 1 is a flowchart showing an example of the flow of speech segment classification processing according to the first embodiment, illustrating an example of speech segment classification rules. [Figure 9] FIG. 4 is a diagram showing an example of a speech segment classification result according to the first embodiment. [Figure 10] 10 is a flowchart showing an example of the flow of the speech segment classification process according to the second embodiment, illustrating another example of the speech segment classification rule. [Figure 11] FIG. 10 is a diagram showing an example of a speech segment classification result according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] An example of an embodiment of the disclosed technology will be described below with reference to the drawings. Note that in each drawing, the same or equivalent components and parts are given the same reference numerals. Also, the dimensional proportions in the drawings are exaggerated for the convenience of explanation and may differ from the actual proportions.

[0014] [First embodiment] The speech segment classification device of the first embodiment provides certain improvements over conventional methods of classifying speech segments by passing the entire utterance contained in a speech segment through a classifier, and represents an advancement in the technical field of classifying speech segments contained in a dialogue.

[0015] Conventional methods that apply a classifier to the entire utterance contained in a speech section have the problem of being unable to accurately classify when the speech section contains a large amount of utterance that is irrelevant to the classification. Furthermore, conventional methods that use information that contributes to classification from the speech section have the problem of being unable to accurately classify when the contributing information differs in the way it contributes to the final classification and cannot be uniquely determined.

[0016] In contrast, in this embodiment, the utterance type of each utterance included in an utterance section is estimated, and the utterance section is classified based on whether or not there is a specific type among the estimated utterance types, or on the combination and order of multiple types. This makes it possible to accurately classify the utterance section even when the utterance section contains many utterances that are not relevant to classification, or when information that contributes to classification cannot be uniquely determined.

[0017] For example, consider the utterance sections shown in the following dialogue examples 1 and 2. The utterance content is shown in " " and the determined utterance label is shown in ( ).

[0018] Example dialogue 1: First speaker: "I'm sorry, but we are unable to accommodate your inquiry regarding adding a line." (Operator explaining the negative situation) Second speaker: "Just to confirm, we discussed last time whether it would be possible to get two more lines, one for home and one for the office, with our current contract." (Customer explanation / answer) First speaker: "Yes, your current contract allows you to use a maximum of five lines, so you can use just one more line." (Operator explanation and response) Second speaker: "I see, I understand." (Customer explanation / answer)

[0019] Example dialogue 2: First speaker: "I'm sorry, but we are unable to accommodate your inquiry regarding adding a line." (Operator explaining the negative situation) Second speaker: "Just to confirm, we discussed last time whether it would be possible to get two more lines, one for home and one for the office, with our current contract." (Customer explanation / answer) First speaker: "Yes, your current contract allows you to use a maximum of five lines, so you can use just one more line." (Operator explanation and response) Second speaker: "But, in your previous explanation, I heard that it was possible, so is it really not possible?" (Customer question)

[0020] In both the above dialogue example 1 and dialogue example 2, the first through third utterances and the determined utterance labels are the same, but only the final utterance in response to these utterances is different. In this example, in dialogue example 2, the second speaker, who is a customer, expresses doubts and dissatisfaction in response to the explanation given by the first speaker, who is an operator, and from the perspective of collecting customer feedback, it needs to be classified as customer feedback. On the other hand, in dialogue example 1, there is no need to classify it as customer feedback. Furthermore, the second and third utterances do not contribute to classification. However, if the utterance labels are simply used to run a classifier to determine and classify customer feedback, the utterances and utterance labels that make up dialogue example 1 and dialogue example 2 are almost identical, so accurate classification cannot be performed and classification accuracy will decrease.

[0021] In this embodiment, a speech section is estimated from speech text data, an utterance type is estimated for each utterance included in the estimated speech section, and the estimated utterance type is used to classify the speech section. By selectively using the utterance type according to the purpose of classification, it is possible to accurately classify the speech section even if the utterance included in the speech section includes an utterance that does not contribute to classification. Note that speech text data includes one or more speech sections and is a concept representing a set of all utterances in one dialogue. A speech section is a concept representing a set of continuous utterances. An utterance is a concept representing a segment obtained from voice recognition, text chat, or the like. A speech type is a concept representing a type of utterance.

[0022] First, with reference to FIG. 1, the hardware configuration of the speech segment classification device 10 according to this embodiment will be described.

[0023] FIG. 1 is a block diagram showing an example of the hardware configuration of an utterance segment classification device 10 according to this embodiment.

[0024] 1, the speech segment classification device 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, a display unit 16, and a communication interface (I / F) 17. Each component is connected to each other via a bus 18 so as to be able to communicate with each other.

[0025] The CPU 11 is a central processing unit that executes various programs and controls each part. That is, the CPU 11 reads a program from the ROM 12 or the storage 14 and executes the program using the RAM 13 as a work area. The CPU 11 controls each of the above components and performs various arithmetic processing in accordance with the program stored in the ROM 12 or the storage 14. In this embodiment, the ROM 12 or the storage 14 stores a speech segment classification program for executing the speech segment classification processing.

[0026] The ROM 12 stores various programs and various data. The RAM 13 temporarily stores programs or data as a working area. The storage 14 is configured with an HDD (Hard Disk Drive) or SSD (Solid State Drive) and stores various programs including the operating system and various data.

[0027] The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used to input various information to the device itself.

[0028] The display unit 16 is, for example, a liquid crystal display, and displays various information. The display unit 16 may also function as the input unit 15 by adopting a touch panel system.

[0029] The communication interface 17 is an interface for the device itself to communicate with other external devices. For this communication, a wired communication standard such as Ethernet (registered trademark) or FDDI (Fiber Distributed Data Interface) or a wireless communication standard such as 4G, 5G, or Wi-Fi (registered trademark) is used.

[0030] The speech segment classification device 10 according to this embodiment is implemented by a general-purpose computer such as a server computer or a personal computer (PC).

[0031] Next, the functional configuration of the speech segment classification device 10 will be described with reference to FIG.

[0032] FIG. 2 is a block diagram showing an example of the functional configuration of the speech segment classification device 10 according to this embodiment.

[0033] 2, the speech segment classification device 10 includes, as functional components, a sentence input unit 101, a speech segment estimation unit 102, an utterance type estimation unit 103, an utterance segment classification unit 104, and an output unit 105. Each functional component is realized by the CPU 11 reading out the speech segment classification program stored in the ROM 12 or the storage 14, expanding it in the RAM 13, and executing it.

[0034] Note that the utterance DB (Data Base) 20 that stores utterance data and the classification result DB 24 that stores classification result data may each be stored in the storage 14 or in an external, accessible storage device. Similarly, the utterance text DB 21 that stores utterance text data, the utterance section DB 22 that stores utterance section data, and the utterance section / utterance type DB 23 that stores utterance section / utterance type data may each be stored in the storage 14 or in an external, accessible storage device. Note that, although the utterance text data, the utterance section data, and the utterance section / utterance type data are stored in different DBs in the example of Fig. 2, they may also be stored in a single DB.

[0035] As an example, the following describes a case where a speech section in which a first speaker, who is an operator, describes a negative situation and a second speaker, who is a customer (hereinafter also referred to as "customer"), responds, is classified as to whether or not it contains "customer feedback." Note that "customer feedback" refers to the part in which the customer expresses dissatisfaction or requests regarding the service provided or the operator's response.

[0036] The configuration of each functional unit (sentence input unit 101, speech section estimation unit 102, utterance type estimation unit 103, speech section classification unit 104, and output unit 105) shown in FIG. 2 will be specifically described with reference to FIGS. 3 to 6.

[0037] The sentence input unit 101 shown in FIG. 3 acquires utterance data from the utterance DB 20, converts the acquired utterance data, and stores the resulting utterance text data in the utterance text DB 21. The utterance data is data including utterances from two or more people, and may be a character string or a voice. When the utterance data is a voice, the sentence input unit 101 converts the utterance into text by performing voice recognition and stores the converted text in the utterance text DB 21. When the utterance data is a character string, the sentence input unit 101 stores the converted text as it is in the utterance text DB 21 since the text has already been converted into text. For example, the utterances in the above-mentioned dialogue examples 1 and 2 are stored as voice in the utterance DB 20 as the utterance data. When the utterance data is input as voice, the sentence input unit 101 converts the utterance data into text by using voice recognition and stores the resulting utterance text data in the utterance text DB 21.

[0038] The speech section estimation unit 102 shown in FIG. 4 acquires utterance text data from the speech text DB 21, estimates utterance sections from the acquired utterance text data, and stores the acquired utterance section data in the speech section DB 22. Specifically, when the utterance text data is input, the speech section estimation unit 102 estimates utterance sections using the utterance section estimation model 30, and stores the acquired utterance section data in the utterance section DB 22. The utterance section estimation model 30 is a trained model that receives utterance text data as input and outputs utterance section data. For example, a DNN (Deep Neural Network), which is a multi-layered neural network, is used as the speech section estimation model 30. The speech section estimation model 30 may be stored in the storage 14 or in an external storage device. For example, the speech section estimation model 30 is generated by assigning teacher labels to utterances containing clue words indicating topic changes, such as "Well then" or "By the way," and performing machine learning using the labeled utterance text data as training data. This speech section estimation model 30 is used to determine speech changes, and the speech from one change to the next is estimated as the speech section.

[0039] The speech type estimation unit 103 shown in FIG. 5 acquires utterance section data from the speech section DB 22, estimates the utterance type of each utterance included in the acquired utterance section data, and stores the obtained utterance section and utterance type data in the utterance section and utterance type DB 23. Specifically, when the utterance section data is input, the utterance type estimation unit 103 estimates the utterance type of each utterance included in the utterance section using the utterance type estimation model 31, and stores the obtained utterance section and utterance type data in the utterance section and utterance type DB 23. The utterance type estimation model 31 is a trained model that receives the utterance section data as input and outputs the utterance section and utterance type data. For example, a DNN is used for the utterance type estimation model 31. The utterance type estimation model 31 may be stored in the storage 14 or in an external storage device. For example, the following labels (Type 1 to Type 9) are defined as the utterance types. Note that an explanation of each label is shown in < >. A model for classifying these utterance types is generated in advance by machine learning using speech section data in which these labels are attached to each utterance as training data. Using this utterance type estimation model 31, the utterance type of each utterance is estimated for the input utterance section.

[0040] (Type 1) Customer Question <Question spoken by a customer to an operator> (Type 2) Customer explanation / answer <Utterances in which the customer answers or explains the operator's question> (Type 3) Customer requests and demands <Utterances expressing requests or demands from customers to operators> (Type 4) Operator Negative Situation <Operator utterances describing a negative situation> (Type 5) Customer Negative Situation <Customer utterances describing a negative situation> (Type 6) Operator Negative Buffering <Operator utterances using expressions that soften negative circumstances> (Type 7) Customer Positive Evaluation <Utterances in which customers evaluate using positive expressions> (Type 8) Customer Negative Evaluation <Utterances in which customers evaluate using negative expressions> (Type 9) Understanding the purpose of the call <Customer and operator's speech regarding the purpose of the call>

[0041] The speech section classification unit 104 shown in FIG. 6 acquires speech section and utterance type data from the speech section and utterance type DB 23, and classifies the speech section estimated by the speech section estimation unit 102 using the utterance type of each utterance estimated by the utterance type estimation unit 103 and the speech section classification rule 32. The utterance section classification rule 32 is a predetermined rule for classifying the speech section based on the utterance type. The utterance section classification rule 32 classifies the speech section based on whether or not a specific utterance type is included in the speech section, or on the combination and order relationship of multiple utterance types included in the speech section. Note that the speech section classification process requires only one or more utterance types estimated from the utterance text; the utterance text itself included in the speech section is not required for the process.

[0042] Specifically, the speech section classification rule 32 classifies a speech section as a section including a part in which a customer expresses dissatisfaction or a request (i.e., "customer feedback") if the speech section includes an utterance type (type 7) that indicates an utterance evaluated by a customer using positive expressions, or an utterance type (type 8) that indicates an utterance evaluated by a customer using negative expressions. This allows "customer feedback" to be accurately understood and collected.

[0043] Furthermore, the speech section classification rule 32 classifies an utterance section as a section including a part in which a customer expresses dissatisfaction or a request (i.e., "customer feedback") if the utterance section includes an utterance type (type 9) indicating an utterance by a customer and an operator regarding a matter, and the utterance with the utterance type (type 9) is also classified as any of the following types: an utterance type (type 1) indicating a question uttered by a customer to an operator, an utterance type (type 3) indicating an utterance in which a customer expresses a request or a demand to an operator, an utterance type (type 2) indicating an utterance in which a customer answers or explains a question uttered by an operator, and an utterance type (type 5) indicating an utterance in which a customer describes a negative situation. This makes it possible to accurately understand and collect "customer feedback" in the same way as above.

[0044] Furthermore, the speech section classification rule 32 classifies an utterance section as a section including a part in which a customer expresses dissatisfaction or a request (i.e., "customer feedback") if the utterance section includes an utterance type (type 4) indicating an utterance by an operator describing a negative situation, or an utterance type (type 6) indicating an utterance by an operator using an expression that softens the negative circumstances, and if, within two utterances after the utterance with the utterance type (type 4 or type 6), either an utterance type (type 1) indicating a question uttered by a customer to an operator, or an utterance type (type 3) indicating a request or demand made by a customer to an operator is included. This makes it possible to accurately understand and collect "customer feedback" in the same way as above.

[0045] The output unit 105 shown in FIG. 6 acquires the classification result data classified by the speech segment classification unit 104, and stores the acquired classification result data in the classification result DB 24.

[0046] Next, the operation of the speech segment classification device 10 according to the first embodiment will be described with reference to FIG.

[0047] 7 is a flowchart showing an example of the flow of processing by the speech section classification program according to the embodiment 1. The processing by the speech section classification program is realized by the CPU 11 of the speech section classification device 10 writing the speech section classification program stored in the ROM 12 or the storage 14 into the RAM 13 and executing it.

[0048] In step S101 of FIG. 7, the CPU 11 accepts input of utterance data from the utterance DB 20, converts the accepted utterance data, and stores the resulting utterance text data in the utterance text DB 21.

[0049] In step S102, the CPU 11 acquires utterance text data from the utterance text DB 21, estimates an utterance section corresponding to the acquired utterance text data using the utterance section estimation model 30, and stores the acquired utterance section data in the utterance section DB 22.

[0050] In step S103, the CPU 11 acquires speech section data from the speech section DB 22, estimates the utterance type corresponding to each utterance included in the acquired speech section data using the utterance type estimation model 31, and stores the acquired utterance section / utterance type data in the speech section / utterance type DB 23.

[0051] In step S104, the CPU 11 acquires the utterance section / utterance type data from the utterance section / utterance type DB 23, and classifies the utterance section estimated in step S102 using the utterance type of each utterance estimated in step S103 and the utterance section classification rule 32. A specific example of this utterance section classification process will be described with reference to FIG.

[0052] FIG. 8 is a flowchart showing an example of the flow of the speech segment classification process according to the first embodiment, and shows an example of the speech segment classification rule 32.

[0053] In step S111, the CPU 11 acquires speech section / utterance type data from the speech section / utterance type DB 23. As described above, the speech section classification process requires only one or more utterance types estimated from the utterance text, and does not require the utterance text itself included in the speech section.

[0054] In step S112, the CPU 11 determines whether the utterance types identified from the utterance section / utterance type data acquired in step S111 include "Type 7: Customer Positive Evaluation" or "Type 8: Customer Negative Evaluation" among the labels "Type 1" to "Type 9" described above. If it is determined that "Type 7: Customer Positive Evaluation" or "Type 8: Customer Negative Evaluation" is included (in the case of a positive determination), the process proceeds to step S117, and if it is determined that "Type 7: Customer Positive Evaluation" or "Type 8: Customer Negative Evaluation" is not included (in the case of a negative determination), the process proceeds to step S113.

[0055] In step S113, the CPU 11 determines whether or not the utterance types include "Type 9: Understanding the Subject." If it is determined that "Type 9: Understanding the Subject" is included (in the case of a positive determination), the process proceeds to step S114, and if it is determined that "Type 9: Understanding the Subject" is not included (in the case of a negative determination), the process proceeds to step S115.

[0056] In step S114, the CPU 11 determines whether the utterance with "Type 9: Understanding the Context" is also assigned any of the following types: "Type 1: Customer Question," "Type 3: Customer Request / Request," "Type 2: Customer Explanation / Response," and "Type 5: Customer Negative Situation." If it is determined that any of the types is assigned (positive determination), the process proceeds to step S117, and if it is determined that any of the types is not assigned (negative determination), the process proceeds to step S118.

[0057] In step S115, the CPU 11 determines whether the utterance types include "Type 4: Operator Negative Situation" or "Type 6: Operator Negative Buffer." If it is determined that "Type 4: Operator Negative Situation" or "Type 6: Operator Negative Buffer" is included (in the case of a positive determination), the process proceeds to step S116, and if it is determined that "Type 4: Operator Negative Situation" or "Type 6: Operator Negative Buffer" is not included (in the case of a negative determination), the process proceeds to step S118.

[0058] In step S116, the CPU 11 determines whether or not either of the types "Type 1: Customer Question" and "Type 3: Customer Request / Demand" is included within two utterances after the utterance with "Type 4: Operator Negative Situation" or "Type 6: Operator Negative Buffer." If it is determined that either of the types is included (in the case of a positive determination), the process proceeds to step S117, and if it is determined that either of the types is not included (in the case of a negative determination), the process proceeds to step S118.

[0059] In step S117, the speech section identified by the speech section / speech type data is classified as "customer's voice," and the process returns to step S105 in FIG.

[0060] In step S118, the speech section identified by the speech section / speech type data is classified as "not a customer's voice," and the process returns to step S105 in FIG.

[0061] Returning to step S105 in FIG. 7, the CPU 11 outputs the classification result data obtained by classifying the speech segments in step S104 to the classification result DB 24, and ends the series of processes according to the speech segment classification program.

[0062] Fig. 9 is a diagram showing an example of a speech section classification result according to the first embodiment. The speech section classification result shown in Fig. 9 shows a classification result obtained by classification according to the speech section classification rule 32 shown in Fig. 8 described above.

[0063] In the case of the speech section W1 shown in Figure 9, the speech type of the first speaker's utterance is estimated to be "Type 4: Operator Negative Situation", and the speech type of the second speaker's utterance is estimated to be "Type 2: Customer Explanation and Response". Next, the speech type of the first speaker's utterance is estimated to be "Operator Explanation and Response", and the speech type of the second speaker's utterance is estimated to be "Type 2: Customer Explanation and Response".

[0064] In this case, as shown in the classification example, it is determined whether "Type 7: Customer Positive Evaluation" or "Type 8: Customer Negative Evaluation" is included in the utterance section W1. Here, the answer is "NO." Next, it is determined whether "Type 9: Understanding the Item" is included in the utterance section W1. Here, the answer is "NO." Next, it is determined whether "Type 4: Operator Negative Situation" or "Type 6: Operator Negative Buffer" is included in the utterance section W1. Here, the answer is "YES." Next, it is determined whether either "Type 1: Customer Question" or "Type 3: Customer Request / Demand" is included within two utterances after the utterance with "Type 4: Operator Negative Situation" or "Type 6: Operator Negative Buffer." Here, the answer is "NO."

[0065] In this case, as shown in the classification result, the speech section W1 is classified as "not a customer's voice."

[0066] In the case of the speech section W2 shown in Figure 9, the speech type of the first speaker's utterance is estimated to be "Type 4: Operator Negative Situation", and the speech type of the second speaker's utterance is estimated to be "Type 2: Customer Explanation and Answer". Next, the speech type of the first speaker's utterance is estimated to be "Operator Explanation and Answer", and the speech type of the second speaker's utterance is estimated to be "Type 1: Customer Question".

[0067] In this case, as shown in the classification example, it is determined whether or not "Type 7: Customer Positive Evaluation" or "Type 8: Customer Negative Evaluation" is included in the utterance section W2. Here, the answer is "NO." Next, it is determined whether or not "Type 9: Understanding the Item" is included in the utterance section W2. Here, the answer is "NO." Next, it is determined whether or not "Type 4: Operator Negative Situation" or "Type 6: Operator Negative Buffer" is included in the utterance section W2. Here, the answer is "YES." Next, it is determined whether or not either "Type 1: Customer Question" or "Type 3: Customer Request / Demand" is included within two utterances after the utterance with "Type 4: Operator Negative Situation" or "Type 6: Operator Negative Buffer." Here, the answer is "YES."

[0068] In this case, as shown in the classification result, the speech section W2 is classified as "customer's voice."

[0069] As described above, according to this embodiment, the speech section is estimated from the speech text data obtained by converting the input speech data, the speech type of each utterance included in the speech section is estimated, and the speech section is classified using the obtained speech type and the speech section classification rule. This makes it possible to accurately classify the speech sections required for analyzing "customer feedback."

[0070] [Second embodiment] The speech segment classification device according to the second embodiment, like the first embodiment, provides certain improvements over conventional methods of classifying speech segments by passing the entire speech included in a speech segment through a classifier, and represents an advancement in the technical field of classifying speech segments included in a dialogue.

[0071] In this embodiment, as another example of the speech section classification process, a case will be described in which a speech section in which an operator is making a sales talk is classified using the speech type.

[0072] In contact centers, with the aim of improving the quality of service provided by agents and efficiently training new agents, there has been growing interest in the conversational flow of excellent agents and how they differ from incompetent agents. When agents engage in sales dialogues, because the customer's needs are unclear at the beginning, they are expected to conduct vague inquiries such as, "Is there anything I can help you with?" Once the customer's needs become more concrete, they are expected to conduct inquiries about specific content and themes. Furthermore, at the end of the conversation, they are expected to conduct inquiries specific to the end, such as, "Is there anything else I can help you with?" In other words, the way in which needs are heard at the beginning of a sales dialogue differs from the way they are heard in the middle and end of the dialogue. Therefore, it is possible to categorize conversational periods into three types: "open sales periods," which are periods in which the conversation is not focused on a specific topic or theme; "topic sales periods," which are periods in which the conversation is focused on a specific topic or theme; and "end sales periods," which are periods in which the presence or absence of other topics or themes is confirmed. These three types can be further specified as follows: "Open sales section," which is a section in which the conversation is conducted by vaguely asking about needs without mentioning any specific service or topic; "Theme sales section," which is a section in which a specific sales talk is conducted about a specific service or topic; and "End sales section," which is a section in which the conversation feels like the conversation is closing on a specific service or topic, or a conversation in which the presence or absence of other needs is confirmed.

[0073] The components of the speech segment classification device according to the second embodiment (hereinafter referred to as the speech segment classification device 10A) are the same as the components of the speech segment classification device 10 according to the first embodiment. That is, the speech segment classification device 10A has, as functional components, the above-mentioned sentence input unit 101, speech segment estimation unit 102, utterance type estimation unit 103, speech segment classification unit 104, and output unit 105. Note that repeated explanations of the sentence input unit 101, speech segment estimation unit 102, and output unit 105 will be omitted.

[0074] As shown in FIG. 5 , when speech section data is input, the utterance type estimation unit 103 uses the utterance type estimation model 31 to estimate the utterance type of each utterance included in the speech section, and stores the obtained utterance section / utterance type data in the utterance section / utterance type DB 23. The utterance type estimation model 31 is a trained model that receives speech section data as input and outputs speech section / utterance type data. For example, the following labels (type 11 to type 16) are defined as speech types. A model for classifying these utterance types is generated in advance by machine learning using training data with these labels. The utterance type estimation model 31 is used to estimate the utterance type for the input speech section. Note that an open question is a question that asks about needs, mainly at the beginning of a conversation. For example, it is a question that vaguely asks about needs without mentioning a specific service or topic, such as "Is there anything you're looking for?". A thematic question is a question about a specific topic or theme, mainly asked in the middle of a conversation. Topic questions are questions other than open questions and closing questions, such as questions that specifically ask about needs regarding a specific service or topic. Closing questions are questions that are mainly asked at the end of a conversation to check whether there are any other topics or themes. Closing questions are questions that give the impression that the conversation on a specific topic has come to an end, but vaguely ask whether there are any other needs.

[0075] (Type 11) Operator needs hearing / open questions (Type 12) Operator needs interview / topic questions (Type 13) Operator needs interview / end question (Type 14) Operator Proposal (Type 15) Operator response (Type 16) Customer Response

[0076] As shown in the above-mentioned FIG. 6 , the speech section classification unit 104 acquires speech section / utterance type data from the speech section / utterance type DB 23, and classifies the speech sections estimated by the speech section estimation unit 102 using the utterance type of each utterance estimated by the speech type estimation unit 103 and the speech section classification rule 32.

[0077] Specifically, the speech section classification rule 32 classifies an utterance section as an open-type sales section if the utterance section includes an utterance type indicating an utterance by an operator hearing about customer needs (hereinafter also referred to as a "needs hearing utterance") and the type of the first needs hearing utterance in the utterance section is an open question (type 11). Furthermore, if the utterance section includes a "needs hearing utterance" and the type of the first needs hearing utterance in the utterance section is a theme question (type 12), the utterance section is classified as a theme-type sales section. Furthermore, if the utterance section includes a "needs hearing utterance" and the type of the first needs hearing utterance in the utterance section is an end question (type 13), the utterance section is classified as an end-type sales section. This makes it possible to accurately grasp and collect utterance sections including an operator's sales talk according to their content.

[0078] Next, the speech section classification process according to the second embodiment will be described with reference to Fig. 10. Note that, as described above, in the speech section classification process, it is sufficient to have one or more utterance types estimated from the utterance text, and the utterance text itself included in the utterance section is not necessary for the process.

[0079] FIG. 10 is a flowchart showing an example of the flow of the speech segment classification process according to the second embodiment, and shows another example of the speech segment classification rule 32.

[0080] In step S121, the CPU 11 acquires the utterance section / utterance type data from the utterance section / utterance type DB 23.

[0081] In step S122, the CPU 11 determines whether or not the utterance types identified from the utterance section / utterance type data acquired in step S121 include "needs hearing utterance" among the labels of "Type 11" to "Type 16" described above. If it is determined that a "needs hearing utterance" is included (in the case of a positive determination), the process proceeds to step S123, and if it is determined that a "needs hearing utterance" is not included (in the case of a negative determination), the process proceeds to step S126.

[0082] In step S123, the CPU 11 determines the type of the first needs hearing utterance in the utterance section. If it is determined to be "Type 11: operator needs hearing open question," the process proceeds to step S124; if it is determined to be "Type 12: operator needs hearing theme question," the process proceeds to step S125; and if it is determined to be "Type 13: operator needs hearing end question," the process proceeds to step S126.

[0083] In step S124, the CPU 11 classifies the utterance period specified by the utterance period / utterance type data as an "open type business period," and returns to step S105 in FIG.

[0084] In step S125, the CPU 11 classifies the utterance section identified by the utterance section / utterance type data as a "theme-type business section," and returns to step S105 in FIG.

[0085] In step S126, the CPU 11 classifies the utterance period specified by the utterance period / utterance type data as an "end-type business period," and returns to step S105 in FIG.

[0086] Fig. 11 is a diagram showing an example of a speech section classification result according to the second embodiment. The speech section classification result shown in Fig. 11 shows the classification result obtained by classification according to the speech section classification rule 32 shown in Fig. 10 described above.

[0087] In the case of the speech section W11 shown in Figure 11, the speech type of the operator's utterance is estimated to be "Type 11: Operator needs hearing / open question", the speech type of the customer's utterance is estimated to be "Type 16: Customer response", and the speech type of the operator's utterance is estimated to be "Type 15: Operator response".

[0088] In this case, as shown in the classification result, the speech section W11 is classified as an "open-type business section."

[0089] In the case of the speech section W12 shown in Figure 11, the speech type of the operator's speech is estimated to be "Type 12: Operator needs hearing, topic question", the speech type of the customer's speech is estimated to be "Type 16: Customer response", and the speech type of the operator's speech is estimated to be "Type 14: Operator suggestion".

[0090] In this case, as shown in the classification result, the utterance section W12 is classified as a "thematic sales section."

[0091] In the case of the speech section W13 shown in FIG. 11, the speech type of the operator's speech is estimated to be "Type 13: Operator needs hearing, end question," and the speech type of the customer's speech is estimated to be "Type 16: Customer response."

[0092] In this case, as shown in the classification result, the utterance section W13 is classified as an "end-type business section."

[0093] According to this embodiment, the speech section is estimated from the speech text data obtained by converting the input speech data, the speech type of each utterance included in the speech section is estimated, and the speech section is classified using the obtained speech type and the speech section classification rule. This allows accurate classification of business sections, which is useful for analyzing excellent customer service in contact centers.

[0094] As a result, even in cases where accurate classification was previously difficult, it is possible to accurately classify speech sections by estimating the speech type for each utterance included in the speech section and selectively using the estimated speech type depending on the purpose of classification.

[0095] As for the method of estimating the speech section, in addition to the above, the speech section may be estimated by any of the following methods. (Method 1) A predetermined number N of utterances (N is 2 or more) are collected together to form one utterance section. (Method 2) One input utterance text data, that is, one utterance, is regarded as one speech section.

[0096] The speech section classification process executed by the CPU 11 after reading the speech section classification program in the above embodiment may be executed by various processors other than the CPU 11. Examples of processors in this case include dedicated electrical circuits, such as programmable logic devices (PLDs) (such as field-programmable gate arrays (FPGAs)) whose circuit configuration can be changed after manufacture, and application-specific integrated circuits (ASICs) that are processors having a circuit configuration specifically designed to execute specific processes. Furthermore, the speech section classification process may be executed by one of these various processors, or may be executed by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). Furthermore, the hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor elements.

[0097] In the above embodiment, the speech segment classification program is described as being stored (also referred to as "installed") in advance in the ROM 12 or the storage 14, but this is not limiting. The speech segment classification program may be provided in a form stored in a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The speech segment classification program may also be downloaded from an external device via a network.

[0098] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[0099] The following additional notes are provided regarding the above-described embodiments.

[0100] (Additional note 1) Memory and at least one processor coupled to said memory; Including, The processor: Estimating speech sections from speech text data including speech by two or more people; estimating an utterance type for each utterance included in the estimated utterance section; The estimated speech sections are classified using the speech type of each estimated utterance and a speech section classification rule that is predetermined as a rule for classifying the speech sections based on the speech type. The speech segment classification device is configured as follows.

[0101] (Additional note 2) A non-transitory storage medium storing a program executable by a computer to perform speech segment classification processing, The speech segment classification process includes: Estimating speech sections from speech text data including speech by two or more people; estimating an utterance type for each utterance included in the estimated utterance section; The estimated speech sections are classified using the speech type of each estimated utterance and a speech section classification rule that is predetermined as a rule for classifying the speech sections based on the speech type. Non-transitory storage medium. [Explanation of symbols]

[0102] 10 Speech segment classification device 11 CPU 12 ROM 13 RAM 14. Storage 15 Input section 16 Display section 17 Communication I / F 18 Bus 20 Speech DB 21 Speech Text DB 22 Speech Segment DB 23 Speech section / speech type DB 24 Classification result DB 30 Voice segment estimation model 31 Speech type estimation model 32 Speech segment classification rules 101 Sentence input section 102 Speech segment estimation unit 103 Speech type estimation unit 104 Speech segment classification unit 105 Output section

Claims

1. a speech section estimation unit that estimates a speech section from speech text data including speeches by two or more people; an utterance type estimation unit that estimates an utterance type for each utterance included in the utterance section estimated by the utterance section estimation unit; an utterance section classification unit that classifies the utterance sections estimated by the utterance section estimation unit using the utterance type of each utterance estimated by the utterance type estimation unit and a predetermined utterance section classification rule as a rule for classifying the utterance sections based on the utterance type; Equipped with the speech section classification rule determines whether a specific speech type is included in the speech section, or defines a combination and order of a plurality of speech types included in the speech section; the spoken text data includes an operator's utterance and a customer's utterance; The speech section classification rule classifies the speech section as a section including a customer's dissatisfaction or request when the speech section includes an utterance type indicating an utterance that the customer evaluates using a positive expression or an utterance type indicating an utterance that the customer evaluates using a negative expression. Speech segment classification device.

2. A speech section estimation unit that estimates a speech section from speech text data including speeches from two or more people; an utterance type estimation unit that estimates an utterance type for each utterance included in the utterance section estimated by the utterance section estimation unit; an utterance section classification unit that classifies the utterance sections estimated by the utterance section estimation unit using the utterance type of each utterance estimated by the utterance type estimation unit and a predetermined utterance section classification rule as a rule for classifying the utterance sections based on the utterance type; Equipped with the speech section classification rule determines whether a specific speech type is included in the speech section, or defines a combination and order of a plurality of speech types included in the speech section; the spoken text data includes an operator's utterance and a customer's utterance; The speech section classification rule classifies the speech section as a section including a customer's dissatisfaction or request if the speech section includes an utterance type indicating an utterance by the customer and the operator regarding a matter, and the utterance with the utterance type is also classified as an utterance type indicating a question uttered by the customer to the operator, an utterance type indicating an utterance by the customer expressing a request or a demand to the operator, an utterance type indicating an utterance by the customer answering or explaining a question uttered by the operator, or an utterance type indicating an utterance by the customer describing a negative situation. Speech segment classification device.

3. A speech section estimation unit that estimates a speech section from speech text data including speeches from two or more people; an utterance type estimation unit that estimates an utterance type for each utterance included in the utterance section estimated by the utterance section estimation unit; an utterance section classification unit that classifies the utterance sections estimated by the utterance section estimation unit using the utterance type of each utterance estimated by the utterance type estimation unit and a predetermined utterance section classification rule as a rule for classifying the utterance sections based on the utterance type; Equipped with the speech section classification rule determines whether a specific speech type is included in the speech section, or defines a combination and order of a plurality of speech types included in the speech section; the spoken text data includes an operator's utterance and a customer's utterance; The speech section classification rule classifies the speech section as a section including a customer's dissatisfaction or request if the speech section includes an utterance type indicating an utterance by the operator that describes a negative situation or an utterance type indicating an utterance by the operator that uses an expression that alleviates a negative situation, and if, within two utterances after the utterance with the utterance type, the utterance type includes either an utterance type indicating a question uttered by the customer to the operator or an utterance type indicating an utterance expressing a request or a demand from the customer to the operator. Speech segment classification device.

4. A speech section estimation unit that estimates a speech section from speech text data including speeches from two or more people; an utterance type estimation unit that estimates an utterance type for each utterance included in the utterance section estimated by the utterance section estimation unit; an utterance section classification unit that classifies the utterance sections estimated by the utterance section estimation unit using the utterance type of each utterance estimated by the utterance type estimation unit and a predetermined utterance section classification rule as a rule for classifying the utterance sections based on the utterance type; Equipped with the speech section classification rule determines whether a specific speech type is included in the speech section, or defines a combination and order of a plurality of speech types included in the speech section; the spoken text data includes an operator's utterance and a customer's utterance; The utterance section classification rule classifies the utterance section as an open-type sales section when the utterance section includes an utterance type indicating an utterance of the agent who hears about the customer's needs and the utterance type of the first utterance in the utterance section to hear about the customer's needs is an open question; If the speech section includes an utterance type indicating an utterance of the operator who hears about the customer's needs and the utterance type of the first hearing about the customer's needs within the utterance section is a theme question, the utterance section is classified as a theme-type sales section; If the speech section includes an utterance type indicating an utterance of the operator who hears about the customer's needs and the utterance type of the first hearing about the customer's needs within the speech section is an end question, the speech section is classified as an end-type sales section. Speech segment classification device.

5. Estimating a speech section from speech text data including speeches by two or more people; estimating an utterance type for each utterance included in the estimated utterance section; classifying the estimated speech sections using a speech type of each estimated utterance and a speech section classification rule that is predetermined as a rule for classifying the speech sections based on the utterance type; the speech section classification rule determines whether a specific speech type is included in the speech section, or defines a combination and order of a plurality of speech types included in the speech section; the spoken text data includes an operator's utterance and a customer's utterance; The speech section classification rule includes a process of classifying the speech section as a section including a customer's dissatisfaction or request when the speech section includes an utterance type indicating an utterance that the customer evaluates using a positive expression or an utterance type indicating an utterance that the customer evaluates using a negative expression: A computer-implemented method for speech segment classification.

6. Estimating a speech section from speech text data containing speech from two or more people, estimating an utterance type for each utterance included in the estimated utterance section; classifying the estimated speech sections using a speech type of each estimated utterance and a speech section classification rule that is predetermined as a rule for classifying the speech sections based on the utterance type; the speech section classification rule determines whether a specific speech type is included in the speech section, or defines a combination and order of a plurality of speech types included in the speech section; the spoken text data includes an operator's utterance and a customer's utterance; The speech section classification rule includes a process of classifying an utterance section as a section including a customer's dissatisfaction or request when the utterance section includes an utterance type indicating an utterance by the customer and the operator regarding a matter, and the utterance with the utterance type is also included in the utterance with the utterance type, which is an utterance type indicating a question uttered by the customer to the operator, an utterance type indicating an utterance by the customer expressing a request or a demand to the operator, an utterance type indicating an utterance by the customer answering or explaining a question uttered by the operator, or an utterance type indicating an utterance by the customer describing a negative situation, A computer-implemented method for speech segment classification.

7. Estimating a speech section from speech text data including speeches by two or more people; estimating an utterance type for each utterance included in the estimated utterance section; classifying the estimated speech sections using a speech type of each estimated utterance and a speech section classification rule that is predetermined as a rule for classifying the speech sections based on the utterance type; the speech section classification rule determines whether a specific speech type is included in the speech section, or defines a combination and order of a plurality of speech types included in the speech section; the spoken text data includes an operator's utterance and a customer's utterance; The speech section classification rule includes a process of classifying the speech section as a section including a customer's dissatisfaction or request when the speech section includes an utterance type indicating an utterance that the customer evaluates using a positive expression or an utterance type indicating an utterance that the customer evaluates using a negative expression: A speech segment classification program to be executed by a computer.

8. Estimating a speech section from speech text data including speeches from two or more people, estimating an utterance type for each utterance included in the estimated utterance section; classifying the estimated speech sections using a speech type of each estimated utterance and a speech section classification rule that is predetermined as a rule for classifying the speech sections based on the utterance type; the speech section classification rule determines whether a specific speech type is included in the speech section, or defines a combination and order of a plurality of speech types included in the speech section; the spoken text data includes an operator's utterance and a customer's utterance; The speech section classification rule includes a process of classifying an utterance section as a section including a customer's dissatisfaction or request when the utterance section includes an utterance type indicating an utterance by the customer and the operator regarding a matter, and the utterance with the utterance type is also included in the utterance with the utterance type, which is an utterance type indicating a question uttered by the customer to the operator, an utterance type indicating an utterance by the customer expressing a request or a demand to the operator, an utterance type indicating an utterance by the customer answering or explaining a question uttered by the operator, or an utterance type indicating an utterance by the customer describing a negative situation, A speech segment classification program to be executed by a computer.

Images