System, method, program, and recording medium recording program for extracting feature words common between unit conversation sets

Abstract

The present invention provides a system, method, program, and recording medium recording a program, which make it possible to identify which word was important for causing a change in inner thoughts of a conversation partner in a conversation.　Provided is a system for extracting feature words common between unit conversation sets, the system including: an utterance text generation unit for performing, for each unit conversation, voice recognition of voice signals of a conversation between a target person and a conversation partner of the target person to generate utterance text, the voice signals being acquired from the conversation between the target person and the conversation partner of the target person, and storing the utterance text in association with a time; an inner thought-related event detection unit for detecting, for each unit conversation, occurrence of an inner thought-related event of the target person on the basis of information derived from voice signals of the target person acquired from the conversation between the target person and the conversation partner of the target person, image signals of an image capturing the target person during the conversation between the target person and the conversation partner of the target person, and / or biological signals detected from the target person during the conversation between the target person and the conversation partner of the target person, and storing the detected inner thought-related event into a storage unit in association with a time at which the inner thought-related event occurred, the information being information other than the utterance text; a feature word extraction unit for extracting all or a part of content words in the utterance text within a range related to the detected inner thought-related event, and storing the extracted content words into the storage unit; and a common feature word extraction unit for extracting the extracted content words that are common among designated unit conversation sets.

Description

System, method, program for extracting common characteristic words between unit conversation sets, and recording medium having the program recorded thereon

[0001] The present invention relates to a system, method, program, and recording medium storing the program for extracting characteristic words common to unit conversation sets.

[0002] Patent Literature 1 proposes a technique for examining the number of words in an utterance in a conversation record and extracting, as desired content, conversation content corresponding to an utterance with a word count equal to or greater than a threshold. Patent Literature 2 proposes a technique for displaying and comparing conversation records side by side. Patent Literature 3 proposes a technique for improving the accuracy of case search by using text data of different cases to extract causal relationships from each text data and searching for cases that are similar to the extracted causal relationships.

[0003] Japanese Patent No. 6055804 Japanese Patent Application Laid-Open No. 2022-071874 Japanese Patent No. 5034580

[0004] However, with the technologies proposed in the above Patent Documents 1 to 3, it is difficult to know which words in a conversation triggered a change in the conversation partner's inner thoughts, in other words, which words were important in causing a change in the conversation partner's inner thoughts.

[0005] Therefore, one of the objects of the present invention is to provide a system, method, program, and recording medium on which the program is recorded, which make it possible to know which words in a conversation triggered a change in the inner thoughts of a conversation partner, in other words, which words were important in causing a change in the inner thoughts of a conversation partner.

[0006] One aspect of the present invention is a system for extracting characteristic words common to a set of unit conversations, the system comprising: a speech text generation unit that performs speech recognition on an audio signal of a conversation between a subject and a conversation partner of the subject, obtained from the conversation between the subject and the conversation partner of the subject, to generate an utterance text for each unit conversation, and stores the utterance text in association with time; and a speech signal generation unit that performs speech recognition on an audio signal of the subject obtained from the conversation between the subject and the conversation partner of the subject, an image signal of an image of the subject taken during the conversation between the subject and the conversation partner of the subject, and / or an image signal of an image of the subject taken during the conversation between the subject and the conversation partner of the subject, obtained from the conversation between the subject and the conversation partner of the subject, The system detects the occurrence of a mentally related event of the subject based on information obtained from bio-signals detected from the subject, and stores the mentally related event in correspondence with the time at which it occurred in a memory unit, wherein the information is information other than the spoken text; a characteristic word extraction unit extracts all or part of the content words in the spoken text within a range related to the detected mentally related event and stores them in a memory unit; and a common characteristic word extraction unit extracts the extracted content words that are common between specified unit conversation sets.

[0007] The information may include the spoken text.

[0008] The system may further include a common characteristic word display unit that displays the extracted content words that are common between the extracted specified unit conversation sets, and the common characteristic word display unit may display the relationship between the representative attributes of each unit conversation set and the extracted content words that are common between the specified unit conversation sets.

[0009] The mind-related events may be sympathy, antipathy, positive decision, and / or negative decision / indecision.

[0010] The range associated with the detected heart-related event may be a specified predetermined time range.

[0011] The range related to the detected inner thought-related event can be a sentence of the utterance text or a plurality of sentences adjacent to the sentence including the sentence immediately before the utterance at the time closest to the time when the inner thought-related event occurred, which is a sentence of the utterance text of the utterance or a plurality of sentences adjacent to the sentence including the sentence, when the similarity to the set of sentences corresponding to the inner thought-related event is compared with a predetermined standard value in a temporal order from the set of sentences corresponding to the inner thought-related event, and the range related to the sentence of the utterance text immediately before the sentence at which the similarity falls below the predetermined standard value a predetermined number of times in a row, which is a sentence of the utterance text of the utterance at the time closest to the time when the inner thought-related event occurred, or a plurality of sentences adjacent to the sentence including the sentence, when the similarity to the set of sentences corresponding to the inner thought-related event is compared with a predetermined standard value in a temporal order from the set of sentences corresponding to the inner thought-related event, and the range related to the sentence of the utterance text of the utterance at the time closest to the time when the inner thought-related event occurred can be a sentence of the utterance text or a plurality of sentences adjacent to the sentence including the sentence, when the similarity to the set of sentences corresponding to the inner thought-related event is compared with a predetermined standard value in a temporal order from the set of sentences corresponding to the inner thought-related event, and the range related to the sentence of the utterance text immediately before the sentence at which the similarity falls below the predetermined standard value a predetermined number of times in a row.

[0012] One aspect of the present invention provides a computer-implemented method for extracting characteristic words common to unit conversation sets, the method comprising: for each unit conversation, performing speech recognition on an audio signal of a conversation between a subject and the subject's conversation partner, obtained from the conversation between the subject and the subject's conversation partner, to generate spoken text, and storing the spoken text in association with time; for each unit conversation, detecting the occurrence of a mentally related event of the subject based on information obtained from the subject's audio signal obtained from the conversation between the subject and the subject's conversation partner, image signals of an image of the subject taken during the conversation between the subject and the subject's conversation partner, and / or bio-signals detected from the subject during the conversation between the subject and the subject's conversation partner, and storing the mentally related event in association with the time at which it occurred, wherein the information is information other than the spoken text; extracting and storing all or part of the content words in the spoken text within a range related to the mentally related event; and extracting the extracted content words that are common to specified unit conversation sets.

[0013] The information may include the spoken text.

[0014] The method may further include displaying the extracted content words that are common between the extracted specified unit conversation sets, and in the displaying, the display may be such that the relationship between each representative attribute of the unit conversation sets and the extracted content words that are common between the specified unit conversation sets is shown.

[0015] The inner mind-related events may be sympathy, antipathy, decision, and / or indecision.

[0016] The range associated with the detected heart-related event may be a specified predetermined time range.

[0017] The range related to the detected inner thought-related event can be a sentence of the utterance text or a plurality of sentences adjacent to the sentence including the sentence immediately before the utterance at the time closest to the time when the inner thought-related event occurred, which is a sentence of the utterance text of the utterance or a plurality of sentences adjacent to the sentence including the sentence, when the similarity to the set of sentences corresponding to the inner thought-related event is compared with a predetermined standard value in a temporal order from the set of sentences corresponding to the inner thought-related event, and the range related to the sentence of the utterance text immediately before the sentence at which the similarity falls below the predetermined standard value a predetermined number of times in a row, which is a sentence of the utterance text of the utterance at the time closest to the time when the inner thought-related event occurred, or a plurality of sentences adjacent to the sentence including the sentence, when the similarity to the set of sentences corresponding to the inner thought-related event is compared with a predetermined standard value in a temporal order from the set of sentences corresponding to the inner thought-related event, and the range related to the sentence of the utterance text of the utterance at the time closest to the time when the inner thought-related event occurred can be a sentence of the utterance text or a plurality of sentences adjacent to the sentence including the sentence, when the similarity to the set of sentences corresponding to the inner thought-related event is compared with a predetermined standard value in a temporal order from the set of sentences corresponding to the inner thought-related event, and the range related to the sentence of the utterance text immediately before the sentence at which the similarity falls below the predetermined standard value a predetermined number of times in a row.

[0018] One aspect of the present invention provides a program for causing a computer to execute the above method.

[0019] One aspect of the present invention provides a computer-readable recording medium on which the above program is recorded.

[0020] One aspect of the present invention provides a method for generating a system for extracting common characteristic words between unit conversation sets by installing the program on the computer.

[0021] According to the present invention having the above configuration, it is possible to provide a system, method, program, and recording medium having a program recorded thereon, which make it possible to know which words in a conversation triggered a change in the inner thoughts of a conversation partner, in other words, which words were important in causing a change in the inner thoughts of a conversation partner.

[0022] 1 is a diagram illustrating an overall configuration of a common characteristic word extraction system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating a hardware configuration of a common characteristic word extraction system according to an embodiment of the present invention. FIG. 3 is a schematic flowchart of an example of a common characteristic word extraction process according to an embodiment of the present invention. FIG. 4 is a flowchart of an example of a conversation history generation process according to an embodiment of the present invention. FIG. 5 is a flowchart of an example of a conversation history generation process according to an embodiment of the present invention. FIG. 6 is a diagram illustrating an example of a relationship between a conversation history 1, which is a history of a conversation between a user and a first customer displayed on a screen, and the elapsed time since the start of the conversation and detected points of empathy. FIG. 7 is a diagram illustrating an example of a relationship between a conversation history 2, which is a history of another conversation between the user and the first customer displayed on a screen, and the elapsed time since the start of the conversation and detected points of empathy. FIG. 8 is a diagram illustrating an example of a relationship between a conversation history 3, which is a history of a conversation between a user and a second customer displayed on a screen, and the elapsed time since the start of the conversation and detected points of empathy. FIG. 9 is a diagram illustrating an example of a relationship between a conversation history 4, which is a history of another conversation between the user and a second customer displayed on a screen, and the elapsed time since the start of the conversation and detected points of empathy. FIG. 10 is a flowchart of an example of a common characteristic word extraction and display process according to an embodiment of the present invention. FIG. 11 is a diagram illustrating a table of empathy words for the first customer according to an embodiment of the present invention. FIG. 12 is a diagram illustrating a table of empathy words for the second customer according to an embodiment of the present invention. 1 is a diagram showing an example of displaying characteristic words, common characteristic words, and representative attributes of a conversation history set for a specified conversation history set according to one embodiment of the present invention. FIG. 2 is a diagram showing another example of the relationship between the conversation history between a user and a first customer displayed on a screen, the elapsed time since the start of the conversation, and detected points of empathy. FIG. 3 is a flowchart of an example of a process for extracting all or part of content words in spoken text within a range related to a detected innermost related event according to a second embodiment of the present invention. FIG. 4 is a diagram showing the correspondence relationship between time, spoken sentence ID, spoken sentence, and similarity.

[0023] Hereinafter, one embodiment of the present invention will be described with reference to the drawings.

[0024] (First embodiment) In this embodiment, an example will be described in which empathetic words, which are characteristic words common to a conversation history set, which is a unit conversation set between a first customer who is a first target and a user who is a conversation partner of the first target, and a conversation history set, which is a unit conversation set between a second customer who is a second target and a user who is a conversation partner of the second target, are extracted.

[0025] 1 is a diagram showing the overall configuration of a common characteristic word extraction system according to one embodiment of the present invention. The common characteristic word extraction system 1 is connected via a network 2 to a user terminal 3, a first customer terminal 4 which is the terminal of a first customer with whom the user is conversing, a second customer terminal 5 which is the terminal of a second customer with whom the user is conversing, and an administrator terminal 6 which is the terminal of an administrator such as the user's boss. Each of the common characteristic word extraction system 1, the user terminal 3, the first customer terminal 4, the second customer terminal 5, and the administrator terminal 6 does not have to be configured as a single physical device, and may be configured as multiple physical devices.

[0026] The user terminal 3, the first customer terminal 4, the second customer terminal 5, and the administrator terminal 6 can be any suitable terminal equipped with a microphone and camera and having data communication capabilities, such as a PC, smartphone, or tablet terminal.

[0027] The common characteristic word extraction system 1 includes a voice / image / biological signal acquisition unit 101, a voice analysis unit 103, a spoken text generation unit 105, an inner-mind-related event detection unit 107, a characteristic word extraction unit 109, a common characteristic word extraction unit 111, a common characteristic word display unit 113, a conversation history management unit 115, and a memory unit 119. The common characteristic word extraction system 1 does not need to be configured as a single physical device, and may be configured as multiple physical devices.

[0028] The audio / image / biosignal acquisition unit 101 acquires audio signals of a conversation between the subject and the subject's conversation partner, image signals of an image of the subject captured during a conversation between the subject and the subject's conversation partner, image signals of an image of the subject's conversation partner during a conversation between the subject and the subject's conversation partner, biosignals detected from the subject during a conversation between the subject and the subject's conversation partner, and / or biosignals detected from the subject's conversation partner during a conversation between the subject and the subject's conversation partner.

[0029] The voice analysis unit 103 performs voice analysis of the voice signal of the conversation between the subject and the subject's conversation partner acquired by the voice / image / biosignal acquisition unit 101, detects speech sections, and performs speaker identification, thereby identifying the speech sections of the subject and the speech sections of the subject's conversation partner.

[0030] The spoken text generation unit 105 performs voice recognition on the audio signals of the speech section of the identified target person and the speech section of the target person's conversation partner to generate spoken text, and stores the spoken text in the memory unit 119 in association with the time and the conversation history ID.

[0031] The inner thoughts-related event detection unit 107 detects the occurrence of an inner thoughts-related event of the subject based on information obtained from the audio signal of the speech section of the identified subject, the audio signal of the speech section of the identified subject's conversation partner, the speech text generated by speech recognition of the audio signal of the speech section of the identified subject's conversation partner, the speech text generated by speech recognition of the audio signal of the speech section of the identified subject's conversation partner, the image signal of an image of the subject, the image signal of an image of the subject's conversation partner, the bio-signal detected from the subject, and / or the bio-signal detected from the subject's conversation partner, and stores the inner thoughts-related event and the time when the event occurred in association with each other in the memory unit 119.

[0032] The characteristic word extraction unit 109 extracts all or part of the content words in the utterance text within the range related to the detected innermost related event, and stores them in the storage unit 119 .

[0033] The common feature word extraction unit 111 extracts content words that are common between the specified unit conversation sets.

[0034] The common characteristic word display unit 113 displays the extracted content words that are common between the specified unit conversation sets, and displays them in a manner that shows the relationship between each representative attribute of the specified unit conversation sets and the extracted content words that are common between the specified unit conversation sets.

[0035] The conversation history management unit 115 issues a conversation history ID and stores it in the storage unit 119. Furthermore, at the end of the conversation, the conversation history management unit 115 stores in the storage unit 119 the customer name and conversation history ID input by the user, the generated utterance text, and the detected inner thought-related event in association with each other.

[0036] The storage unit 119 stores various types of information. The storage unit 119 may be configured as a single physical device, or may be distributed across multiple physical devices.

[0037] 2 is a diagram showing the hardware configuration of a common characteristic word extraction system according to a first embodiment of the present invention. The common characteristic word extraction system 1 includes a CPU 10a, a RAM 10b, a ROM 10c, an external memory 10d, an input unit 10e, an output unit 10f, and a communication unit 10g. The RAM 10b, the ROM 10c, the external memory 10d, the input unit 10e, the output unit 10f, and the communication unit 10g are connected to the CPU 10a via a system bus 10h.

[0038] The CPU 10a comprehensively controls each device connected to the system bus 10h.

[0039] The ROM 10c and the external memory 10d store the BIOS and OS, which are control programs for the CPU 10a, as well as various programs and data required to realize the functions executed by the computer.

[0040] The RAM 10b functions as the CPU's main memory, work area, etc. The CPU 10a loads programs and the like required for executing processes from the ROM 10c or the external memory 10d into the RAM 10b, and executes the loaded programs to realize various operations.

[0041] The external memory 10d is composed of, for example, a flash memory, a hard disk, a DVD-RAM, a USB memory, or the like.

[0042] The input unit 10e receives operation instructions, etc. from a user, etc. The input unit 10e is composed of input devices, such as an input button, a keyboard, a pointing device, a wireless remote control, a microphone, and a camera.

[0043] The output unit 10f outputs data processed by the CPU 10a and data stored in the RAM 10b, ROM 10c, and external memory 10d. The output unit 10f is composed of output devices such as an LCD, an organic EL panel, a printer, and a speaker.

[0044] The communication unit 10g is an interface for connecting and communicating with external devices via a network or directly, and is configured from interfaces such as a serial interface and a LAN interface.

[0045] Each part of the common characteristic word extraction system 1 is realized by various programs stored in ROM or external memory using the CPU, RAM, ROM, external memory, input unit, output unit, communication unit, etc. as resources.

[0046] Based on the above system configuration, an example of dialogue task support processing of the common characteristic word extraction system according to the first embodiment of the present invention will be described below with reference to FIGS. 1 to 12.

[0047] 3 is a schematic flowchart of an example of a common characteristic word extraction process according to the first embodiment of the present invention. The dialogue task support process of the common characteristic word extraction system according to the first embodiment of the present invention comprises a conversation history generation process (S1) and a common characteristic word extraction and display process (S2). Each process will be described in detail below.

[0048] <Conversation History Creation Process> FIGS. 4A and 4B are flowcharts illustrating an example of conversation history creation process according to the first embodiment of the present invention.

[0049] When a conversation between the user and the first customer begins, the conversation history management unit 115 issues a conversation history ID and stores it in the storage unit 119 (S101).

[0050] The audio signal of the speech of the user who is the target user's conversation partner input via the microphone (not shown) of the user terminal 3, the image signal of the user's image captured by the camera (not shown) of the user terminal 3, the audio signal of the speech of the first customer who is the target user's conversation partner received by the user terminal 3, and / or the image signal of the first customer who is the user's conversation partner received by the user terminal 3 are transmitted from the user terminal 3 to the common characteristic word extraction system 1 and acquired by the audio / image / biosignal acquisition unit 101 (S103). The audio signal and / or image signal acquired by the audio / image / biological signal acquisition unit 101 is not limited to this, and may be any other appropriate audio signal of a conversation between a user and the user's conversation partner, such as an audio signal of a conversation between a user and the user's conversation partner input via a microphone (not shown) of the user terminal 3, and / or an image signal of an image of the user and the user's conversation partner captured by the camera of the user terminal 3 in a conversation between the user and the user's conversation partner, an image signal of an image of the user captured in a conversation between the user and the user's conversation partner, and / or an image signal of an image of the user's conversation partner in a conversation between the user and the user's conversation partner. Furthermore, the audio signal of a conversation between a user and the user's conversation partner, the image signal of an image of the user captured in a conversation between the user and the user's conversation partner, and / or an image signal of an image of the user's conversation partner in a conversation between the user and the user's conversation partner acquired by the audio / image / biological signal acquisition unit 101 may be any other appropriate audio signal and / or image signal of a conversation between a user and any number of the user's conversation partners. In addition, the audio / image / biosignal acquisition unit 101 may acquire biosignals such as pulse, sweat, brain waves, etc. detected from the user during a conversation between the user and the user's conversation partner, and / or biosignals detected from the user's conversation partner during a conversation between the user and the user's conversation partner.

[0051] The voice analysis unit 103 performs voice analysis of the voice signal of the conversation between the user and the first customer acquired by the voice / image / biosignal acquisition unit 101, detects the speech section, and identifies the speaker, thereby identifying the user's speech section and the first customer's speech section (S105).

[0052] The speech text generation unit 105 performs speech recognition on the speech signals of the user's speech section and the first customer's speech section identified in step S105 to generate speech text, associates the speech text with time, and stores the time in the storage unit 119 in association with the conversation history ID (S107). The time can be, for example, the elapsed time from the start of the conversation, but is not limited to this, and any other appropriate time can be used. The association of the speech text with time can be, for example, by associating the time with the start time of each utterance sentence in the speech text, but is not limited to this, and can be performed by any other appropriate method.

[0053] Meanwhile, the inner thoughts-related event detection unit 107 detects the occurrence of an inner thoughts-related event of the first customer, who is the target, based on information obtained from the audio signal of the specified speech section of the first customer and / or the image signal of an image captured of the first customer, and stores the inner thoughts-related event, the time at which the inner thoughts-related event occurred, and the conversation history ID in association with each other in the storage unit 119 (S109). For example, when the intensity of the audio signal is equal to or greater than a predetermined intensity threshold and this state of being equal to or greater than the predetermined threshold continues for a predetermined period of time or longer, the inner thoughts-related event detection unit 107 detects that an event related to the inner thoughts state of "empathy" has occurred at the time when this state began.

[0054] FIG. 5 shows an example of the relationship between a conversation history 1, which is a history of a conversation between a user and a first customer, displayed on the screen (not shown) of the user terminal 3 or the administrator terminal 6, and the elapsed time from the start of the conversation, which is the time based on the start of the conversation, and the empathy point, which is the time when a detected empathy event occurred. The screen displays the utterances of the user and the first customer, converted into text data by the conversation history generation process described above, in chronological order. The user's utterances are displayed in speech bubbles from the left, and the first customer's utterances are displayed in speech bubbles from the right. The numbers written next to each speech bubble indicate the elapsed time from the start of the conversation; for example, "10:15" means 10 minutes and 15 seconds. Furthermore, a "empathy point" mark written next to a speech bubble indicates that an empathy event was detected at the time of the utterance in that speech bubble. These elapsed time and "empathy point" marks are not actually displayed, but are included to explain the relationship between each utterance.

[0055] 5, the occurrence of an empathy event (empathy point 1) is detected at the same time as the first customer's utterance, "I'd like to try using our system," at 10 minutes and 30 seconds. Also, the occurrence of an empathy event (empathy point 2) is detected at the same time as the first customer's utterance, "I understand, thank you," at 15 minutes and 20 seconds.

[0056] The detected inner thoughts-related events can be events related to any inner thoughts state, such as "antipathy," "positive decision (e.g., decision to purchase a product)," "negative decision / indecision (e.g., decision not to purchase a product or postponement of purchasing a product)," "anxiety," etc. Furthermore, the number of types of inner thoughts-related events to be detected can be one or more. The inner thoughts-related events can be detected using, for example, known emotion analysis techniques or machine learning.

[0057] Detection of heart-related events may use biosignals in addition to or instead of audio and / or image signals.

[0058] Furthermore, in addition to the audio signal, image signal, and / or biometric signal, the detection of a mental state-related event may use a speech text generated by speech recognition of the audio signal of the speech section of the identified first customer. For example, a word dictionary for determining empathy may be prepared, and the presence, number, frequency, etc. of words that match words registered in the word dictionary may be used as auxiliary data for determining empathy based on the audio signal, image signal, and / or biometric signal. Using such a configuration, the accuracy of determining the occurrence of a mental state-related event can be further improved.

[0059] When the conversation ends, the user inputs the name of the first customer via the user terminal 3, and the conversation history management unit 115 stores the name of the first customer input by the user, the conversation history ID, the generated utterance text, and the detected inner thought-related event in association with each other in the memory unit 119 (S111).

[0060] Thereafter, processing similar to steps S101 to S111 is performed for another conversation between the user and the first customer, another conversation between the user and the second customer, and another conversation between the user and the second customer. Speech text for the conversation between the user and the customer is generated and stored in the storage unit 119 in association with the time and the conversation history ID. A self-relevant event is detected and stored in the storage unit 119 in association with the time when the self-relevant event occurred. The customer's name and conversation history ID are also stored in the storage unit 119 in association with the event (S113). Examples of the relationship between these conversation histories displayed on the screens (not shown) of the user terminal 3 and the administrator terminal 6, the elapsed time from the start of the conversation (based on the start of the conversation) and the empathy point (the time when the detected empathy event occurred) are shown in Figures 6 to 8. Conversation history 2 corresponds to another conversation between the user and the first customer, conversation history 3 corresponds to a conversation between the user and the second customer, and conversation history 4 corresponds to another conversation between the user and the second customer.

[0061] <Common Characteristic Word Extraction and Display Processing> FIG. 9 is a flowchart showing an example of common characteristic word extraction and display processing according to the first embodiment of the present invention.

[0062] When a user or administrator specifies, on the user terminal 3 or administrator terminal 6, a conversation history set from which common characteristic words are to be extracted (S201), the user terminal 3 or administrator terminal 6 transmits data indicating the conversation history set specified by the user or administrator to the common characteristic word extraction system 1 (S203). Here, each of the specified conversation history sets is composed of conversation histories that share a common representative attribute, which is an attribute that representatively represents the attributes of the conversation histories. For example, in the example described below, a conversation history set composed of conversation histories whose representative attribute is "customer A" and a conversation history set composed of conversation histories whose representative attribute is "customer B" are specified. When multiple attributes are considered as attributes of the conversation histories, the representative attribute can be selected arbitrarily from among those multiple attributes.

[0063] Next, the characteristic word extraction unit 109 extracts all or part of the content words in the spoken text within a range related to the detected innermost related event for each conversation history included in the conversation history set specified by the user or administrator, and stores the extracted content words in the storage unit 119 (S205). In this embodiment, a case will be described in which a specified predetermined time range is used as the range related to the detected innermost related event.

[0064] As an example, a specific description will be given of a case where the conversation history sets specified by the user or administrator are a set of conversation histories between the user and a first customer (conversation history 1 and conversation history 2) and a set of conversation histories between the user and a second customer (conversation history 3 and conversation history 4). The conversation history set specified by the user or administrator may consist of a single conversation history or may consist of multiple conversation histories. Furthermore, the number of conversation history sets specified by the user or administrator can be any number greater than or equal to two.

[0065] In step S201 above, the user or administrator specifies a conversation history set from which common characteristic words are to be extracted, and also specifies a time range relative to the time when the internally related event occurred as a range related to the detected internally related event. Here, as an example, the time range specified is 15 seconds before and after the empathy point, which is the time when the empathy event occurred. The user terminal 3 or administrator terminal 6 transmits the specified time range from the user terminal 3 or administrator terminal 6 to the common characteristic word extraction system 1. Note that the initial time range may be specified by the characteristic word extraction unit 109 based on a predetermined time range prepared on the common characteristic word extraction system 1 side.

[0066] The conversation histories between the user and the first customer and the conversation histories between the user and the second customer stored in the storage unit 119 are conversation history 1, conversation history 2, and conversation history 3, and conversation history 4, respectively. Therefore, the characteristic word extraction unit 109 first extracts all or part of the content words in the speech text of the first customer within a specified time range of the first empathy point in conversation history 1 from conversation history 1 and conversation history 2, which are the sets of conversation histories between the user and the first customer, and stores them in the storage unit 119. In this example, the nouns and adverbs "among them," "linkage function," "mind," "our company," "system," "help page," and "aware" that are part of the content words present in the speech text of the first customer between 10 minutes 15 seconds and 10 minutes 45 seconds, which is a time range of 15 seconds before and after 10 minutes 30 seconds, are extracted as empathy words that are characteristic words in conversation history 1. Here, content words are words that have semantic content and contribute to the meaning of the sentence in which they appear. Typically, these words are nouns, verbs, adjectives, adjectival verbs, and adverbs. However, the extracted content words may be all parts of speech that belong to the content words, or only a portion of the parts of speech that belong to the content words. Furthermore, the extracted content words may be only a portion of the content words based on arbitrary criteria. Furthermore, for parts of speech that have conjugated forms, such as verbs, adjectives, and adjectival verbs, their base forms (original forms) may be extracted. Next, similarly, for conversation history 2 of conversation history 1 and conversation history 2, which are conversation history sets between the user and the first customer, the following words are extracted as empathetic words: "place," "help page," "solution," "recording function," "meeting," "thanks," "conference," "content," "confirmation," and "time." Empathetic word IDs are issued for the empathetic words in the conversation history sets between the user and the first customer in the order of extraction. The conversation history IDs, empathic point IDs, and empathetic words are associated with each other and stored in the storage unit 119 as a first customer empathetic word table 801, as shown in FIG. 10 .

[0067] If empathetic words are not successfully extracted, the characteristic word extraction unit 109 may change and re-specify the time range. This is because, for example, the speaking speed of speakers varies depending on the conversation, and unless an appropriate time range is set accordingly, empathetic words may not be successfully extracted as characteristic words for the desired conversation history. By providing a configuration for specifying the time range, an appropriate time range can be set depending on the conversation. The time range may be specified based on input by a human such as a user, or may be specified by the system, for example, while referring to the results of empathetic word extraction.

[0068] Next, the same process is performed for conversation history 3 and conversation history 4, which are sets of conversation histories between the user and the second customer, and a second customer empathy word table 803 as shown in Figure 11 is generated and stored in the memory unit 119.

[0069] The common characteristic word extraction unit 111 extracts the extracted content words that are common between the specified unit conversation sets and stores them in the storage unit 119 (S207). In this example, the common characteristic word extraction unit 111 extracts "collaboration function" and "conference," which are common empathic words between the first customer and the second customer, from among the empathic words stored in the empathic word table 801 and the empathic word table 803, and stores them in the storage unit 119 as common empathic words, which are common characteristic words.

[0070] The common characteristic word display unit 113 displays the extracted content words common to the specified unit conversation sets, and displays the association between the representative attributes of each of the specified unit conversation sets and the extracted content words common to the specified unit conversation sets (S209). In this example, the display is as shown in FIG. 12. That is, the names of the first customer, "Customer A," and the second customer, "Customer B," which are the representative attributes of each of the specified conversation history sets, are displayed. The empathetic words extracted from the conversation history set between the user and the first customer are displayed around the name of the first customer, "Customer A." A line is displayed between "Customer A" and the empathetic words extracted from the conversation history set between the user and the first customer, thereby indicating the association between the empathetic words extracted from the conversation history set between the user and the first customer and "Customer A." The name of the second customer, "Customer B," and the empathetic words extracted from the conversation history set between the user and the second customer are displayed in a similar manner. Then, for the extracted content words common to the specified unit conversation sets, that is, the common empathy words extracted from the conversation history sets of the user and the first customer and the common empathy words extracted from the conversation history sets of the user and the second customer, "collaboration function" and "conference," a straight line is displayed between both "Customer A" and "Customer B," indicating that these words are common to both conversation history sets.

[0071] According to this embodiment, the occurrence of a mentally-related event is detected not only based on spoken text data but also based on information obtained from the subject's voice signal acquired from a conversation between the subject and the subject's conversation partner, the image signal of an image of the subject captured during the conversation between the subject and the subject's conversation partner, and / or information obtained from a biosignal detected from the subject during the conversation between the subject and the subject's conversation partner, thereby enabling highly accurate detection of the occurrence of a mentally-related event. For example, in conversation history 3 and conversation history 4, the word "okay" is included in the utterance, but "okay" in conversation history 4 means "no," so a method of determining whether a word in the utterance matches a word registered in the empathy word dictionary as described above will result in an erroneous detection.

[0072] In addition to this effect, this embodiment can extract characteristic words related to inner thoughts that are common across multiple conversation history sets, making it possible to highlight characteristic words that are more highly correlated with specific representative attributes.

[0073] Second Embodiment In the first embodiment, when extracting all or a part of the content words in the speech text within a range related to the detected inner thought-related event, a specified predetermined time range was used as the range related to the detected inner thought-related event. However, in the present embodiment, the range related to the detected inner thought-related event is a configuration in which the similarity to a set of sentences corresponding to an inner thought-related event, which is a sentence of the speech text of the utterance closest to the time when the inner thought-related event occurred, or a plurality of sentences including that sentence and adjacent to that sentence, is compared with a predetermined reference value in a temporally forward direction from the set of sentences corresponding to an inner thought-related event, and the range related to the detected inner thought-related event is a sentence of the speech text or a plurality of sentences including that sentence and adjacent to that sentence, which is a sentence of the speech text immediately before the time when the inner thought-related event occurred, is compared with a predetermined reference value in a temporally backward ... similarity to the set of sentences corresponding to an inner thought-related event is below the predetermined reference value in a predetermined number of consecutive times. Since this embodiment is otherwise similar to the first embodiment, only the configuration for extracting all or part of the content words in the utterance text within the range related to the detected inner-mind-related event will be described. The overall configuration, functional configuration, and hardware configuration of the common characteristic word extraction system according to the second embodiment are similar to those of the first embodiment.

[0074] As an example, a case will be described in which the conversation history between the user and the first customer is as shown in FIG.

[0075] FIG. 14 is a flowchart of an example process for extracting all or part of content words in spoken text within a range associated with a detected innermost event according to the second embodiment of the present invention.

[0076] In step S105 of the above-described conversation history generation process, the conversation history management unit 115 issues an utterance ID for each utterance sentence in the order of utterance, associates it with time, and stores it in the storage unit 119. The correspondence between the elapsed time (time), utterance sentence ID, and utterance sentence is as shown in Fig. 15. The occurrence of a sympathy event was detected during the utterance of the first customer, "I'd like to try using our system," with utterance sentence ID 32 at 5 minutes 40 seconds.

[0077] The characteristic word extraction unit 109 selects a sentence from the utterance text of the utterance closest to the time when the innermost related event occurred (S301). In this example, the sentence "I'd like to try using our company's system" by the first customer with an utterance sentence ID of 32 is selected.

[0078] The characteristic word extraction unit 109 calculates the similarity of each sentence in the utterance text to the sentence in the selected utterance text, and stores the calculated similarity in the storage unit 119 in association with the time, utterance sentence ID, and utterance sentence (S303). In this example, the correspondence between the time, utterance sentence ID, utterance sentence, and similarity is as shown in Fig. 15. The calculation of the similarity can be performed using a known method such as a method using machine learning such as BERT.

[0079] The characteristic word extraction unit 109 compares the similarity of the sentences of the utterance text with a predetermined reference value in a temporally forward direction from the selected sentence of the utterance text, and compares the similarity of the sentences of the utterance text with the predetermined reference value in a temporally backward direction from the selected sentence of the utterance text (S305). The characteristic word extraction unit 109 sets the sentences of the utterance text between the sentence of the utterance text immediately before whose similarity falls below the predetermined reference value a predetermined number of consecutive times in a temporally forward comparison and the sentence of the utterance text immediately before whose similarity falls below the predetermined reference value a predetermined number of consecutive times in a temporally backward comparison, as a range related to the detected innermost thought-related event (S307). In this example, if the specified number of times is two and the specified reference value is 0.6, the similarity between utterance sentence IDs 28 and 27 falls below 0.6 twice in a row, and the similarity between utterance sentence IDs 33 and 34 falls below 0.6 twice in a row. Therefore, utterances with utterance sentence IDs 29 to 32 are set as the range related to the detected empathy event.

[0080] The characteristic word extraction unit 109 extracts all or part of the content words from the sentences of the spoken text within the range set in step S307 and stores them in the storage unit 119 (S309). In this example, nouns and adverbs that are part of the content words are extracted from the sentences of the spoken text within the range set in step S307, and the content words present in the spoken sentences with spoken sentence IDs 29 to 32, such as "That's all," "App," "Function overview," "Kindness," "Function," "Especially," "Kindness," "Linkage function," "Our company," and "System," are extracted.

[0081] In the above embodiment, the calculation of similarity and comparison with a predetermined reference value were performed on a sentence-by-sentence basis, but the calculation may also be performed on a multiple-sentence basis for either or both of the sentences in the selected spoken text and the sentences in the spoken text.

[0082] While the present invention has been described with respect to several embodiments for illustrative purposes, it will be apparent to those skilled in the art that the present invention is not limited thereto and that various changes and modifications in form and detail may be made therein without departing from the scope and spirit of the invention.

[0083] 1 Common characteristic word extraction system 101 Voice / image / biological signal acquisition unit 103 Voice analysis unit 105 Speech text generation unit 107 Inner-mind related event detection unit 109 Characteristic word extraction unit 111 Common characteristic word extraction unit 113 Conversation history management unit 119 Storage unit 10a CPU 10b RAM 10c ROM 10d External memory 10e Input unit 10f Output unit 10g Communication unit 10h System bus 2 Network 3 User terminal 4 First customer terminal 5 Second customer terminal 6 Administrator terminal 801 First customer's empathy word table 803 Second customer's empathy word table

Claims

1. A system for extracting characteristic words common to a set of unit conversations, comprising: a spoken text generation unit which performs speech recognition on a voice signal of a conversation between a subject and the subject's conversation partner, obtained from the conversation between the subject and the subject's conversation partner, for each unit conversation, to generate a spoken text, and stores the spoken text in association with time; an inner thoughts related event detection unit which detects the occurrence of an inner thoughts related event of a subject based on information obtained from the voice signal of the subject obtained from the conversation between the subject and the subject's conversation partner, an image signal of an image of the subject during the conversation between the subject and the subject's conversation partner, and / or a biological signal detected from the subject during the conversation between the subject and the subject's conversation partner, and stores the inner thoughts related event in association with the time when the inner thoughts related event occurred in a memory unit, wherein the information is information other than the spoken text; and a characteristic word extraction unit which extracts all or part of the content words in the spoken text within a range related to the detected inner thoughts related event, and stores the extracted content words in the memory unit. A common feature word extraction unit that extracts the extracted content words that are common between specified unit conversation sets.

2. The system of claim 1, wherein the information includes the spoken text.

3. The system described in claim 1 or 2, further comprising a common characteristic word display unit that displays the extracted content words that are common between the extracted specified unit conversation sets, wherein the common characteristic word display unit displays a relationship between the representative attributes of each of the unit conversation sets and the extracted content words that are common between the specified unit conversation sets.

4. The system according to any one of claims 1 to 3, wherein the inner mind-related event is sympathy, antipathy, positive decision, and / or negative decision / indecision.

5. A system according to any one of claims 1 to 4, wherein the range associated with the detected heart-related event is a specified predetermined time range.

6. The system described in any one of claims 1 to 5, wherein the range related to the detected inner thought-related event is a sentence of the utterance text or a plurality of sentences adjacent to the sentence including the sentence, from the sentence of the utterance text immediately before the time when the inner thought-related event occurred, or a plurality of sentences adjacent to the sentence including the sentence, when the similarity to the set of sentences corresponding to the inner thought-related event is compared with a predetermined standard value in a forward chronological order from the set of sentences corresponding to the inner thought-related event, and the range to the sentence of the utterance text immediately before the sentence or a plurality of sentences adjacent to the sentence including the sentence, when the similarity to the set of sentences corresponding to the inner thought-related event is compared with a predetermined standard value in a backward chronological order from the set of sentences corresponding to the inner thought-related event, and the range to the sentence of the utterance text immediately before the sentence or a plurality of sentences adjacent to the sentence including the sentence, when the similarity to the set of sentences corresponding to the inner thought-related event is compared with a predetermined standard value in a backward chronological order from the set of sentences corresponding to the inner thought-related event, and 7. A method executed by a computer for extracting characteristic words common between unit conversation sets, comprising: for each unit conversation, performing speech recognition on a voice signal of a conversation between a subject and the subject's conversation partner, obtained from the conversation between the subject and the subject's conversation partner, to generate a spoken text, and storing the spoken text in association with a time; for each unit conversation, detecting the occurrence of a mind-related event of the subject based on information obtained from the subject's voice signal obtained from the conversation between the subject and the subject's conversation partner, an image signal of an image of the subject during the conversation between the subject and the subject's conversation partner, and / or a bio-signal detected from the subject during the conversation between the subject and the subject's conversation partner, and storing the mind-related event in association with the time when the mind-related event occurred, wherein the information is information other than the spoken text; extracting and storing all or a part of the content words in the spoken text within a range related to the mind-related event; and extracting the extracted content words that are common between specified unit conversation sets.

8. The method of claim 7, wherein the information includes the spoken text.

9. A method according to claim 7 or 8, further comprising displaying the extracted content words common between the specified unit conversation sets, wherein in said displaying, the association between the representative attributes of each of the unit conversation sets and the extracted content words common between the specified unit conversation sets is shown.

10. The method according to any one of claims 7 to 9, wherein the mind-related events are sympathy, antipathy, decision, and / or non-decision.

11. A method according to any one of claims 7 to 10, wherein the range associated with the detected heart-related event is a specified predetermined time range.

12. A method according to any one of claims 7 to 11, wherein the range related to the detected inner thought-related event is a sentence of the utterance text or a plurality of sentences adjacent to the sentence including the sentence, from the sentence of the utterance text immediately before the time when the inner thought-related event occurred, or a plurality of sentences adjacent to the sentence including the sentence, when the similarity to the set of sentences corresponding to the inner thought-related event is compared with a predetermined standard value in a chronological order from the set of sentences corresponding to the inner thought-related event, and the sentence of the utterance text immediately before the sentence ...

13. A program for causing a computer to execute the method according to any one of claims 7 to 12.

14. A computer-readable recording medium having the program according to claim 13 recorded thereon.

15. A method for generating a system for extracting common characteristic words between unit conversation sets by installing the program according to claim 13 on said computer.

Images