Communication support device, communication support system, communication support method, and program

Abstract

To support communication considering properties of a plurality of participants by using a communication support device.SOLUTION: A communication support device for supporting communication by a plurality of participants includes: an input unit for receiving input of utterance in the communication; an utterer specification unit for specifying a participant who has performed the input utterance of the plurality of participants; an identification unit for identifying a property of the participant specified by the utterer specification unit; a generation unit for generating a response content on the basis of the property of the specified participant and a content of the utterance; and a control unit for performing control so as to output the response content generated by the generation unit.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a communication support device, a communication support system, a communication support method, and a program. [Background technology]

[0002] There are dialogue systems in which a dialogue agent automatically responds to user utterances. There is also known a technology for improving speech recognition accuracy by generating training data according to the recording environment of the user's speech (see, for example, Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0003] In conventional technologies, for example, AI speakers, call centers, or chatbots respond to questions without considering the attributes of the speaker, making it difficult to support communication that takes into account the attributes (roles, titles, positions, specific industries or line of business, occupations, gender, etc.) of multiple participants in, for example, business negotiations.

[0004] An embodiment of the present invention has been made in consideration of the above-mentioned problems, and aims to support communication that takes into account the attributes of multiple participants using a communication support device. [Means for solving the problem]

[0005] In order to solve the above problem, one embodiment of a communication support device is a communication support device that supports communication between multiple participants, and includes an input unit that accepts input of utterances in the communication, a speaker identification unit that identifies the participant among the multiple participants who made the input utterance, an identification unit that identifies the attributes of the participant identified by the speaker identification unit, a generation unit that generates a response content based on the attributes of the identified participant and the content of the utterance, and a control unit that controls the response content generated by the generation unit to be output. [Effects of the Invention]

[0006] According to one embodiment of the present invention, a communication support device can be used to support communication that takes into account the attributes of multiple participants. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a diagram illustrating an example of a system configuration of a communication support system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer according to an embodiment. [Figure 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a terminal device according to an embodiment. [Figure 4] 1 is a diagram illustrating an example of a functional configuration of a communication support system according to a first embodiment. [Figure 5] 10 is a flowchart illustrating an example of a support process according to the first embodiment. [Figure 6] 10 is a flowchart showing an example of a response process to a customer according to the first embodiment. [Figure 7] FIG. 2 is a diagram showing an image of a description of a commercial product according to the first embodiment. [Figure 8] 10 is a flowchart showing an example of a response process to a sales representative according to the first embodiment. [Figure 9]10 is a flowchart showing an example of a response process to a question according to the first embodiment. [Figure 10] FIG. 10 is a diagram showing an image of a business negotiation according to the second embodiment. [Figure 11] 10 is a flowchart illustrating an example of a support process according to the second embodiment. [Figure 12] 10 is a flowchart showing an example of a response process to an utterance according to the second embodiment. [Figure 13] FIG. 10 is a diagram illustrating an example of a display screen of a terminal device according to the second embodiment. [Figure 14] 10 is a flowchart illustrating an example of a support process according to the third embodiment. [Figure 15] 13 is a flowchart showing an example of a support process according to the fourth embodiment. [Figure 16] FIG. 10 is a diagram illustrating an example of a functional configuration of a communication support system according to a fifth embodiment. [Figure 17] 13 is a flowchart illustrating an example of a feedback collection process according to the fifth embodiment. [Figure 18] FIG. 13 is a diagram illustrating an example of a database according to the fifth embodiment. [Figure 19] 13 is a flowchart showing an example of a content determination process according to the fifth embodiment. [Figure 20] FIG. 13 is a diagram illustrating an example of a functional configuration of a communication support system according to a sixth embodiment. [Figure 21] 13 is a flowchart showing an example of a support process according to the sixth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. <System configuration> Fig. 1 is a diagram showing an example of a system configuration of a communication support system according to an embodiment. In the example of Fig. 1, the communication support system 1 includes a communication support device 10 and a terminal device 100, which are connected to a communication network N such as the Internet and a LAN (Local Area Network).

[0009] As an example of a usage scenario, a sales representative 2 brings a terminal device 100 displaying a virtual salesperson 101 to a business negotiation with a customer 3, and conducts the business negotiation with the virtual salesperson 101. Here, the business negotiation is an example of communication. The sales representative 2 is an example of a host participating in the communication, and the customer 3 is an example of a guest participating in the communication. The virtual salesperson 101 is an example of a dialogue agent.

[0010] The terminal device 100 is an information terminal used by the sales representative 2, such as a PC (Personal Computer), tablet terminal, or smartphone. The terminal device 100 acquires the speech of the sales representative 2 and the customer 3, who are participants in the business negotiation, and transmits the acquired speech (voice data) to the communication support device 10. Note that the speech is an example of utterances in communication. Utterances in communication include, for example, utterances in the form of text data such as chat. Here, the following explanation will be given assuming that the utterances in communication are utterances.

[0011] The communication support device (server device) 10 is, for example, an information processing device having a computer configuration, or a system including multiple computers. The communication support device 10 acquires the speech transmitted by the terminal device 100, analyzes the acquired speech, and generates a response content that proposes recommendation information according to the needs of the customer 3. This response content includes, for example, linguistic information and non-linguistic information of the virtual sales 101, the recommendation information, and an explanation of the recommendation information.

[0012] Furthermore, the communication support device 10 controls the virtual salesperson 101 displayed on the terminal device 100 in accordance with the content of this response. For example, the communication support device 10 controls the speech of the virtual salesperson 101 based on the linguistic information of the virtual salesperson 101. Furthermore, the communication support device 10 controls the display of the virtual salesperson 101 and the output of paralanguage, etc., based on non-linguistic information of the virtual salesperson 101. Here, the non-linguistic information includes, for example, one or more pieces of information such as the facial expression, gaze, posture, emotions, behavior, and paralanguage of the virtual salesperson 101. Furthermore, the paralanguage includes non-linguistic audio information such as tone of voice, speaking rate, voice pitch, voice intensity, throat clearing, sighs, laughter, or silence.

[0013] The terminal device 100 displays a virtual salesperson 101 controlled by the communication support device 10, and outputs the speech and paralanguage of the virtual salesperson 101. This allows the virtual salesperson 101 to propose recommendation information to the customer 3, the salesperson 2, etc., according to the needs of the customer 3, for example, in accordance with the flow of business negotiations.

[0014] As a specific example, if it is determined that the need of customer 3 is "digitization of invoices," the virtual salesperson 101 proposes merchandise related to an electronic invoice issuing system, saying, "If you are looking for electronic invoices, how about product A?" Furthermore, the virtual salesperson 101 may explain the merchandise by showing the basis for the proposal, for example, saying, "Product A is the best-selling product this year in the manufacturing industry in the prefecture," or may compare it with other products.

[0015] Preferably, the communication support device 10 generates different response content for each speaker depending on the acquired speech. For example, when the speaker is salesperson 2, virtual salesperson 101 displays materials for a product to be proposed to customer 3 on the terminal device 100 and waits for instructions from salesperson 2. This allows salesperson 2 to explain the product based on the materials displayed by virtual salesperson 101, or to have virtual salesperson 101 explain the product. On the other hand, when the speaker is customer 3, virtual salesperson 101 signals that there is a product to propose, for example, by a gesture such as raising his or her hand. This allows salesperson 2 to have virtual salesperson 101 propose a product when a proposal by virtual salesperson 101 is necessary, and to suppress the proposal by virtual salesperson 101 when a proposal by virtual salesperson 101 is not necessary.

[0016] As described above, the communication support system 1 according to this embodiment makes it easy to support communication that takes into account the attributes of multiple participants by using a dialogue system such as the virtual sales 101. Here, communication refers to a means of conveying information, opinions, emotions, etc. between individuals or groups. This means includes linguistic information using words and non-linguistic information such as gestures, facial expressions, and body language.

[0017] Note that business negotiations are an example of communication. The communication support system 1 may also support other types of communication, such as meetings, interviews, lectures, lessons, medical care, nursing care, or counseling. Furthermore, the virtual sales 101 and the dialogue agent are examples of dialogue systems. <Hardware configuration> (Computer hardware configuration) The communication support device 10 has, for example, the hardware configuration of a computer 200 as shown in Fig. 2. Alternatively, the communication support device 10 is configured by a plurality of computers 200. Furthermore, the terminal device 100 may have, for example, the hardware configuration of a computer 200 as shown in Fig. 5.

[0018] Fig. 2 is a diagram showing an example of the hardware configuration of a computer according to an embodiment. For example, as shown in Fig. 2, a computer 200 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, a HD (Hard Disk) 204, an HDD (Hard Disk Drive) controller 205, a display 206, an external device connection I / F (Interface) 207, a network I / F 208, a keyboard 209, a pointing device 210, a DVD-RW (Digital Versatile Disk Rewritable) drive 212, a media I / F 214, and a bus line 215.

[0019] Furthermore, when the computer 200 is the terminal device 100, the computer 200 further includes a microphone 221, a speaker 222, an audio input / output I / F 223, a CMOS (Complementary Metal Oxide Semiconductor) sensor 224, an image sensor I / F 225, and the like.

[0020] Of these, the CPU 201 controls the overall operation of the computer 200. The ROM 202 stores programs used to start up the computer 200, such as an IPL (Initial Program Loader). The RAM 203 is used, for example, as a work area for the CPU 201. The HD 204 stores programs such as an OS (Operating System), applications, and device drivers, as well as various data. The HDD controller 205 controls the reading and writing of various data from and to the HD 204, for example, under the control of the CPU 201. The HD 204 and the HDD controller 205 are examples of storage devices.

[0021] The display 206 displays various types of information such as a cursor, a menu, a window, characters, or an image. The display 206 may be provided outside the computer 200. The external device connection I / F 207 is an interface for connecting various external devices to the computer 200. The network I / F 208 is an interface for connecting the computer 200 to a communication network N and communicating with other devices.

[0022] The keyboard 209 is a type of input means having multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 210 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The keyboard 209 and pointing device 210 may be provided outside the computer 200.

[0023] The DVD-RW drive 212 controls reading and writing of various data from and to a DVD-RW 211, which is an example of a removable recording medium. The DVD-RW 211 is not limited to a DVD-RW, and may be another removable recording medium. The media I / F 214 controls reading and writing (storing) of data from and to a medium 213 such as a flash memory. The bus line 215 includes an address bus, a data bus, various control signals, and the like for electrically connecting the above components.

[0024] The microphone 221 is a built-in circuit that converts sound into an electrical signal. The speaker 222 is a built-in circuit that converts the electrical signal into physical vibrations to produce sound such as music or voice. The sound input / output I / F 223 is a circuit that processes input and output of sound signals between the microphone 221 and the speaker 222 under the control of the CPU 201.

[0025] The CMOS sensor 224 is a type of built-in imaging means that captures an image of a subject (e.g., a self-portrait) and obtains image data under the control of the CPU 201. Note that the computer 200 may have an imaging means such as a CCD (Charge Coupled Device) sensor instead of the CMOS sensor 224. The imaging element I / F 225 is a circuit that controls the driving of the CMOS sensor 224.

[0026] (Example of hardware configuration of terminal device) 3 is a diagram illustrating an example of the hardware configuration of a terminal device according to an embodiment. Here, an example of the hardware configuration of the terminal device 100 will be described in the case where the terminal device 100 is an information terminal such as a smartphone or a tablet terminal.

[0027] In the example of Figure 3, the terminal device 100 includes a CPU 301, a ROM 302, a RAM 303, a storage device 304, a CMOS sensor 305, an image sensor I / F 306, an acceleration / direction sensor 307, a media I / F 309, and a GPS (Global Positioning System) receiving unit 310.

[0028] Of these, the CPU 301 controls the overall operation of the terminal device 100 by executing a predetermined program. The ROM 302 stores a program used to start up the CPU 301, such as an IPL. The RAM 303 is used as a work area for the CPU 301. The storage device 304 is a large-capacity storage device that stores programs such as an OS and applications, various types of data, and the like, and is realized by, for example, an SSD (Solid State Drive), a flash ROM, or the like.

[0029] The CMOS sensor 305 is a type of built-in imaging means that captures an image of a subject (mainly a self-portrait) under the control of the CPU 301 to obtain image data. Note that the terminal device 100 may have an imaging means such as a CCD sensor instead of the CMOS sensor 305. The imaging element I / F 306 is a circuit that controls the driving of the CMOS sensor 305. The acceleration / azimuth sensor 307 is one of various sensors, such as an electronic magnetic compass or gyrocompass that detects geomagnetism, and an acceleration sensor. The media I / F 309 controls the reading or writing (storage) of data from or to a media (storage media) 308, such as a flash memory. The GPS receiver 310 receives GPS signals (positioning signals) from GPS satellites.

[0030] The terminal device 100 also includes a long-distance communication circuit 311, an antenna 311a of the long-distance communication circuit 311, a CMOS sensor 312, an image sensor I / F 313, a microphone 314, a speaker 315, an audio input / output I / F 316, a display 317, an external device connection I / F 318, a short-distance communication circuit 319, an antenna 319a of the short-distance communication circuit 319, and a touch panel 320.

[0031] Of these, the long-distance communication circuit 311 is a circuit that communicates with other devices, for example, via the communication network N. The CMOS sensor 312 is a type of built-in imaging means that captures an image of a subject and obtains image data under the control of the CPU 301. The image sensor I / F 313 is a circuit that controls the driving of the CMOS sensor 312. The microphone 314 is a built-in circuit that converts sound into an electrical signal. The speaker 315 is a built-in circuit that converts an electrical signal into physical vibrations to generate sound such as music or voice. The sound input / output I / F 316 is a circuit that processes the input and output of sound signals between the microphone 314 and the speaker 315 under the control of the CPU 301.

[0032] The display 317 is a type of display means such as a liquid crystal display or organic electroluminescence (EL) display that displays an image of a subject, various icons, etc. The external device connection I / F 318 is an interface for connecting various external devices. The short-range communication circuit 319 includes a circuit for performing short-range wireless communication. The touch panel 320 is a type of input means that allows a user to operate the terminal device 100 by pressing the display 317.

[0033] The terminal device 100 also includes a bus line 321. The bus line 321 includes an address bus, a data bus, various control signals, and the like for electrically connecting the components such as the CPU 301 shown in FIG.

[0034] 3 is an example of the hardware configuration of the terminal device 100. The terminal device 100 may have other hardware configurations as long as it has a computer configuration, a communication circuit, a display, a microphone, a speaker, and the like.

[0035] <Functional configuration> FIG. 4 is a diagram illustrating an example of a functional configuration of the communication support system 1 according to an embodiment.

[0036] (Functional configuration of terminal device) The terminal device 100 realizes, for example, each functional configuration as shown in Fig. 4 by the CPU 301 (or CPU 201) included in the terminal device 100 executing a predetermined program. In the example of Fig. 4, the terminal device 100 has a data transmission unit 421, a display unit 422, an audio output unit 423, an operation reception unit 424, etc.

[0037] The data transmission unit 421 acquires the speech of the participants participating in the communication, and executes a data transmission process to transmit the acquired speech (voice data) to the communication support device 10. In the example of FIG. 1, the data transmission unit 421 acquires the speech of the sales representative 2 and the customer 3 participating in the business negotiation, and transmits the acquired speech to the communication support device 10. The data transmission unit 421 also transmits operation data and the like accepted by the operation acceptance unit 424 to the communication support device 10. The speech of the participants is an example of what the participants say.

[0038] The display unit 422 executes a display process to display the video of the dialogue agent, proposal materials, etc. received from the communication support device 10. For example, the display unit 422 displays the video of the virtual sales 101 as shown in FIG. 1 and recommendation information to be proposed to the customer 3, etc., on the terminal device 100.

[0039] The voice output unit 423 executes a voice output process for outputting the voice of the dialogue agent received from the communication support device 10. The operation reception unit 424 executes an operation reception process for receiving an operation on the terminal device 100.

[0040] (Functional configuration of communication support device) The communication support device 10 realizes, for example, a functional configuration as shown in Fig. 4 by causing the computer 200 included in the communication support device 10 to execute a predetermined program stored in a storage medium. In the example of Fig. 4, the communication support device 10 has functional components such as an input unit 401, a voice recognition unit 402, a speaker identification unit 403, an identification unit 404, a response unit 405, a voice synthesis unit 410, a drawing unit 411, and an output unit 412. Note that at least a part of the above functional components may be realized by hardware.

[0041] The communication support device 10 also implements a storage unit 413 using storage devices such as the HD 204 and the HDD controller 205. The storage unit 413 may be implemented by a storage server provided outside the communication support device 10, a cloud service, or the like.

[0042] The input unit 401 executes an input process for accepting input of utterances in communication. In the example of Fig. 1, the input unit 401 accepts input of the speech of the salesperson 2 and the speech of the customer 3 transmitted by the terminal device 100.

[0043] The voice recognition unit 402 performs voice recognition processing on the utterance received by the input unit 401. For example, the voice recognition unit 402 performs known voice recognition processing on the utterance received by the input unit 401 to convert the acquired utterance into text. Note that if the utterance in communication input to the input unit 401 is text data, the communication support device 10 does not need to include the voice recognition unit 402.

[0044] The speaker identification unit 403 executes a speaker identification process to identify the participant who made the statement input to the input unit 401 from among the multiple participants. In the example of FIG. 1, the speaker identification unit 403 may identify the statement made by sales representative 2 using a known speaker recognition technology or the like, and identify any statement other than that made by sales representative 2 as that made by customer 3. Alternatively, the speaker identification unit 403 may identify the statements made by sales representative 2 and a support representative using a known speaker recognition technology or the like, and identify any statement other than that made by sales representative 2 and a support representative as that made by customer 3.

[0045] If the utterance in the communication input to the input unit 401 is text data, the speaker identification unit 403 may determine the participant who made the utterance based on the terminal device that input the text. For example, if the text data is input from the terminal device 100 of sales representative 2, the participant who made the utterance may be determined to be sales representative 2, and if the text data is input from another terminal device, the participant who made the utterance may be determined to be customer 3.

[0046] The identification unit 404 executes an identification process for identifying the attributes of the participant identified by the speaker identification unit 403. Note that it is sufficient for the identification unit 404 to identify the attributes of the participant who made the remark, and it is not essential for the speaker identification unit 403 to identify the participant. For example, the identification unit 404 may analyze the characteristics of the spoken voice or the context of the speech to identify whether the speaker is the sales representative 2 or the customer 3.

[0047] Furthermore, the communication support device 10 may be configured to be set in advance as to which attribute (role) the voice of a participant corresponds to, such as customer, sales representative, or support representative. In this case, the identification unit 404 can identify the attribute from the voice of the participant who spoke. If it is difficult to register the voice of the customer in advance, only the sales representative and support representative may be set in the communication support device 10, and the other voices may be automatically set to the attribute (role) of the customer and the attribute of the participant who spoke.

[0048] Alternatively, the communication support device 10 may be configured to register in advance the profile information of the participants (role, job title, position, specific industry, line of business, occupation, gender, etc.). In this case, the identification unit 404 may identify the attribute of the participant who made the comment (sales representative 2, customer 3, etc.) based on the profile information of the speaker identified by the speaker identification unit 403. If it is difficult to register the profile information of the customer in advance, the attribute of the participant who made the comment may be automatically set to the attribute (role) of the customer when the profile information of the participant who made the comment is not registered.

[0049] Alternatively, the communication support device 10 may collect voice data from various speakers in advance, analyze voice characteristics such as tone, intonation, speed, and language accent from the collected voice data, and use machine learning to create a model for identifying attributes. The pre-collected voice data may include speakers with various roles, genders, ages, occupations, and social backgrounds. In this case, the identification unit 404 can identify the attributes of the participants who spoke by inputting their voices into the machine-learned model.

[0050] Alternatively, the communication support device 10 may collect voice data from various speakers in advance, extract and analyze language patterns and expressions related to each category from the collected voice data, and use machine learning to create a model that identifies attributes. In this case, the identification unit 404 inputs the voice of the participant who spoke into the machine-learned model, and can identify the attribute of the participant who spoke as a salesperson, for example, based on specific phrases and technical terms used by salespeople.

[0051] Alternatively, the communication support device 10 may be configured to register in advance account information used by participants when logging in to a PC, a smart device, etc. In this case, the identification unit 404 can identify the attributes of the speaker by using attribute information associated with the account information of the participant who made a speech or a chat.

[0052] The response unit 405 executes a response process to respond to a utterance input to the input unit 401. The response unit 405 includes, for example, a determination unit 406, an intention interpretation unit 407, a generation unit 408, a control unit 409, and the like.

[0053] The determination unit 406 executes a determination process for determining recommendation information to be proposed to a guest based on, for example, the speech of the host or the speech of the guest participating in the communication. For example, in FIG. 1, the determination unit 406 determines products and the like to be proposed to the customer 3 based on the speech of the sales representative 2 or the speech of the customer 3 that has been converted into text by the speech recognition unit 402.

[0054] The intention interpretation unit 407 analyzes the speech (hereinafter referred to as the speech text) converted into text by the speech recognition unit 402 using a natural language processing (NLP) algorithm, and extracts, for example, the intention of the speech, keywords, areas of interest, etc.

[0055] The generation unit 408 executes a generation process to generate different response contents for each speaker according to the speech voice acquired by the input unit 401. For example, when the speech voice acquired by the input unit 401 is the speech voice of sales representative 2, the generation unit 408 generates a response content for sales representative 2. This response content includes information such as verbal responses, non-verbal responses, and proposal materials. The verbal responses include the speech content to be spoken by the virtual salesperson 101. The non-verbal responses include one or more pieces of information such as facial expressions, gaze, posture, emotions, behavior, and paralanguage. The proposal materials include materials for the products to be proposed to the customer 3 (for example, web pages, image data, document data, interactive media, etc.).

[0056] To generate the linguistic and non-linguistic information, natural language processing using, for example, a rule-based or large-scale language model can be used. As an example of a large-scale language model, a sentence generation language model called GPT-3 (Generative Pre-trained Transformer 3) can be applied. In addition, in rule-based natural language processing, the response content of the virtual salesperson 101 is generated based on pre-defined rules for the response content in response to the user's utterance intent.

[0057] For example, the generation unit 408 may acquire the proposal materials for the product determined by the determination unit 406 from a database of proposal materials prepared in advance. Alternatively, the generation unit 408 may collect information on the web about the product determined by the determination unit 406 and create the proposal materials. Alternatively, the proposal materials may be prepared by the determination unit 406.

[0058] The control unit 409 executes a control process for controlling the dialogue agent in accordance with the response content generated by the generation unit 408. For example, the control unit 409 outputs a verbal response included in the response content to a speech synthesis unit 410, outputs a non-verbal response to a drawing unit 411, and outputs a proposal document to an output unit 412, thereby generating a video of the virtual sales 101.

[0059] The speech synthesis unit 410 executes speech synthesis processing to convert the input verbal response into speech using speech synthesis technology. The drawing unit 411 executes drawing processing to draw a dialogue agent according to the input non-verbal response. For example, the drawing unit 411 reflects facial expressions, gaze, posture, emotions, behavior, paralanguage, etc. in the drawing of the virtual salesperson 101 according to the non-verbal response.

[0060] Preferably, the drawing unit 411 also performs lip-sync drawing, which moves the mouth of the virtual salesperson 101 in accordance with the speech of the virtual salesperson 101.

[0061] The output unit 412 executes an output process to output a video including the voice of the dialogue agent converted into voice by the voice synthesis unit 410, the dialogue agent drawn by the drawing unit 411, and the proposal materials to the terminal device 100 etc. For example, the output unit 412 transmits a video of the virtual sales 101 to the terminal device 100 via the communication network N.

[0062] The system configuration of the communication support system 1 shown in Fig. 4 is an example. For example, the communication support system 1 may be configured by one terminal device 100 having the functional configuration of the communication support device 10 shown in Fig. 4. In this case, the terminal device 100 serves as the communication support device. Furthermore, at least some of the functional configurations of the communication support device 10 may be included in the terminal device 100. For example, the terminal device 100 may include a voice synthesis unit 410, a drawing unit 411, an output unit 412, and the like.

[0063] <Processing flow> Next, the processing flow of the communication support method according to the present embodiment will be described by exemplifying a number of embodiments.

[0064] [First embodiment] (Support processing) FIG. 5 is a flowchart showing an example of support processing according to the first embodiment. This processing shows an example of support processing in which the communication support device 10 supports communication. Here, an example of processing will be described in which the communication support device 10 supports a business negotiation between a salesperson 2 and a customer 3 using a virtual sales 101 displayed on the terminal device 100, as shown in FIG. 1. For example, the communication support device 10 repeatedly executes the processing of FIG. 5 during the business negotiation. Note that the following description will be given assuming that the utterances in the communication input to the input unit 401 are the spoken voices of the participants.

[0065] In step S501, the input unit 401 of the communication support device 10 acquires the speech of the participants in the business negotiation. For example, the input unit 401 acquires the speech of the salesperson 2 and the speech of the customer 3 transmitted from the terminal device 100.

[0066] In step S502, the communication support device 10 performs speech recognition and speaker identification on the speech acquired by the input unit 401. For example, the speech recognition unit 402 converts the speech acquired by the input unit 401 into text to create a speech text. Furthermore, the speaker identification unit 403 identifies the participant who spoke the speech acquired by the input unit 401, and the identification unit 404 identifies whether the participant who spoke is the sales representative 2 or the customer 3.

[0067] In step S503, the response unit 405 of the communication support device 10 determines whether the device is in question mode. For example, if the utterance text converted by the speech recognition unit 402 contains a wake word for transitioning to question mode, the response unit 405 determines that the device is in question mode, and if the wake word is not contained, the response unit 405 determines that the device is not in question mode. If the device is in question mode, the response unit 405 transitions the process to step S504. On the other hand, if the device is not in question mode, the response unit 405 transitions the process to step S505. Note that the question mode is merely an example and may not be essential to the configuration of the present application.

[0068] When the process proceeds from step S503 to S504, the response unit 405 executes a process of responding to the question. The process of responding to the question will be described later.

[0069] When the process proceeds from step S503 to step S505, the response unit 405 determines whether the speaker is customer 3. If the speaker is customer 3, the response unit 405 shifts the process to step S506. On the other hand, if the speaker is not customer 3, the response unit 405 shifts the process to step S507.

[0070] When the process proceeds from step S505 to step S506, the response unit 405 executes a response process for the customer 3. The details of the response process for the customer 3 will be described later.

[0071] When the process moves from step S505 to step S507, the response unit 405 determines whether the speaker is sales representative 2. If the speaker is sales representative 2, the response unit 405 moves the process to step S508.

[0072] On the other hand, if the speaker is not the sales representative 2, the response unit 405 ends the processing in Fig. 5. In this way, it is desirable that the communication support device 10 can be set so as not to take into account the remarks of a speaker who is neither the sales representative 2 nor the customer 3.

[0073] When the process moves from step S507 to step S508, the response unit 405 executes a response process corresponding to the sales representative 2. The content of the response process for the sales representative 2 will be described later.

[0074] (Response processing to customers) 6 is a flowchart showing an example of a response process to a customer according to the first embodiment. This process shows an example of a response process to the customer 3 executed by the communication support device 10 in step S506 in FIG.

[0075] In step S601, the determination unit 406 determines whether or not a problem or a product name is included in the utterance of the customer 3. If a problem or a product name is included in the utterance of the customer 3, the determination unit 406 shifts the process to step S602. On the other hand, if a problem or a product name is not included in the utterance of the customer 3, the determination unit 406 ends the process of FIG. 6.

[0076] When the process proceeds to step S602, the determination unit 406 determines a product to be proposed to the customer 3 based on the content of the utterance of the customer 3. For example, if the utterance of the customer 3 includes a problem, the determination unit 406 determines a product to solve the problem from a database prepared in advance as the product to be proposed to the customer 3. Furthermore, if the utterance of the customer 3 includes a product name, the determination unit 406 determines a product of the product name included in the utterance or a product related to the product name included in the utterance as the product to be proposed to the customer 3. Note that, since it would be problematic if the determination unit 406 were to propose any competing product, when determining a product based on the content of the utterance, the determination unit 406 proposes a predetermined product (a product that the determination unit wants to sell) from a database prepared in advance.

[0077] In step S603, the generation unit 408 generates a response content to the utterance of the customer 3. For example, the generation unit 408 generates a verbal response and a non-verbal response of the virtual salesperson 101 to explain the product determined by the determination unit 406.

[0078] In step S604, the generation unit 408 searches for or generates materials for the product to be proposed to the customer 3.

[0079] In step S605, the control unit 409 outputs a non-verbal response for displaying a cue indicating that a suggestion is being made to the drawing unit 411. As a result, the cue is displayed on the terminal device 100.

[0080] The signal may be, for example, an action indicating that there is a proposal, such as having the virtual salesperson 101 raise his / her hand or waving his / her hand, or may be an utterance such as "I have a proposal." Alternatively, the signal may be something other than an action of the virtual salesperson 101, such as displaying materials to be proposed on the terminal device 100.

[0081] In step S606, the control unit 409 determines whether a predetermined operation has been received within a predetermined time. For example, when the virtual salesperson 101 signals that he or she has a proposal, the salesperson 2 can allow the virtual salesperson to explain the product by performing a predetermined operation (for example, pressing a button). If the predetermined operation has been received within the predetermined time, the control unit 409 transitions the process to step S607. On the other hand, if the predetermined operation has not been received within the predetermined time, the control unit 409 terminates the process of FIG. 6.

[0082] In step S607, the generation unit 408 generates, for example, a catchphrase that summarizes the proposed content. Note that this process is optional and not essential.

[0083] In step S608, the control unit 409 displays the materials for the product prepared in step S804 on the terminal device 100, and has the virtual salesperson 101 explain the product using the response content generated in step S603 and the catchphrase generated in step S607.

[0084] FIG. 7 is a diagram showing an example of a commercial product explanation according to the first embodiment. For example, the communication support device 10 determines that the issue of customer 3 is "digital invoices" based on spoken text 701 as shown in FIG. 7. In this case, the control unit 409 displays materials 711 for product A to be proposed to customer 3, as shown on a display screen 710, and causes the virtual salesperson 101 to explain product A 712. Preferably, the control unit 409 explains product A while showing a basis 721 for the proposal, as shown on a display screen 720.

[0085] If the sales representative 2 does not need the proposal by the virtual sales representative 101, he or she can cancel the proposal by the virtual sales representative 101 by not performing the predetermined operation in step S606 of FIG.

[0086] (Response processing for sales representatives) 8 is a flowchart showing an example of a response process to the sales representative 2 according to the first embodiment. This process shows an example of a response process to the sales representative 2 executed by the communication support device 10 in step S508 in FIG.

[0087] In step S801, the determination unit 406 determines whether or not the problem or product name is included in the utterance of the sales representative 2. If the problem or product name is included in the utterance of the sales representative 2, the determination unit 406 shifts the process to step S802. On the other hand, if the problem or product name is not included in the utterance of the sales representative 2, the determination unit 406 ends the process of FIG.

[0088] In step S802, the determination unit 406 determines the product to be proposed to the customer 3 based on the content of the utterance of the sales representative 2. For example, if the utterance of the sales representative 2 includes a problem, the determination unit 406 determines, from a database prepared in advance, a product that solves the problem as the product to be proposed to the customer 3. Furthermore, if the utterance of the sales representative 2 includes a product name, the determination unit 406 determines, as the product to be proposed to the customer 3, a product of the product name included in the utterance or a product related to the product name included in the utterance.

[0089] In step S803, the generation unit 408 generates a response to the utterance of the salesperson 2. For example, the generation unit 408 generates a verbal response and a non-verbal response of the virtual salesperson 101 to explain the product determined by the determination unit 406.

[0090] In step S804, the generation unit 408 searches for or generates materials for the product to be proposed to the customer 3.

[0091] In step S805, the control unit 409 causes the terminal device 100 to display the materials for the product prepared in step S804.

[0092] In step S806, the control unit 409 determines whether the virtual salesperson 101 will explain the product. For example, if the salesperson 2 performs a predetermined operation (for example, a button operation or a voice signal) within a predetermined time, the control unit 409 determines that the virtual salesperson 101 will explain the product. On the other hand, if the salesperson 2 does not perform a predetermined operation within the predetermined time, the control unit 409 determines that the virtual salesperson 101 will not explain the materials.

[0093] If the virtual sales representative 101 determines that the materials will be explained, the control unit 409 shifts the process to step S807. On the other hand, if the virtual sales representative 101 determines that the materials will not be explained, the control unit 409 ends the process of Fig. 8. In this case, the sales representative 2 may explain the product to the customer 3 using the materials displayed on the terminal device 100.

[0094] In step S807, the generation unit 408 generates, for example, a catchphrase that summarizes the proposed content. Note that this process is optional and not essential.

[0095] In step S808, the control unit 409 causes the virtual salesperson 101 to explain the product using, for example, the materials displayed in step S805 and the catchphrase generated in step S807.

[0096] In this way, the communication support system 1 can generate different response contents for each speaker in accordance with the speech contents of the participants in the business negotiation.

[0097] (Response processing to questions) 9 is a flowchart showing an example of a process of responding to a question according to the first embodiment. This process shows an example of a process of responding to a question executed by the communication support device 10 in step S504 in FIG.

[0098] In step S901, the intention interpretation unit 407 understands the content of the question from the spoken text that has been converted into text by the voice recognition unit 402.

[0099] In step S902, the generation unit 408 generates a response to the question. For example, if the intention interpretation unit 407 understands the question, the generation unit 408 generates a response that answers the question. This response includes, for example, a verbal response and a non-verbal response. On the other hand, if the intention interpretation unit 407 does not understand the question, the communication support device 10 may cancel the processing from step S903 onwards.

[0100] In step S903, the generation unit 408 searches for or creates materials to be used in answering the question. Note that if no materials are required to answer the question, the generation unit 408 may omit the process of step S903.

[0101] In step S904, the control unit 409 displays the materials generated by the generation unit 408 on the terminal device 100, and causes the virtual salesperson 101 to answer the questions.

[0102] 5, 6, 8, and 9, the virtual sales person 101 can propose products that meet the needs of the customer 3 during the business negotiation, and can also answer questions from the sales representative 2 or the customer 3. Furthermore, the virtual sales person 101 can respond differently to each speaker depending on the speech of the participants participating in the business negotiation.

[0103] [Second embodiment] Fig. 10 is a diagram showing an image of a business negotiation according to the second embodiment. In the example of Fig. 1, the communication support system 1 displays a display screen on the terminal device 100 to be shown to the sales representative 2 and the customer 3. As another example, the communication support system 1 may have a function to display a display screen 1001 on the terminal device 100 to be shown only to the sales representative 2, as shown in Fig. 10. This enables the communication support system 1 to provide the sales representative 2 with more specific information that the sales representative 2 does not want to show to the customer 3, for example.

[0104] The functional configuration of the communication support system 1 according to the second embodiment may be the same as that of the first embodiment.

[0105] (Support processing) Fig. 11 is a flowchart showing an example of support processing according to the second embodiment. This processing shows another example of support processing in which the communication support device 10 supports communication. Here, an example of processing will be described in which the communication support device 10 displays a display screen for the sales representative 2 on the terminal device 100, as shown in Fig. 10. For example, the communication support device 10 repeatedly executes the processing of Fig. 11 during a business negotiation.

[0106] The basic processing contents are the same as the support processing according to the first embodiment described with reference to FIG. 5, and therefore detailed description of the processing contents that are the same as those according to the first embodiment will be omitted here.

[0107] In step S1101, the input unit 401 of the communication support device 10 acquires the speech of the participants in the business negotiation.

[0108] In step S1102, the communication assistance apparatus 10 performs voice recognition and speaker identification on the speech acquired by the input unit 401.

[0109] In step S1103, the response unit 405 of the communication support device 10 determines whether the mode is question mode. If the mode is question mode, the response unit 405 shifts the process to step S1104. On the other hand, if the mode is not question mode, the response unit 405 shifts the process to step S1105.

[0110] When the process proceeds from step S1103 to S1104, the response unit 405 executes a response process to the question. Note that the response process to the question according to the second embodiment may be the same as the response process to the question according to the first embodiment described with reference to FIG.

[0111] When the process moves from step S1103 to step S1105, the response unit 405 determines whether or not the mode is the customer's voice mode. Here, the customer's voice mode is a mode in which the virtual sales 101 provides recommendation information on products and the like only in response to the utterances of the customer 3, ignoring the utterances of the sales representative 2. The sales representative 2 can set the terminal device 100 to the customer's voice mode, for example, by using a setting button or the like displayed on the display screen.

[0112] If the mode is the customer's voice mode, the response unit 405 shifts the process to step S1106. On the other hand, if the mode is not the customer's voice mode, the response unit 405 shifts the process to step S1107.

[0113] In step S1106, the response unit 405 determines whether the speaker of the utterance voice acquired by the input unit 401 is customer 3. If the speaker is customer 3, the response unit 405 shifts the process to step S1107. On the other hand, if the speaker is not customer 3, the response unit 405 ends the process of FIG. 11.

[0114] In step S1107, the response unit 405 executes a response process to the utterance.

[0115] (Response processing to speech) Fig. 12 is a flowchart showing an example of a process of responding to an utterance according to the second embodiment. This process shows an example of a process of responding to an utterance executed by the communication support device 10 in step S1107 in Fig. 11. Note that detailed description of the process contents similar to those in the first embodiment will be omitted here.

[0116] In step S1201, the determination unit 406 determines whether the utterance includes a problem or a product name. If the mode is customer voice mode, the "utterance" includes the utterance of customer 3. If the mode is not customer voice mode, the "utterance" includes the utterance of sales representative 2 and the utterance of customer 3. If the utterance includes a problem or a product name, the determination unit 406 transitions the process to step S1202. On the other hand, if the utterance does not include a problem or a product name, the determination unit 406 terminates the process of FIG. 12.

[0117] In step S1202, the determination unit 406 determines the product to be proposed to the customer 3 based on the content of the utterance.

[0118] In step S1203, the generating unit 408 generates data related to the product determined by the determining unit 406. This data is not shown to the customer 3, and may therefore include detailed data such as sales data related to the product or comparison data with products from other companies.

[0119] In step S1204, the control unit 409 causes the terminal device 100 to display the data generated by the generation unit 408. For example, the control unit 409 causes the terminal device 100 to display a display screen 1300 as shown in FIG.

[0120] Fig. 13 is a diagram showing an example of a display screen of a terminal device according to the second embodiment. This diagram shows an example of a display screen 1300 for a sales representative displayed by the terminal device 100. In the example of Fig. 13, the display screen 1300 for a sales representative displays data 1301 generated by the generation unit 408, a "Listen to customer feedback" button 1302, a "Publish to customer" button 1303, and the like.

[0121] The data 1301 generated by the generation unit 408 includes, for example, detailed information on popular products related to the electronic invoice system, such as the number of sales of product A in the last three months and information on the industry of the customer.

[0122] Furthermore, on the display screen 1300, the sales representative 2 can set or cancel the customer voice mode by selecting the "Listen to customer voice" button 1302. Furthermore, the sales representative 2 can switch the display screen 1300 for the sales representative to a display screen 710 that can be made public to the customer 3, for example, as shown in FIG. 7, by selecting the "Open to customer" button 1303.

[0123] In step S1205, the control unit 409 determines whether the virtual salesperson 101 will explain the product. For example, if the salesperson 2 performs a predetermined operation (for example, a button operation or a voice signal) within a predetermined time, the control unit 409 determines that the virtual salesperson 101 will explain the product. On the other hand, if the salesperson 2 does not perform a predetermined operation within the predetermined time, the control unit 409 determines that the virtual salesperson 101 will not explain the materials.

[0124] If the virtual salesperson 101 determines that the materials will be explained, the control unit 409 shifts the process to step S1206. On the other hand, if the virtual salesperson 101 determines that the materials will not be explained, the control unit 409 ends the process of Fig. 12. In this case, the salesperson 2 can provide a detailed explanation of the product (e.g., product A) to the customer 3 while looking at the detailed data 1301 displayed on the salesperson display screen 1300.

[0125] In step S1206, the generation unit 408 generates a response to the utterance. For example, the generation unit 408 generates a verbal response and a non-verbal response of the virtual salesperson 101 to explain the product determined by the determination unit 406.

[0126] In step S1207, the generation unit 408 searches for or generates materials for the product to be proposed to the customer 3.

[0127] In step S1208, the control unit 409 displays the generated material on the terminal device 100. For example, the control unit 409 displays the generated material 711 on a display screen 710 that can be made public to the customer 3, as shown in FIG.

[0128] In step S1208, the control unit 409 causes the virtual salesperson 101 to explain the product using, for example, the materials displayed in step S1207.

[0129] In this way, the communication support system 1 can display the sales representative display screen 1300 to the sales representative 2, and provide more detailed information about the product.

[0130] [Third embodiment] The communication support system 1 can distinguish whether the speaker is a salesperson 2 or a customer 3, and can provide different responses to the same topic depending on the speaker. For example, when salesperson 2 says, "This product is highly energy efficient," the virtual salesperson 101 can provide comparative information with competitors' products, and when customer 3 says the same thing, the virtual salesperson 101 can provide specific advantages and use examples of the product.

[0131] Furthermore, the communication support system 1 can analyze the questions or responses of the customer 3 and display product flyers or the like according to the customer's interests or needs. For example, if the customer 3 says, "I'm interested in eco-friendly products," the virtual sales 101 can automatically present flyers and information about environmentally friendly products.

[0132] 14 is a flowchart showing an example of the support process according to the third embodiment. This process shows another example of the communication support process executed by the communication support system 1.

[0133] In step S1401, the input unit 401 of the communication support device 10 acquires the speech of the participants in the business negotiation. For example, the input unit 401 acquires the speech of the salesperson 2 and the speech of the customer 3 transmitted by the terminal device 100.

[0134] In step S1402, the communication support device 10 performs speech recognition and speaker identification on the speech acquired by the input unit 401. For example, the speech recognition unit 402 converts the speech acquired by the input unit 401 into text to create a speech text. Furthermore, the speaker identification unit 403 identifies the participant who spoke the speech acquired by the input unit 401, and the identification unit 404 identifies whether the participant who spoke is the sales representative 2 or the customer 3.

[0135] In step S1403, the response unit 405 determines whether the speaker is customer 3. If the speaker is customer 3, the response unit 405 shifts the process to step S1405. On the other hand, if the speaker is not customer 3, the response unit 405 shifts the process to step S1408.

[0136] When the process proceeds from step S1403 to step S1404, the response unit 405 analyzes the content (response policy) from the speaker (customer 3) and the content of the utterance.

[0137] In step S1405, the response unit 405 determines appropriate content for the speaker (customer 3) from the database. For example, the determination unit 406 selects relevant flyers, information, etc. from the database based on the speaker's interests, needs, etc.

[0138] In step S1406, the response unit 405 generates a response content for the speaker (customer 3). This response content is customized to include, for example, information about specific advantages and usage examples of the product.

[0139] In step S1407, the response unit 405 controls the virtual sales 101 in accordance with the generated response content.

[0140] When the process proceeds from step S1403 to step S1408, the response unit 405 determines whether the speaker is sales representative 2. If the speaker is sales representative 2, the response unit 405 proceeds to step S1409. On the other hand, if the speaker is not sales representative 2, the response unit 405 ends the process of FIG.

[0141] In step S1409, the response unit 405 analyzes the content (response policy) from the speaker (sales representative 2) and the content of the speech.

[0142] In step S1410, the response unit 405 determines appropriate content for the speaker (sales representative 2) from the database. For example, the response unit 405 selects relevant flyers, information, etc. from the database based on the speaker's interests, needs, etc.

[0143] In step S1411, the response unit 405 generates a response content for the speaker (sales representative 2). This response content is customized to include, for example, the positioning of the product in the market, or information on a competitive comparison.

[0144] In step S1412, the response unit 405 controls the virtual sales 101 in accordance with the response content generated by the generation unit 408.

[0145] By the process of FIG. 14, the communication support system 1 can identify whether the speaker is the sales representative 2 or the customer 3, and can give different responses depending on the speaker, even for the same topic.

[0146] [Fourth embodiment] The communication support system 1 can change the type of information provided, the way it is expressed, etc., based on the role of the speaker (salesperson 2, customer 3, decision maker, etc.). For example, if customer 3 asks about a specific function, the virtual salesperson 101 can provide the customer with an answer that focuses on the merits of the product, the user experience, etc. Furthermore, if salesperson 2 asks about the same specific function, the virtual salesperson 101 can provide more technical detailed information or advice on sales strategies, etc.

[0147] 15 is a flowchart showing an example of the support process according to the fourth embodiment. This process shows an example of the communication support process executed by the communication support system 1.

[0148] Of the processes shown in FIG. 15, the processes of steps S1401 to S1406 and steps S1408 to S1411 are the same as the support process according to the third embodiment described with reference to FIG. 14, and therefore will not be described here.

[0149] In step S1501, the response unit 405 adjusts the expression method for the response content generated in step S1406 to suit the role of the speaker (customer 3).

[0150] In step S1502, the control unit 409 controls the virtual sales 101 in accordance with the response content adjusted by the response unit 405 to suit the role of the speaker (customer 3).

[0151] In step S1503, response unit 405 adjusts the response content generated in step S1411 to a manner of expression that matches the role of the speaker (sales representative 2).

[0152] In step S1504, the control unit 409 controls the virtual sales 101 in accordance with the response content that the response unit 405 has adjusted to suit the role of the speaker (salesperson 2).

[0153] By the process of FIG. 15, the communication support system 1 can change the type of information to be provided, the way of expression, etc., based on the role of the speaker (sales representative 2, customer 3, decision maker, etc.).

[0154] [Fifth embodiment] <Functional configuration> Fig. 16 is a diagram showing the functional configuration of a communication support system 1 according to the fifth embodiment. The communication support device 10 according to the embodiment in Fig. 5 has a feedback unit 1601 in addition to the functional configuration of the communication support device 10 according to the first embodiment described in Fig. 4.

[0155] The feedback unit 1601 is realized by, for example, a program executed by the CPU 201. The feedback unit 1601 executes a feedback process to determine whether the proposed recommendation information was effective or not, and to provide feedback to the response unit 405. For example, the feedback unit 1601 analyzes voice data during a business negotiation, determines which pieces of information were effective or ineffective, and provides feedback to the response unit 405. The response unit 405 also continuously learns based on this feedback and generates more effective response content. For example, if the explanation of a particular product is well received by customer 3, the response unit 405 will adopt that style in other business negotiations.

[0156] The communication support system 1 may analyze the reactions and feedback of each speaker during the dialogue and adaptively adjust the next response. For example, if a certain customer 3 shows a price-sensitive reaction, the communication support system 1 may make a proposal to the customer 3 that focuses on cost efficiency, while providing advice to the sales representative 2 regarding price negotiation strategies or the presentation of alternative proposals.

[0157] The functional configuration of the communication support system 1 according to the fifth embodiment, other than the feedback unit 1601, may be the same as the functional configuration of the communication support system 1 according to the first embodiment described with reference to FIG.

[0158] <Processing flow> (Feedback collection process) 17 is a flowchart showing an example of a feedback collection process according to the fifth embodiment. Note that this process can be implemented in combination with the support process according to each of the other embodiments.

[0159] In step S1701, the communication support device 10 executes a response process for controlling the virtual sales 101 in accordance with the generated response content. This process corresponds to, for example, the processes in steps S1407 and S1412 in FIG.

[0160] In step S1702, the feedback unit 1601 determines whether the effect of the response content is positive. For example, after the response process, the feedback unit 1601 analyzes the speaker's speech in real time to determine whether the response content was effective. If the effect of the response content is positive, the feedback unit 1601 shifts the process to step S1703. On the other hand, if the effect of the response content is not positive, the feedback unit 1601 shifts the process to step S1704.

[0161] When the process proceeds to step S1703, the feedback unit 1601 stores the response content as learning data of successful cases, for example, in a database 1800 as shown in Fig. 18. On the other hand, when the process proceeds to step S1704, the feedback unit 1601 stores the response content as learning data of unsuccessful cases in the database 1800. This database 1800 is stored in, for example, the storage unit 413.

[0162] Fig. 18 is a diagram showing an example of a database according to the fifth embodiment. In the example of Fig. 18, the database 1800 includes information such as items such as "ID," "speaker identification," "utterance content," "content," "response content," "output device," "output format," "analysis result (customer response)," "analysis result (sales response)," and "effect of response content."

[0163] "ID" is identification information that identifies each record stored in database 1800. "Speaker identification" is information that indicates whether the speaker is sales representative 2 or customer 3. "Utterance content" is spoken text that indicates the content of the speaker's utterance in response to the response process. "Content" is information that indicates proposal materials, etc. for the response process. "Response content" is information that indicates the response content of the response process. "Output format" is information that indicates the format of proposal materials, etc. "Analysis result" is information that indicates the analysis result of "Utterance content". "Effect of response content" is information that indicates whether the effect of the response content is positive or negative, as determined in step S1702.

[0164] (Content determination process) Fig. 19 is a flowchart showing an example of content determination processing according to the fifth embodiment. This processing shows an example of processing for determining appropriate content for a speaker from a database, for example, in steps S1405, S1410, etc. in Fig. 14.

[0165] In step S1901, the response unit 405 analyzes whether the approach content is similar to the utterance text by comparing it with a past database. For example, the response unit 405 refers to a database 1800 as shown in FIG. 18 to extract learning data (records) similar to the utterance text.

[0166] In step S1902, if the approach contents are similar, the response unit 405 determines appropriate content based on learning data of past successful cases. For example, the response unit 405 determines appropriate "content" based on the "analysis results (customer response)" or "analysis results (sales response)" of the extracted learning data in which the "effect of the response contents" is positive.

[0167] In addition, if the approach content is not similar, appropriate "content" may be determined based on the "analysis results (customer response)" or "analysis results (sales response)" of training data that is not similar to the spoken text and has a positive "effectiveness of the response content."

[0168] By the process of FIG. 19, the response unit 405 can determine appropriate content for the speaker from the database 1800.

[0169] [Sixth embodiment] The communication support system 1 can automatically create daily reports of business negotiations, minutes, etc., using the virtual sales 101. For example, if a salesperson 2 inquires about the features of a product, the virtual sales 101 can provide advice on sales strategies, and if a customer asks the same question, the virtual sales 101 can highlight the user experience of the product or actual usage scenarios.

[0170] Furthermore, the virtual sales 101 automatically records the details of the sales negotiations and compiles them into a daily report for the sales representative 2. This daily report includes, for example, the main points of the sales negotiations, the customer's reactions, the proposed products, and the feedback obtained.

[0171] This approach allows the communication support system 1 to better respond to the needs and expectations of salespeople and customers and provide the most relevant information from each perspective, thereby contributing to improving customer experience and optimizing sales efficiency.

[0172] <Functional configuration> Fig. 20 is a diagram showing the functional configuration of a communication support system 1 according to the fifth embodiment. The communication support device 10 according to the embodiment in Fig. 6 has a creation unit 2001 in addition to the functional configuration of the communication support device 10 according to the fifth embodiment described in Fig. 16.

[0173] The creation unit 2001 executes creation processing to record the content of communication such as business negotiations based on the speech acquired by the input unit 401 and create a daily report, minutes, or the like.

[0174] Note that the functional configurations other than the creation unit 2001 may be the same as the functional configurations of the communication support system 1 according to the first embodiment described in FIG. 4 and the functional configurations of the communication support system 1 according to the fifth embodiment described in FIG. 16.

[0175] <Processing flow> 21 is a flowchart showing an example of the support process according to the sixth embodiment. This process shows another example of the communication support process executed by the communication support system 1.

[0176] In step S2101, the communication support system 1 selects a daily report scenario, and then repeatedly executes the processes from step S2002 onwards. The scenario defines, for example, the negotiation level (new customer, existing customer, etc.), product features, market trends, and other related information. In addition, the roles of speakers (sales representative 2, customer 3, other stakeholders, etc.) are identified, and their respective needs and expectations are defined.

[0177] In step S2102, the input unit 401 of the communication support device 10 acquires the speech of the participants in the business negotiation. For example, the input unit 401 acquires the speech of the salesperson 2 and the speech of the customer 3 transmitted from the terminal device 100.

[0178] In step S2103, the communication support device 10 performs speech recognition and speaker identification on the speech acquired by the input unit 401. For example, the speech recognition unit 402 converts the speech acquired by the input unit 401 into text to create a speech text. Furthermore, the identification unit 404 analyzes the speech acquired by the input unit 401 and identifies whether the speaker is the sales representative 2 or the customer 3.

[0179] In step S2104, the response unit 405 determines whether the speaker is customer 3. If the speaker is customer 3, the response unit 405 shifts the process to step S2105. On the other hand, if the speaker is not customer 3, the response unit 405 shifts the process to step S2110.

[0180] When the process proceeds from step S2104 to step S2105, the response unit 405 selects an appropriate response strategy based on the role of the speaker (customer 3) and the content of the utterance.

[0181] In step S2106, the response unit 405 determines appropriate content for the speaker (customer 3) from the database. For example, the determination unit 406 selects relevant flyers, information, etc. from the database based on the speaker's interests, needs, etc.

[0182] In step S2107, the response unit 405 generates a response content for the speaker (customer 3). This response content is customized to meet the needs of the speaker based on the selected response strategy. For example, the response content for customer 3 includes information that focuses on the product's usability or experience.

[0183] In step S2108, the response unit 405 controls the virtual sales 101 in accordance with the generated response content.

[0184] In step S2109, the creation unit 2001 records the content of the customer 3's utterances and the like in the daily report based on the scenario selected in step S2101.

[0185] When the process proceeds from step S2104 to step S2110, the response unit 405 determines whether the speaker is sales representative 2. If the speaker is sales representative 2, the response unit 405 proceeds to step S2111. On the other hand, if the speaker is not sales representative 2, the response unit 405 ends the process of FIG.

[0186] In step S2111, the response unit 405 selects an appropriate response strategy based on the role of the speaker (sales representative 2) and the content of the speech.

[0187] In step S2112, the response unit 405 determines appropriate content for the speaker (sales representative 2) from the database. For example, the determination unit 406 selects relevant flyers, information, etc. from the database based on the speaker's interests, needs, etc.

[0188] In step S2113, response unit 405 generates a response content for the speaker (salesperson 2). This response content is customized to meet the needs of the speaker based on the selected response strategy. For example, the response content for salesperson 2 includes information about the sales strategy for the product.

[0189] In step S2114, the response unit 405 controls the virtual sales 101 in accordance with the generated response content.

[0190] In step S2115, the creation unit 2001 records the content of the speech of the sales representative 2 and the reaction of the customer 3 in a daily report based on the scenario selected in step S2101.

[0191] By the process of FIG. 21, the communication support system 1 can automatically create a daily report of business negotiations by the virtual sales 101. <Examples of additional features> (Personalized suggestions) The communication support system 1 can also analyze the past purchase data and interests of the customer 3 to make more personalized product suggestions.

[0192] For example, the communication support system 1 collects the past purchase history, online behavior, preferences, feedback, etc. of the customer 3 and stores it in a database, etc. The communication support system 1 also analyzes the collected data to identify the preferences, areas of interest, purchasing patterns, etc. of the customer 3.

[0193] In addition, based on the analysis results, the communication support system 1 proposes the most suitable products to the customer 3 during business negotiations, etc., and also customizes the information explaining the products to suit the customer 3's preferences, areas of interest, purchasing patterns, etc.

[0194] Preferably, the communication support system 1 collects reactions, feedback, etc. from the customers 3, and continuously improves the suggestion algorithm based on the collected information.

[0195] (Language Identification) The communication support system 1 can also identify the language used from the content of speech of the customer 3, and perform language switching, translation, and the like.

[0196] For example, the communication support system 1 automatically identifies the language used by the customer 3 from the content of the customer 3's speech, etc. Furthermore, the communication support system 1 switches the response language of the virtual sales person 101 or the language of the display screen, etc., depending on the identified language. This allows the communication support system 1 to translate the response voice of the virtual sales person 101 in real time and provide the response voice in a language that is easy for the customer 3 to understand.

[0197] Preferably, the communication support system 1 continuously improves the language translation algorithm based on feedback from the customers 3 .

[0198] <Examples of usage scenarios> (Metaverse space) Usage scenario: The Metaverse is an immersive digital space that utilizes virtual reality (VR) or augmented reality (AR) technology. Within this space, Virtual Sales 101 appears as an avatar and can introduce and demonstrate products in real time.

[0199] Features: Virtual Sales 101 in the Metaverse can provide customers with a more realistic product experience and enable interactive sales conversations.

[0200] Specific example: Virtual Salesperson 101 identifies the conversation between Salesperson 2 and Customer 3 in the metaverse and responds differently depending on who is speaking. For example, when Salesperson 2 asks a question about the features of a product, Virtual Salesperson 101 responds from the perspective of sales strategy or market analysis. On the other hand, when Customer 3 asks the same question, Virtual Salesperson 101 responds by focusing on how to use the product or the user experience.

[0201] (Online conference system) Usage scenario: Virtual Sales 101 is used on online meeting platforms to showcase products and services through video calls and screen sharing.

[0202] Features: With online conference systems, Virtual Sales 101 can support efficient communication in remote work and international business negotiations.

[0203] Specific example: Virtual Sales 101 analyzes conversations in real time during online meetings and distinguishes between what Salesperson 2 and Customer 3 say. If Salesperson 2 mentions a competing product, Virtual Sales 101 will provide a detailed explanation of how it compares to that product. On the other hand, if Customer 3 asks the same question, Virtual Sales 101 will provide an answer that focuses on the direct benefits or value to Customer 3.

[0204] (web-based chatbot) Usage scenario: As a chatbot integrated into a company's website or e-commerce platform, Virtual Sales 101 responds to customer inquiries in real time and suggests related products and services.

[0205] Features: Chatbots are available 24 / 7 and can improve customer satisfaction with instant responses.

[0206] Specific example: A chatbot identifies Salesperson 2 and Customer 3 from their conversation on a website. If Salesperson 2 asks about product availability, the chatbot will provide information on inventory management or logistics. On the other hand, if Customer 3 requests the same information, the chatbot will prioritize providing information on available product options or delivery times.

[0207] (Social Media Platforms) Usage scenario: On social media, Virtual Sales 101 can showcase products and communicate directly with customers through interactive content or live streaming.

[0208] Why it's great: Social media can reach a wide audience and help raise brand awareness.

[0209] Example: On social media, Virtual Sales 101 responds to technical questions from Salesperson 2 with detailed product information or market analysis, and responds to similar questions from Customer 3 with examples of everyday use of the product or user reviews.

[0210] (Mobile application) Usage scenario: Within a smartphone or tablet app, Virtual Sales 101 provides customers with personalized promotions, product information, and more.

[0211] Features: Mobile apps increase customer convenience and provide information anytime, anywhere.

[0212] Specific example: Within the mobile app, Virtual Sales 101 responds to inquiries from Salesperson 2 by providing information on sales promotion strategies or market trends, and responds to inquiries from Customer 3 by providing information on product features or how to use them.

[0213] (Interactive Kiosk or Digital Signage) Usage scenarios: Kiosks or digital signage incorporating Virtual Sales 101 can be installed in brick-and-mortar stores or at exhibitions to provide product information to customers.

[0214] What makes it special: It allows you to create a digital, interactive customer experience in a physical space.

[0215] Specific example: At a kiosk in a physical store, the virtual salesperson 101 provides the salesperson 2 with store sales data or customer trends, etc., and provides general customers with interactive demonstrations of product features or usage methods, etc.

[0216] As described above, according to each embodiment of the present invention, it becomes easy to support communication involving multiple participants using a dialogue agent.

[0217] <Supplementary information> Each function of each embodiment described above can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and conventional circuit modules designed to perform each of the functions described above.

[0218] Furthermore, the devices described in the examples are merely one of multiple computing environments for implementing the embodiments disclosed herein. In one embodiment, the communication assistance device 10 includes multiple computing devices, such as a server cluster. The multiple computing devices are configured to communicate with each other via any type of communication link, including a network, shared memory, etc., and perform the processes disclosed herein.

[0219] The functional components of the communication support device 10 may be integrated into one server device or may be divided among multiple devices. Furthermore, at least some of the functional components of the communication support device 10 may be included in the terminal device 100.

[0220] <Additional Notes> This specification discloses the following communication support device, communication support system, communication support method, and program. (Section 1) A communication support device that supports communication between multiple participants, an input unit that accepts input of utterances in the communication; a speaker identification unit that identifies the participant who made the inputted statement from among the plurality of participants; an identification unit that identifies attributes of the participants identified by the speaker identification unit; a generation unit that generates a response content based on the attributes of the identified participant and the content of the comment; a control unit that controls the response content generated by the generation unit to be output; A communication support device having the above features. (Section 2) The participants include a host and guests who participate in the communication; The response content includes recommendation information to be proposed to the guest. 2. The communication support device according to claim 1. (Section 3) said communication is a business negotiation; The response content includes information about a product to be proposed to the guest and a description of the product. 3. The communication support device according to claim 2. (Section 4) 4. The communication support device according to claim 3, wherein the response content is different between an explanation of the product to the host and an explanation of the product to the guest. (Section 5) the response content includes non-verbal information; 5. The communication support device according to any one of claims 1 to 4, wherein the control unit causes a dialogue agent to output the non-verbal information. (Section 6) 6. The communication support device according to claim 5, wherein the non-verbal information includes one or more of facial expressions, gaze, posture, emotions, actions, and paralanguage. (Section 7) the response content includes linguistic information of the dialogue agent; 7. The communication support device according to any one of claims 1 to 6, wherein the control unit controls the utterance of the dialogue agent based on the language information. (Section 8) 5. The communication support device according to any one of claims 2 to 4, wherein the dialogue agent proposes recommendation information in accordance with the flow of communication. (Section 9) 4. The communication support device according to claim 3, wherein the conversation agent starts explaining the product in accordance with an instruction from the host. (Section 10) 10. The communication support device according to any one of paragraphs 1 to 9, wherein the generation unit generates different response contents depending on attributes of the participants. (Section 11) 5. The communication support device according to any one of claims 2 to 4, further comprising a feedback unit that determines whether or not the proposed recommendation information is effective and provides feedback. (Section 12) 12. The communication support device according to any one of items 1 to 11, further comprising a creating unit that records the content of the communication based on the utterances and creates a daily report or minutes of the meeting. (Section 13) 13. The communication support device according to claim 12, wherein the creation unit creates the daily report or minutes based on a predefined scenario. (Section 14) 14. The communication support device according to any one of paragraphs 1 to 13, wherein the generation unit generates the response content personalized for the participant based on a past history of the participant. (Section 15) A communication support system that supports communication among multiple participants, comprising a terminal device and a server device, The server device an input unit that accepts input of utterances in the communication; a speaker identification unit that identifies the participant who made the inputted statement from among the plurality of participants; an identification unit that identifies attributes of the participants identified by the speaker identification unit; a generation unit that generates a response content based on the attributes of the identified participant and the content of the comment; a control unit that controls the response content generated by the generation unit to be output; and the terminal device has a data transmission unit that transmits the remarks of the participants to the server device; Communication support system. (Section 16) A computer that supports communication between multiple participants A process of receiving an input of a statement in the communication; A process of identifying the participant who made the inputted statement from among the plurality of participants; A process of identifying attributes of the participants identified by the speaker identification unit; a process of generating a response content based on the attributes of the identified participant and the content of the comment; a process of controlling the output of the response content generated by the generation unit; A communication support method that implements the following. (Section 17) 17. A program for causing a computer to execute the communication support method according to claim 16.

[0221] Although the embodiments of the present invention have been described above, the present invention is not limited to such specific embodiments, and various modifications and applications are possible within the scope of the gist of the present invention described in the claims. [Explanation of symbols]

[0222] 1. Communication Support System 2 Sales Representative (Host) 3 Customers (Guests) 10 Communication support devices 100 Terminal Device 200 computers 401 Input section 403 Speaker Identification Unit 404 Identification Unit 408 Generation part 409 Control Unit 421 Data Transmission Unit 422 Display section 423 Audio Output Unit 1601 Feedback Section 2001 Creation Department [Prior art documents] [Patent documents]

[0223] [Patent Document 1] Japanese Patent Publication No. 2022-108803

Claims

1. A communication support device that supports communication between multiple participants, an input unit that accepts input of utterances in the communication; a speaker identification unit that identifies the participant who made the inputted statement from among the plurality of participants; an identification unit that identifies attributes of the participants identified by the speaker identification unit; a generation unit that generates a response content based on the attributes of the identified participant and the content of the comment; a control unit that controls the response content generated by the generation unit to be output; A communication support device having the above features.

2. The participants include a host and guests who participate in the communication; The response content includes recommendation information to be proposed to the guest. The communication support device according to claim 1 .

3. said communication is a business negotiation; The response content includes information about a product to be proposed to the guest and a description of the product. The communication support device according to claim 2 .

4. The communication support device according to claim 3 , wherein the response content is different between an explanation of the product given to the host and an explanation of the product given to the guest.

5. the response content includes non-verbal information; The communication support device according to claim 1 , wherein the control unit causes a dialogue agent to output the non-verbal information.

6. The communication support device according to claim 5 , wherein the non-verbal information includes one or more of facial expressions, gaze, posture, emotions, actions, and paralanguage.

7. the response content includes linguistic information of the dialogue agent; The communication support device according to claim 5 , wherein the control unit controls the utterance of the dialogue agent based on the linguistic information.

8. The communication support device according to claim 5 , wherein the dialogue agent proposes recommendation information in accordance with the flow of the communication.

9. The communication support device according to claim 5 , wherein the dialogue agent starts explaining the product in accordance with an instruction from the host.

10. The communication support device according to claim 1 , wherein the generation unit generates different response contents depending on attributes of the participants.

11. 5. The communication support device according to claim 2, further comprising a feedback unit that determines whether the proposed recommendation information is effective and provides feedback.

12. The communication support device according to claim 1 , further comprising a creating unit that records the content of the communication based on the utterances and creates a daily report or minutes of a meeting.

13. The communication support device according to claim 12 , wherein the creation unit creates the daily report or the minutes based on a predefined scenario.

14. The communication support device according to claim 1 , wherein the generation unit generates the response content personalized for the participant based on a past history of the participant.

15. A communication support system that supports communication among multiple participants, comprising a terminal device and a server device, The server device an input unit that accepts input of utterances in the communication; a speaker identification unit that identifies the participant who made the inputted statement from among the plurality of participants; an identification unit that identifies attributes of the participants identified by the speaker identification unit; a generation unit that generates a response content based on the attributes of the identified participant and the content of the comment; a control unit that controls the response content generated by the generation unit to be output; and the terminal device has a data transmission unit that transmits the remarks of the participants to the server device; Communication support system.

16. A computer that supports communication between multiple participants A process of receiving an input of a statement in the communication; A process of identifying the participant who made the inputted statement from among the plurality of participants; A process of identifying attributes of the participants identified in the process of identifying the participants; a process of generating a response content based on the attributes of the identified participant and the content of the comment; a control process for outputting the response content generated in the generating process; A communication support method that implements the following.

17. A program that causes a computer to execute the communication support method according to claim 16.

Images