Communication support system, communication support apparatus, communication support method, and program

Abstract

To facilitate participants participating in communication to obtain information supporting the communication during the communication.SOLUTION: A communication support system comprises: an acquisition unit for acquiring speech sounds of participants participating in communication; a determination unit for determining whether or not the communication needs to be supported based on the speech sounds of the participants in response to the speech of the participants; and an information providing unit for providing information to support the communication when it is determined that the communication needs to be supported.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to a communication support system, a communication support device, a communication support method, and a program.

Background Art

[0002] There is a communication system that transmits and receives content data such as video or audio between terminal devices.

[0003] In addition, a FAQ (Frequently Asked Questions) system is known in which a chatbot by an operator or an automatic conversation program answers questions by text input or voice input (see, for example, Patent Document 1).

Summary of the Invention

Problems to be Solved by the Invention

[0004] For example, in a conference system for conducting negotiations online, a salesperson may want to obtain information to support the negotiations from, for example, a FAQ or an assumed Q&A set during the negotiations so that they can smoothly answer questions from clients.

[0005] However, in a conventional technique such as that shown in Patent Document 1, for example, a user cannot obtain a necessary answer unless they explicitly ask a question to a FAQ system or the like, so it has been difficult to obtain a necessary answer during negotiations.

[0006] Note that such a problem is not limited to a conference system for conducting negotiations. For example, it commonly exists in various communication systems that perform communication such as telemedicine, counseling, remote lectures, or school entrance consultations.

[0007] One embodiment of the present invention has been made in view of the above problems, and facilitates a participant participating in communication to obtain information for supporting the communication during the communication.

Means for Solving the Problems

[0008] To solve the above problems, a communication support system according to an embodiment of the present invention includes an acquisition unit that acquires the speech voice of a participant participating in communication, and a determination unit that determines whether it is necessary to support the communication from the speech voice of the participant in response to the speech of the participant. And an information providing unit that provides information for supporting the communication when it is determined that the communication needs to be supported.

Effects of the Invention

[0009] According to one embodiment of the present invention, a participant participating in communication can easily obtain information for supporting the communication during the communication.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Mode for Carrying Out the Invention

[0011] Hereinafter, this embodiment will be described in detail with reference to the drawings.

[0012] <System Configuration> Figs. 1 and 2 are diagrams showing an example of the system configuration of a communication support system according to an embodiment. The communication support system 1 includes, for example, as shown in Fig. 1, a communication support device 10 connected to a communication network N such as the Internet and a LAN (Local Area Network), a conference server 20, a plurality of terminal devices 100a, 100b, 100c, 100d, ···, and an information terminal 101, etc. In the following description, when indicating an arbitrary terminal device among the plurality of terminal devices 100a, 100b, 100c, 100d, ···, "terminal device 100" is used. Also, the number of terminal devices 100 shown in Fig. 1 is an example.

[0013] The communication support system 1 is a system that supports various communications such as business negotiations, meetings, medical consultations, lectures, or counseling, etc., by transmitting and receiving at least voice between a terminal device 100 and another terminal device 100. Here, as an example, the following description will be made on the assumption that the communication support system 1 supports a web conference in which a participant (first participant) using the terminal device 100 transmits and receives video including voice to and from another participant (second participant) using another terminal device 100 and conducts a business negotiation.

[0014] The terminal device 100 is a general-purpose information terminal such as a PC (Personal Computer), a tablet terminal, or a smartphone, etc., used by a participant who participates in a web conference. Alternatively, the terminal device 100 may be an electronic device having a web conference function such as a video conference device or an IWB (Interactive White Board). Note that the IWB is a whiteboard having an electronic blackboard function capable of mutual communication and is also called an electronic blackboard. Here, as an example, the following description will be made on the assumption that the terminal device 100 is a general-purpose information terminal.

[0015] Participants who participate in a web conference access the conference address provided by the conference server 20 using, for example, an application for web conferencing installed on the terminal device 100, or a web browser or the like, to participate in the web conference.

[0016] The communication support device 10 is, for example, an information processing device having a computer configuration, or a system including a plurality of computers. The communication support device 10 provides the communication support service according to the present embodiment.

[0017] The conference server 20 is an information processing device having a computer configuration, or a system including a plurality of computers. The conference server 20 provides a web conferencing service for transmitting and receiving audio or video including audio between a plurality of terminal devices 100. In the present embodiment, the web conferencing service provided by the conference server 20 may be any web conferencing service.

[0018] The information terminal 101 is an information terminal such as a PC, a tablet terminal, or a smartphone used by an administrator who manages the communication support system 1 or a user who uses the communication support system 1. For example, the administrator can set the communication support device 10 using a web browser or the like provided in the information terminal 101.

[0019] Note that the system configuration of the communication support system 1 shown in FIG. 1 is an example. For example, the communication support system 1 may not have the conference server 20 as shown in FIG. 2. In this case, the terminal device 100 may use an external web conferencing service of the communication support system 1 to conduct a web conference. Alternatively, the communication support device 10 may have the functions of the conference server 20, and the terminal device 100 may use the web conferencing service provided by the communication support device 10 to conduct a web conference. Further, the communication support system 1 may support communication in a general meeting where participants participate in a meeting in the same meeting room using the terminal device 100a.

[0020] (Overview of the process) For example, when conducting business negotiations using a web conference provided by the conference server 20, a salesperson (an example of a first participant) participating in the business negotiations may receive various questions from a client (an example of a second participant) participating in the business negotiations. In such a case, the salesperson has a request to obtain information for supporting the business negotiations from various information such as FAQs, anticipated Q&A sets, product information, or industry information during the business negotiations so as to be able to smoothly answer questions from the client.

[0021] However, in a conventional technique as shown in Patent Document 1, for example, since a user cannot obtain a necessary answer unless explicitly asking a question to an FAQ system or the like, it has been difficult to obtain information for supporting the business negotiations during the business negotiations.

[0022] Therefore, the present embodiment provides a communication support system 1 that facilitates a participant participating in communication such as business negotiations to obtain information for supporting the communication during the communication.

[0023] For example, the communication support system 1 acquires the speech voice of participants participating in the communication, and detects one or more predetermined keywords from the speech voice of the participants according to the speech of the participants. Further, when a predetermined keyword is detected, the communication support system 1 provides information for supporting the communication corresponding to the detected keyword.

[0024] Preferably, the communication support system 1 provides, as information for supporting the communication, information corresponding to the detected keyword from a support database in which, for example, FAQs, assumed Q&A sets, product information, or industry information is registered.

[0025] Preferably, the communication support system 1 acquires the speech voice of the participants from the terminal device 100 used by the first participant (for example, a salesperson) participating in the communication. Further, the communication support system 1 acquires, from the support database, information for supporting the communication corresponding to a predetermined keyword detected from the speech voice of the second participant (for example, a client) among the acquired speech voice of the participants. Furthermore, the communication support system 1 causes the terminal device 100 used by the first participant to display the acquired information for supporting the communication.

[0026] Accordingly, according to the communication support system 1 according to the present embodiment, a participant participating in the communication can easily obtain information for supporting the communication during the communication.

[0027] As described above, according to the present embodiment, it becomes easy for a participant participating in the communication to obtain information for supporting the communication during the communication.

[0028] <Hardware Configuration> (Hardware Configuration of Computer) The communication support device 10, the conference server 20, and the information terminal 101 have, for example, the hardware configuration of a computer 300 as shown in FIG. 3. Alternatively, the communication support device 10 or the conference server 20 may be configured by a plurality of computers 300. Further, as an example, the terminal device 100 may have the hardware configuration of a computer 300 as shown in FIG. 3.

[0029] FIG. 3 is a diagram showing an example of the hardware configuration of a computer according to an embodiment. As shown in FIG. 3, for example, the computer 300 includes a CPU (Central Processing Unit) 301, a ROM (Read Only Memory) 302, a RAM (Random Access Memory) 303, an HD (Hard Disk) 304, an HDD (Hard Disk Drive) controller 305, a display 306, an external device connection I / F (Interface) 307, a network I / F 308, a keyboard 309, a pointing device 310, a DVD-RW (Digital Versatile Disk Rewritable) drive 312, a media I / F 314, and a bus line 315, etc.

[0030] When the computer 300 is the terminal device 100, the computer 300 further includes, for example, a microphone 321, a speaker 322, an audio input / output I / F 323, a CMOS (Complementary Metal Oxide Semiconductor) sensor 324, and an imaging device I / F 325, etc.

[0031] Among these, the CPU 301 controls the operation of the entire computer 300. The ROM 302 stores programs used for starting up the computer 300, such as an IPL (Initial Program Loader). The RAM 303 is used, for example, as a work area for the CPU 301. The HD 304 stores programs such as an OS (Operating System), applications, device drivers, etc., and various data. The HDD controller 305 controls the reading or writing of various data to and from the HD 304 according to the control of the CPU 301. Note that the HD 304 and the HDD controller 305 are an example of a storage device included in the computer 300.

[0032] The display 306 displays various information such as a cursor, menu, window, characters, or images. Note that the display 306 may be provided outside the computer 300. The external device connection I / F 307 is an interface for connecting various external devices to the computer 300. The network I / F 308 is an interface for connecting the computer 300 to the communication network N and communicating with other devices.

[0033] The keyboard 309 is a type of input means having a plurality of keys for inputting characters, numerical values, various instructions, etc. The pointing device 310 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. Note that the keyboard 309 and the pointing device 310 may be provided outside the computer 300.

[0034] The DVD-RW drive 312 controls the reading or writing of various data to / from the DVD-RW 311, which is an example of a removable recording medium. Note that the DVD-RW 311 may be other recording media, not limited to DVD-RW. The Media I / F 314 controls the reading or writing (storage) of data to / from the medium 313 such as a flash memory. The bus line 315 includes an address bus, a data bus, and various control signals for electrically connecting the above components.

[0035] The microphone 321 is a built-in circuit that converts sound into an electrical signal. The speaker 322 is a built-in circuit that converts an electrical signal into physical vibrations to produce sounds such as music and voices. The audio I / O I / F 323 is a circuit that processes the input and output of audio signals between the microphone 321 and the speaker 322 according to the control of the CPU 301.

[0036] The CMOS sensor 324 is a type of built-in imaging means that captures a subject (e.g., a self-portrait) according to the control of the CPU 301 to obtain image data. Note that the computer 300 may have an imaging means such as a CCD (Charge Coupled Device) sensor instead of the CMOS sensor 324. The imaging device I / F 325 is a circuit that controls the driving of the CMOS sensor 324.

[0037] (An example of the hardware configuration of the terminal device) FIG. 4 is a diagram showing an example of the hardware configuration of a terminal device according to an embodiment. Here, an example of the hardware configuration of the terminal device 100 when the terminal device 100 is an information terminal such as a smartphone or a tablet terminal will be described. Note that the terminal device 100 may be, for example, an electronic device having a configuration of a computer such as a video conferencing device or an IWB and equipped with a web conferencing function. Also, the information terminal 101 may have the hardware configuration of the computer 300 as shown in FIG. 3.

[0038] In the example of FIG. 4, the terminal device 100 includes a CPU 401, a ROM 402, a RAM 403, a storage device 404, a CMOS sensor 405, an imaging device I / F 406, an acceleration / azimuth sensor 407, a media I / F 409, a GPS (Global Positioning System) receiver 410, and the like.

[0039] Among these, the CPU 401 controls the operation of the entire terminal device 100 by executing a predetermined program. The ROM 402 stores programs used for starting up the CPU 401, such as IPL. The RAM 403 is used as a work area for the CPU 401. The storage device 404 is a large-capacity storage device that stores programs such as an OS and applications, and various data, and is realized by, for example, an SSD (Solid State Drive) or a flash ROM.

[0040] The CMOS sensor 405 is a type of built-in imaging means that captures a subject (mainly a self-image) according to the control of the CPU 401 to obtain image data. Note that the terminal device 100 may have imaging means such as a CCD sensor instead of the CMOS sensor 405. The imaging device I / F 406 is a circuit that controls the driving of the CMOS sensor 405. The acceleration / azimuth sensor 407 is various sensors such as an electronic magnetic compass that detects geomagnetism, a gyrocompass, and an acceleration sensor. The media I / F 409 controls the reading or writing (storage) of data to / from a media (storage media) 408 such as a flash memory. The GPS receiver 410 receives GPS signals (positioning signals) from GPS satellites.

[0041] Further, the terminal device 100 includes a long-distance communication circuit 411, an antenna 411a of the long-distance communication circuit 411, a CMOS sensor 412, an imaging device I / F 413, a microphone 414, a speaker 415, an audio input / output I / F 416, a display 417, an external device connection I / F 418, a short-distance communication circuit 419, an antenna 419a of the short-distance communication circuit 419, and a touch panel 420.

[0042] Among these, the long-distance communication circuit 411 is a circuit that communicates with other devices via, for example, the communication network N. The CMOS sensor 412 is a type of built-in imaging means that captures a subject according to the control of the CPU 401 to obtain image data. The imaging device I / F 413 is a circuit that controls the driving of the CMOS sensor 412. The microphone 414 is a built-in circuit that converts sound into an electrical signal. The speaker 415 is a built-in circuit that converts an electrical signal into physical vibration to produce sounds such as music and voice. The audio input / output I / F 416 is a circuit that processes the input / output of audio signals between the microphone 414 and the speaker 415 according to the control of the CPU 401.

[0043] The display 417 is a type of display means such as a liquid crystal or an organic EL (Electro Luminescence) that displays an image of a subject, various icons, etc. The external device connection I / F 418 is an interface for connecting various external devices. The short-distance communication circuit 419 includes a circuit that performs short-distance wireless communication. The touch panel 420 is a type of input means for operating the terminal device 100 when a user presses the display 417.

[0044] Also, the terminal device 100 includes a bus line 421. The bus line 421 includes an address bus, a data bus, and various control signals for electrically connecting each component such as the CPU 401 shown in FIG. 4.

[0045] Note that the hardware configuration of the terminal device 100 shown in FIG. 4 is an example. The terminal device 100 may have other hardware configurations as long as it has a computer configuration, a communication circuit, a display, a microphone, a speaker, etc.

[0046] <Functional Configuration> Subsequently, the functional configuration of the communication support system 1 will be described. FIG. 5 is a diagram showing an example of the functional configuration of the communication support system according to an embodiment.

[0047] (Functional Configuration of Terminal Device) Here, the following description will be given on the assumption that the terminal device 100 has the hardware configuration of the computer 300 as shown in FIG. 3.

[0048] The terminal device 100a used by the first participant participating in the communication realizes each functional configuration as shown in, for example, FIG. 5 by executing a predetermined program with the CPU 301. In the example of FIG. 5, the terminal device 100a has functional configurations such as a communication unit 511, a conference control unit 512, a voice transmission unit 513, a display unit 514, and an operation reception unit 515. Note that at least a part of the above functional configurations may be realized by hardware.

[0049] In this embodiment, the terminal device 100b used by the second participant participating in the communication may have an arbitrary configuration as long as it can participate in the web conference provided by the conference server 20 and can transmit and receive voice with the first participant.

[0050] The communication unit 511 executes communication processing for connecting the terminal device 100a to the communication network N using, for example, the network I / F 308 or the like, and communicating with other devices such as the communication support device 10, the conference server 20, and the terminal device 100b.

[0051] The conference control unit 512 executes a series of processes related to the web conference, such as connection to the web conference provided by the conference server 20, transmission and reception of conference video (or conference audio), and input and output of conference video (or conference audio). Note that the processes executed by the conference control unit 512 may be the same as those of a general web conference. Note that the web conference is an example of the communication according to this embodiment.

[0052] The voice transmission unit 513 acquires the speech voices of the participants participating in the meeting, and executes a voice transmission process of transmitting the acquired speech voices (voice data) to the communication support device 10. The speech voices of the participants include, for example, the speech voice of the first participant (for example, a salesperson) using the terminal device 100a and the speech voice of the second participant (for example, a client) using another terminal device 100b. However, in this embodiment, it is sufficient that the speech voices of the participants include at least the speech voice of the second participant.

[0053] For example, the voice transmission unit 513 acquires the voice acquired by the audio input / output I / F 323 from the microphone 321 as the speech voice of the first participant, and the voice output from the speaker 322 by the audio input / output I / F 323 as the speech voice of the second participant. However, it is not limited to this, and the voice transmission unit 513 may acquire the speech voice of the first participant or the speech voice of the second participant from the conference control unit 512 or the like.

[0054] The display unit 514 executes a display process of displaying a display screen on, for example, the display 306 or the like. The operation reception unit 515 executes an operation reception process of receiving an operation by the first participant using an input device such as, for example, the keyboard 309 or the pointing device 310. For example, the operation reception unit 515 receives a predetermined operation on the operation screen displayed by the display unit 514, and transmits operation information, which is information on the received predetermined operation, to the communication support device 10 via the communication unit 511.

[0055] (Functional Configuration of Communication Support Device) The communication support device 10 realizes each functional configuration as shown in, for example, FIG. 5 by one or more computers 300 executing a predetermined program. In the example of FIG. 5, the communication support device 10 has each functional configuration such as a communication unit 501, an acquisition unit 502, a conversion unit 503, a determination unit 504, an information providing unit 505, a DB (Database) management unit 506, and a setting reception unit 507. Note that at least a part of the above functional configurations may be realized by hardware.

[0056] Further, the communication support apparatus 10 realizes the storage unit 508 by storage devices such as an HD 304 and an HDD controller 305, for example.

[0057] The communication unit 501 executes communication processing for connecting the communication support apparatus 10 to the communication network N using, for example, a network I / F 308 or the like, and communicating with other devices such as the terminal device 100b and the information terminal 101.

[0058] The acquisition unit 502 executes acquisition processing for acquiring the speech voice (voice data) of the participants participating in the communication. For example, the acquisition unit 502 acquires the speech voice of the participants including the speech voice of the first participant and the speech voice of the second participant, which are transmitted by the voice transmission unit 513 of the terminal device 100a used by the first participant, to the communication support apparatus 10. Note that the speech voice of the participants transmitted to the communication support apparatus 10 includes at least the speech voice of the second participant.

[0059] The conversion unit 503 executes conversion processing for converting the speech voice of the participants acquired by the acquisition unit 502 into text. For example, the conversion unit 503 converts the speech voice of the first participant and the speech voice of the second participant into text, and stores the text-converted speech voice in the storage unit 508 or the like. Note that the conversion unit 503 converts at least the speech voice of the second participant into text. Note that the processing for converting speech voice into text is also called speech recognition or text conversion.

[0060] The determination unit 504 executes a determination process to determine whether it is necessary to assist communication from the speech of the participant in response to the speech of the participant. For example, the determination unit 504 detects one or more predetermined keywords from the speech of the participant in response to the speech of the participant. Preferably, the determination unit 504 detects the extraction keywords registered in the support DB managed by the DB management unit 506 from the texturized speech of the second participant for each speech of the second participant. In addition, when one or more predetermined keywords are detected from the speech of the participant, the determination unit 504 determines that it is necessary to assist communication.

[0061] FIG. 6 is a diagram for explaining a support database according to an embodiment. The DB management unit 506 of the communication support device 10 manages, for example, a support DB 600 including items as shown in FIG. 6.

[0062] In the example of FIG. 6, the support DB 600 includes information such as "assumed question", "answer", and "extraction keyword" as items. The "assumed question" is a question assumed in communication such as business negotiation. The "answer" is an answer to the "assumed question". The "assumed question" and the "answer" can be registered in the support DB 600 by, for example, an administrator who manages the communication support system 1 or a user who uses the communication support system 1 using an information terminal 101 or the like.

[0063] In addition, when the "assumed question" and the "answer" are registered, the communication support device 10 extracts keywords from the "assumed question" and registers the one or more extracted keywords in the "extraction keyword". Note that, for example, as in the data example 601 shown in FIG. 6, in the "assumed question" and the "answer", the text (character string or sentence) registered as the "assumed question" and the "answer" corresponding to the "assumed question" are registered as, for example, text data.

[0064] When it is determined by the determination unit 504 that it is necessary to assist communication, the information providing unit 505 executes an information providing process for providing information to assist communication. For example, when a predetermined keyword is detected by the determination unit 504, the information providing unit 505 provides information to assist communication corresponding to the detected predetermined keyword.

[0065] For example, assume that the determination unit 504 detects predetermined keywords "Service A" and "linkage" registered in the "extracted keywords" of the support DB 600 from the speech of the second participant. In this case, the information providing unit 505 provides, as information to assist communication, a hypothetical question "Can it be linked with Service A?" corresponding to the detected predetermined keyword and an answer corresponding to the hypothetical question from the support DB 600. For example, the information providing unit 505 causes a display screen of an application executed by the terminal device 100 used by the first participant to display the hypothetical question and answer corresponding to the detected predetermined keyword.

[0066] The DB management unit 506 manages, for example, a support DB 600 including information as shown in FIG. 6. Note that the support DB 600 may be stored in the storage unit 508 or the like of the communication support apparatus 10, or may be stored in a cloud service or the like that is available via the communication network N.

[0067] The setting reception unit 507 executes a setting reception process for receiving various settings related to the communication support system 1 from, for example, the information terminal 101 or the terminal device 100. For example, the setting reception unit 507 receives registration of "hypothetical questions" and "answers" in the support DB 600, or settings of the support DB 600.

[0068] The storage unit 508 stores various information such as the speech received by the acquisition unit 502, the speech texturized by the conversion unit 503, the support DB 600, or the setting information received by the setting reception unit 507, as well as data.

[0069] (Functional Configuration of Information Terminal) The information terminal 101 realizes, for example, a communication unit 531, a display unit 532, an operation reception unit 533, etc. when a computer 300 executes a predetermined program. Note that at least a part of each of the above functional configurations may be realized by hardware.

[0070] The communication unit 531 executes communication processing for connecting the information terminal 101 to a communication network N and communicating with other devices, using, for example, a network I / F 308 or the like.

[0071] The display unit 532 executes display processing for displaying, for example, a display screen provided by the communication support device 10 on a display 306 or the like. The operation reception unit 533 executes operation reception processing for receiving operations such as input, setting, or selection on the display screen displayed by the display unit 532. Note that the display unit 532 and the operation reception unit 533 may be realized by, for example, a web browser or the like provided in the information terminal 101. For example, the information terminal 101 may be a general-purpose information terminal equipped with a web browser.

[0072] Note that the functional configuration of the communication support system 1 shown in FIG. 5 is an example. For example, the functions of the conference server 20 may be possessed by the communication support device 10, or may be possessed by an external computer 300 of the communication support system 1, or a cloud service or the like. Also, at least a part of the functional configuration of the communication support device 10 may be possessed by the terminal device 100, or the information terminal 101 or the like. Furthermore, each functional configuration provided in the communication support device 10 may be distributed and provided in a plurality of information processing devices. In short, each functional configuration of the communication support system 1 shown in FIG. 5 only needs to be possessed by any device included in the communication support system 1.

[0073] <Processing Flow>[ Next, the processing flow of the communication support method according to the present embodiment will be described.

[0074] [First Embodiment] (Data Registration Process) FIG. 7 is a flowchart showing an example of the data registration process according to the first embodiment. This process shows, for example, an example of a data registration process in which an administrator who manages the communication support system 1 uses the information terminal 101 to register an assumed question and its answer in the support DB 600. Note that this data registration process may be executed, for example, by a user who uses the communication support system 1 using the terminal device 100 or the like. Hereinafter, the following description will be made on the assumption that the administrator executes the data registration process using the information terminal 101.

[0075] In step S701, the setting reception unit 507 causes the information terminal 101 to display a registration screen for registering an assumed question and an answer, for example. For example, the administrator accesses a predetermined web page provided by the setting reception unit 507 of the communication support device 10 using the web browser provided in the information terminal 101, thereby displaying the registration screen for the assumed question and the answer on the information terminal 101.

[0076] In step S702, the setting reception unit 507 receives the input of the assumed question and the answer for the displayed registration screen.

[0077] In step S703, the setting reception unit 507 extracts keywords from the received assumed question.

[0078] In step S704, the setting reception unit 507 registers the input assumed question, answer, and one or more extracted keywords (extracted keywords) in the support DB 600.

[0079] By the process of FIG. 7, the administrator can register the "assumed question", "answer", and "extracted keywords" in the support DB 600 as shown in FIG. 6.

[0080] Note that the item names "assumed question" and "answer" are just examples. For example, the "assumed question" may be an "assumed utterance" assumed in communication, etc., and the "answer" may be a "response", etc., which is a response to the "assumed utterance".

[0081] (Communication support process) FIG. 8 is a flowchart showing an example of the communication support process according to the first embodiment. This process shows an example of a communication support process in which the communication support system 1 supports communication such as a web conference in which a first participant using the terminal device 100a and a second participant using another terminal device 100b participate. Note that at the start point of the process shown in FIG. 8, it is assumed that the first participant has joined the web conference using the terminal device 100a, and the second participant has joined the same web conference using another terminal device 100b. Also, it is assumed that the terminal device 100a is executing an application program (hereinafter referred to as an app) corresponding to the communication support system 1.

[0082] In step S801, the acquisition unit 502 acquires the speech voice of the participant transmitted by the terminal device 100a used by the first participant. For example, the voice transmission unit 513 of the terminal device 100a transmits the speech voice of the participant including the speech voice of the first participant acquired by the microphone and the speech voice of the second participant output from the speaker to the communication support device 10. The acquisition unit 502 acquires the speech voice of the participant transmitted by the terminal device 100a via the communication unit 501.

[0083] In step S802, the conversion unit 503 converts the speech voice of the participant acquired by the acquisition unit 502 into text (text conversion). For example, the conversion unit 503 converts at least the speech voice of the second participant among the speech voices of the participants into text.

[0084] In step S803, the determination unit 504 detects the extraction keywords of the support DB600 from the text-converted uttered voice. Preferably, for each utterance of the second participant, the determination unit 504 detects one or more keywords registered in the extraction keywords of the support DB600 from the text-converted uttered voice of the second participant.

[0085] In step S804, the information providing unit 505 determines whether a keyword has been detected by the determination unit 504. If a keyword is detected, the information providing unit 505 causes the process to proceed to step S805. On the other hand, if no keyword is detected, the information providing unit 505 causes the process to proceed to step S806.

[0086] When the process proceeds to step S805, the information providing unit 505 causes the support information corresponding to the detected keyword to be displayed on the terminal device 100a used by the first participant.

[0087] FIG. 9 is a diagram showing an example of a display screen of support information according to the first embodiment. The display screen 900 in FIG. 9 shows an example of an application screen displayed by the terminal device 100a in step S805. The display screen 900 includes support information 901 provided by the information providing unit 505. Note that the support information 901 is an example of information for supporting communication corresponding to a detected predetermined keyword.

[0088] For example, when keywords such as "recording", "data", and "saving" are detected from the uttered voice of the second participant, the information providing unit 505 causes support information 901a as shown in FIG. 9 to be displayed on the terminal device 100a based on the support DB600 as shown in FIG. 6. Also, when keywords such as "recording", "data", and "storage" are detected from the uttered voice of the second participant, the information providing unit 505 causes support information 901b as shown in FIG. 9 to be displayed on the terminal device 100a based on the support DB600.

[0089] On the other hand, when shifting to step S806, the information providing unit 505 ends the process of FIG. 8 without performing the display process of the support information in step S805.

[0090] By repeatedly executing the process of FIG. 8, the communication support system 1 can support communication in real time in response to the speech of the second participant. Here, "real time" means providing support information in accordance with the progress of communication. For example, a processing delay from the speech of the second participant until the support information is displayed is allowed.

[0091] [Second Embodiment] In the second embodiment, an example of the process when the DB management unit 506 manages a plurality of support databases will be described.

[0092] FIG. 10 is a diagram for explaining the support database according to the second embodiment. In the example of FIG. 10, the DB management unit 506 manages a plurality of support DBs 1000. The plurality of support DBs 1000 include, for example, an FAQ DB 1000a, an assumed Q&A DB 1000b, a product information DB 1000c, an industry information DB 1000d, a conversation information DB 1000e, and the like.

[0093] The FAQ DB 1000a is a database that summarizes questions and their answers regarding products, services, or support frequently asked by customers, etc., so that they can be searched. The assumed Q&A DB 1000b is a database that summarizes assumed questions and their answers according to the content of communication such as business negotiations, telemedicine, counseling, remote lectures, or school entrance consultations, so that they can be searched.

[0094] The product information DB 1000c is a database that summarizes assumed questions and their answers regarding products that can be the subject of business negotiations, so that they can be searched. Note that the product information DB 1000c may be a service information DB or the like that summarizes assumed questions and their answers regarding services that can be the subject of business negotiations so that they can be searched.

[0095] The industry information DB1000d is a database that, for example, summarizes conversations related to a specific industry and topics related to those conversations so that they can be searched. For example, when a salesperson or the like conducts business negotiations with a client in another industry, by using the industry information DB100d corresponding to the other industry, they can smoothly engage in conversations about topics in the other industry. The conversation information DB1000e is a database that, for example, pre-registers conversation topics (such as topics prepared in advance) corresponding to conversations with a second participant. Note that the configuration of each support DB1000 may be the same as that of the support DB600 described in FIG. 6, for example.

[0096] The setting reception unit 507 receives settings, for example, of the support DB1000 in which an administrator registers assumed questions and answers, or the support DB1000 to be searched in the communication support process described in FIG. 8.

[0097] <Flow of processing> (Data registration process) FIG. 11 is a flowchart showing an example of the data registration process according to the second embodiment. This process shows another example of a data registration process in which, for example, an administrator who manages the communication support system 1 uses the information terminal 101 to register assumed questions and their answers in the support DB600. Since the basic processing content is the same as that of the data registration process according to the first embodiment described in FIG. 7, a detailed description of the same processing content as in the first embodiment is omitted here.

[0098] In step S1101, the setting reception unit 507 causes the information terminal 101 to display, for example, a data registration screen for registering assumed questions and answers. As an example, the setting reception unit 507 causes the information terminal 101 to display a data registration screen 1200 as shown in FIG. 12.

[0099] FIG. 12 is a diagram showing an example of a data registration screen according to the second embodiment. In the example of FIG. 12, on the data registration screen 1200, a selection column 1201 for selecting a database for registering data, an input column 1202 for inputting an assumed question, an input column 1203 for inputting an answer, a registration button 1204, etc. are displayed.

[0100] In step S1102, the setting reception unit 507 receives the selection of the support DB 1000 to be registered. For example, the setting reception unit 507 receives the selection of the support DB 1000 by the selection column 1201 of the data registration screen 1200 as shown in FIG. 12. In the example of FIG. 12, the selection column 1201 is a pull-down menu so that the support DB 1000 to be registered can be selected. Note that the method of selecting the support DB 1000 to be registered may be a selection method other than the pull-down menu (for example, a check box, a radio button, etc.).

[0101] In step S1103, the setting reception unit 507 receives the input of the assumed question and the answer. For example, the setting reception unit 507 acquires the assumed question input in the input column 1202 of the data registration screen 1200 and the answer input in the input column 1203.

[0102] In step S1104, the setting reception unit 507 extracts keywords from the received assumed question.

[0103] In step S1105, the setting reception unit 507 registers the input assumed question, answer, and one or more extracted keywords (extracted keywords) in the support DB 1000 selected in step S1102.

[0104] By the process of FIG. 11, the administrator can register the "assumed question", "answer", and "extracted keywords" in any support DB 1000.

[0105] (Support Database Selection Process) The communication support process according to the second embodiment may be the same as the communication support process according to the first embodiment described with reference to FIG. 8. However, in the second embodiment, before executing the communication support process described with reference to FIG. 8, a selection process for a support database as shown in FIG. 13 is executed.

[0106] FIG. 13 is a flowchart showing an example of a selection process for a support database according to the second embodiment.

[0107] In step S1301, the display unit 514 of the terminal device 100a used by the first participant displays, for example, an application screen (settings screen) 1400 as shown in FIG. 14 on the display 306 or the like.

[0108] FIG. 14 is a diagram (1) showing an example of an application screen according to the second embodiment. In the example of FIG. 14, on the application screen 1400, for example, a “Record” button 1401 for starting recording of the speech of the participant and a pull-down menu 1402 for selecting a support mode are displayed. Note that the pull-down menu 1402 is an example of a selection field for selecting a support mode.

[0109] The communication support system 1 manages, for example, mode management information 1500 as shown in FIG. 15. The mode management information 1500 stores a plurality of support modes and search target databases corresponding to the respective support modes.

[0110] In the example of FIG. 15, the support modes in the mode management information 1500 include a negotiation mode, a support mode, a conversation mode, and the like. Further, the mode management information 1500 indicates that, for example, when the negotiation mode is selected, the search target databases are the assumed Q&A database 1000b, the product information database 1000c, and the industry information database 1000d.

[0111] Preferably, the mode management information 1500 can be registered, for example, by an administrator or the like who manages the communication support system 1 using the information terminal 101.

[0112] In step S1302, the operation reception unit 515 of the terminal device 100a receives a setting operation of the support DB 1000 by the first participant. For example, the operation reception unit 515 receives the setting of the support mode by the pull-down menu 1402 of the application screen 1400 as shown in FIG. 14.

[0113] In step S1303, the communication support system 1 determines the support DB 1000 to be searched from the received support mode and the mode management information 1500 as shown in FIG. 15. For example, when the negotiation mode is set, the communication support system 1 determines the support DB 1000 to be searched as the assumed Q&A DB 1000b, the product information DB 1000c, and the industry information DB 1000d. Note that this process may be executed by the setting reception unit 507 of the communication support device 10, or may be executed by the operation reception unit 515 of the terminal device 100a.

[0114] In step S1304, when the operation reception unit 515 of the terminal device 100a receives a recording start operation, in step S1305, the communication support system 1 executes the communication support process described with reference to FIG. 8. In this case, for example, in step S803 of FIG. 8, the determination unit 504 detects the extraction keyword of the support DB 1000 determined as the search target from the texturized speech.

[0115] By the process of FIG. 13, the first participant can receive a communication support service based on a more appropriate support DB 1000 according to the selected support mode.

[0116] Note that the application screen 1400 shown in FIG. 14 is an example. For example, in step S1301 of FIG. 13, the display unit 514 of the terminal device 100a may display an application screen (setting screen) 1600 as shown in FIG. 16 on the display 306 or the like.

[0117] FIG. 16 is a diagram (2) showing an example of an application screen according to the second embodiment. In the example of FIG. 16, on the application screen 1600, for example, a "Record" button 1401 for starting recording of the speech of a participant and a check box 1601 for selecting a support DB 1000 to be searched are displayed. The first participant can set one or more support DBs 1000 to be searched by selecting the check box next to the names of the plurality of support DBs 100.

[0118] As described above, according to the second embodiment, the communication support system 1 can support communication by using a more appropriate support DB 1000 according to the communication, from among the plurality of support DBs 1000 managed by the DB management unit 506.

[0119] As described above, according to each embodiment of the present invention, it becomes easy for a participant participating in communication to obtain information for supporting the communication during the communication.

[0120] <Supplementary Note> Each function of each of the embodiments described above can be realized by one or a plurality of processing circuits. Here, the "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, an ASIC (Application Specific Integrated Circuit) designed to execute each function described above, a DSP (digital signal processor), an FPGA (field programmable gate array), and devices such as conventional circuit modules.

[0121] In addition, the device group described in the embodiments merely represents one of the multiple computing environments for implementing the embodiments disclosed in this specification. In one embodiment, the communication support device 10 includes a plurality of computing devices such as a server cluster. The plurality of computing devices are configured to communicate with each other via any type of communication link including a network or a shared memory, and implement the processes disclosed in this specification.

[0122] Furthermore, the communication support device 10, the terminal device 100, and the information terminal 101 can be configured to share the disclosed processing steps, for example, the processes shown in FIGS. 7, 8, 11, 13, etc. in various combinations. For example, the process executed by the communication support device 10 can be executed by the terminal device 100 or the information terminal 101. Also, the processes executed by the terminal device 100 and the processes executed by the information terminal 101 can be executed by the communication support device 10. Furthermore, each element of the communication support device 10 may be integrated into one device or divided into multiple devices.

[0123] <Appendix> This specification discloses a communication support system, a communication support device, a communication support method, and a program according to each of the following items. (Item 1) An acquisition unit that acquires the speech audio of a participant participating in the communication; A determination unit that determines whether it is necessary to support the communication from the speech audio of the participant in response to the speech of the participant; An information providing unit that provides information for supporting the communication when it is determined that it is necessary to support the communication; A communication support system having the above. (Item 2) The determination unit detects one or more predetermined keywords from the speech audio of the participant; When the predetermined keyword is detected, the information providing unit provides information for assisting the communication corresponding to the detected predetermined keyword. The communication support system according to claim 1. (Item 3) The acquisition unit acquires the speech voice of a second participant different from the first participant from the terminal device used by the first participant participating in the communication. The determination unit detects the predetermined keyword from the speech voice of the second participant. The communication support system according to claim 2. (Item 4) The information providing unit causes the terminal device used by the first participant to display the information for assisting the communication. The communication support system according to claim 3. (Item 5) The information providing unit provides information for assisting the communication in real time according to the speech of the second participant. The communication support system according to claim 3 or 4. (Item 6) The acquisition unit acquires the speech voice of the second participant from the terminal device that transmits the speech voice of the participant including the speech voice of the second participant. The communication support system according to any one of claims 3 to 5. (Item 7) The determination unit detects the predetermined keyword registered in the support database in which the information for assisting the communication is registered. The first participant can select a support database for detecting the predetermined keyword. The communication support system according to any one of claims 3 to 6. (Item 8) The support database includes a plurality of questions and answers to the questions. The information providing unit causes the terminal device used by the first participant to display the question corresponding to the predetermined keyword detected by the determination unit among the plurality of questions and the answer to the question. The communication support system according to claim 7. (Claim 9) The communication support system according to claim 7 or 8, wherein the support database includes one or more databases among a database regarding FAQs, a database regarding assumed questions and answers, a database regarding product information, a database regarding industry information, and a database regarding conversation information. (Claim 10) The communication support system according to claim 2, wherein the information providing unit does not provide information for supporting the communication when the predetermined keyword is not detected. (Claim 11) An acquisition unit that acquires the speech voice of a participant participating in the communication, A determination unit that determines whether it is necessary to support the communication from the speech voice of the participant in response to the speech of the participant, An information providing unit that provides information for supporting the communication when it is determined that it is necessary to support the communication, A communication support apparatus having: (Claim 12) A computer performs A process of acquiring the speech voice of a participant participating in the communication, A process of determining whether it is necessary to support the communication from the speech voice of the participant in response to the speech of the participant, A process of providing information for supporting the communication when it is determined that it is necessary to support the communication, A communication support method for executing: (Claim 13) A process of acquiring the speech voice of a participant participating in the communication, A process of determining whether it is necessary to support the communication from the speech voice of the participant in response to the speech of the participant, A process of providing information for supporting the communication when it is determined that it is necessary to support the communication, A program that causes a computer to execute.

[0124] As described above, the embodiments of the present invention have been described. However, the present invention is not limited to such specific embodiments, and various modifications and applications are possible within the scope of the gist of the present invention described in the claims.

Explanation of Reference Numerals

[0125] 1 Communication support system 10 Communication support device 100, 100a to 100d Terminal devices 101 Information terminal 300 Computer 502 Acquisition unit 504 Determination unit 505 Information providing unit 506 DB management unit 507 Setting reception unit

Prior Art Documents

Patent Documents

[0126]

Patent Document 1

Claims

1. An acquisition unit that acquires the speech voice of a participant participating in a communication; A determination unit that determines whether it is necessary to support the communication from the speech voice of the participant in response to the speech of the participant; An information providing unit that provides information for supporting the communication when it is determined that it is necessary to support the communication; A communication support system comprising:

2. The determination unit detects one or more predetermined keywords from the speech voice of the participant; The information providing unit provides information for supporting the communication corresponding to the detected predetermined keyword when the predetermined keyword is detected; The communication support system according to claim 1.

3. The acquisition unit acquires the speech voice of a second participant different from the first participant from the terminal device used by the first participant participating in the communication; The determination unit detects the predetermined keyword from the speech voice of the second participant. The communication support system according to claim 2.

4. The information providing unit causes the terminal device used by the first participant to display the information for supporting the communication. The communication support system according to claim 3.

5. The information providing unit provides information for supporting the communication in real time in response to the speech of the second participant. The communication support system according to claim 3 or 4.

6. The acquisition unit acquires the speech voice of the second participant from the terminal device that transmits the speech voice of the participant including the speech voice of the second participant. The communication support system according to claim 3.

7. The determination unit detects the predetermined keyword registered in the support database in which the information for supporting the communication is registered; The first participant can select a support database for detecting the predetermined keyword. The communication support system according to claim 3.

8. The support database includes a plurality of questions and answers to the questions; The information providing unit causes the terminal device used by the first participant to display the question corresponding to the predetermined keyword detected by the determination unit among the plurality of questions and the answer to the question. The communication support system according to claim 7.

9. The communication support system according to claim 7 or 8, wherein the support database includes one or more databases among a database related to FAQs, a database related to assumed questions and answers, a database related to product information, a database related to industry information, and a database related to conversation information.

10. The communication support system according to claim 2, wherein the information providing unit does not provide information for supporting the communication when the predetermined keyword is not detected.

11. An acquisition unit that acquires the speech voice of a participant participating in the communication; A determination unit that determines whether it is necessary to support the communication from the speech voice of the participant in response to the speech of the participant; An information providing unit that provides information for supporting the communication when it is determined that it is necessary to support the communication; A communication support apparatus having the above.

12. A computer performs: A process of acquiring the speech voice of a participant participating in the communication; A process of determining whether it is necessary to support the communication from the speech voice of the participant in response to the speech of the participant; A process of providing information for supporting the communication when it is determined that it is necessary to support the communication; A communication support method.

13. A process of acquiring the speech voice of a participant participating in the communication; A process of determining whether it is necessary to support the communication from the speech voice of the participant in response to the speech of the participant; A process of providing information for supporting the communication when it is determined that it is necessary to support the communication; A program for causing a computer to execute the above.

Images