Communication support apparatus, communication support system, communication support method, and program

The communication support device facilitates easy feedback from non-participating parties by receiving operation information and displaying comments on participants' terminals, improving interaction quality in communication systems.

JP2024168292APending Publication Date: 2024-12-05RICOH CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023084840
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-23
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Conventional communication systems, such as conference systems, telemedicine, counseling, and remote lectures, face challenges in allowing participants to easily receive comments or advice from non-participating parties during interactions.

Method used

A communication support device that receives operation information from a participant's terminal, notifies a predetermined notification destination, and displays comments on the participant's terminal, facilitating easy feedback from related parties.

Benefits of technology

Enables participants to easily obtain comments from related parties, enhancing interaction quality in communication systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024168292000001_ABST
    Figure 2024168292000001_ABST
Patent Text Reader

Abstract

To provide a communication support apparatus which allows a participant participating communication to easily receive a comment from a person concerned.SOLUTION: A communication support apparatus according to the present invention has: a reception unit for receiving operation information indicating reception of a predetermined operation transmitted from a terminal device used by a participant participating communication; a notification unit for, upon reception of the predetermined operation, notifying a notification destination previously associated with the participant of the notification indicating reception of the predetermined operation; and a display control unit for, in order to display a comment input at the notification destination in response to the notification on the terminal device used by the participant, transmitting the comment to the terminal device used by the participant.SELECTED DRAWING: Figure 5
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a communication support device, a communication support system, a communication support method, and a program. [Background technology]

[0002] 2. Description of the Related Art There is a communication system in which content data such as video and audio is transmitted and received between terminal devices.

[0003] Furthermore, in a sales activity support system that manages the progress of cases and supports sales activities, a technique is known that notifies the salesperson in charge of a case that meets predetermined stagnation conditions and his / her manager that there is a stagnant case (for example, see Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0004] For example, in a conference system for conducting online business negotiations, a sales representative may want to get advice from relevant parties such as a manager or other members so that he or she can answer as many questions as possible during the business negotiation without having to take them home with him or her.

[0005] However, in conventional conference systems, it is difficult for a salesperson to receive advice or other comments from other parties while also attending to a client during a conference. This problem is not limited to conference systems used for business negotiations. For example, it is commonly found in various communication systems used for communication in remote medical care, counseling, remote lectures, school consultations, etc.

[0006] One embodiment of the present invention has been made in consideration of the above problems, and provides a communication support device that makes it easy for participants in a communication to obtain comments from related parties. [Means for solving the problem]

[0007] In order to solve the above problems, a communication support device according to one embodiment of the present invention has a receiving unit that receives operation information indicating that a specified operation transmitted from a terminal device used by a participant participating in the communication has been received, a notification unit that, when the operation information is received, sends a notification indicating that the specified operation has been received to a notification destination pre-associated with the participant, and a display control unit that sends a comment entered at the notification destination in response to the notification to the terminal device used by the participant so that the comment can be displayed on the terminal device used by the participant. Effect of the Invention

[0008] According to one embodiment of the present invention, it is possible to provide a communication support device that makes it easy for participants in a communication to obtain comments from related parties. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating an example of a system configuration of a communication support system according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating another example of a system configuration of the communication support system according to an embodiment. [Diagram 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer according to an embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of a hardware configuration of a terminal device according to an embodiment. [Diagram 5] FIG. 1 is a diagram illustrating an example of a functional configuration of a communication support system according to an embodiment. [Figure 6] 10 is a flowchart illustrating an example of processing of a terminal device according to an embodiment. [Figure 7] FIG. 11 is a diagram illustrating an example of an operation screen of a terminal device according to an embodiment. [Figure 8] FIG. 11 is a diagram illustrating an example of a display screen of a terminal device according to an embodiment. [Figure 9] 11 is a flowchart illustrating an example of a speech voice receiving process according to an embodiment. [Figure 10] FIG. 2 is a sequence diagram showing an example of a process of the communication support system according to the first embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of group information according to an embodiment. [Figure 12] FIG. 1 is a diagram showing an example of a display screen of an information terminal according to an embodiment. [Figure 13] FIG. 2 is a diagram showing an example of a display screen of the information terminal according to the embodiment; [Figure 14] FIG. 4 is a sequence diagram showing another example of the processing of the communication support system according to the first embodiment. [Figure 15] FIG. 11 is a diagram illustrating an example of a functional configuration of a communication support system according to a second embodiment. [Figure 16] FIG. 11 is a sequence diagram showing an example of a process of a communication support system according to the second embodiment. [Figure 17] FIG. 1 is a diagram showing an example of an information provision screen according to an embodiment; [Figure 18] FIG. 2 is a diagram showing an example of an information provision screen according to an embodiment; [Figure 19] 13 is a flowchart illustrating an example of a providing process of an information providing screen according to an embodiment. [Figure 20] 13 is a flowchart illustrating an example of a comment section display process according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention (the present embodiment) will be described in detail with reference to the drawings.

[0011] <System configuration> 1 and 2 are diagrams showing an example of a system configuration of a communication support system according to an embodiment. As shown in FIG. 1, the communication support system 1 includes a communication support device 10 connected to a communication network N such as the Internet or a LAN (Local Area network), a conference server 20, a plurality of terminal devices 100a, 100b, 100c, 100d, and one or more information terminals 101. In the following description, when referring to any of the plurality of terminal devices 100a, 100b, 100c, 100d, the term "terminal device 100" is used. The number of terminal devices 100 shown in FIG. 1 is an example.

[0012] The communication support system 1 is a system that supports various communications, such as business negotiations, meetings, medical treatments, lectures, and counseling, by transmitting and receiving at least audio between a terminal device 100 and one or more other terminal devices 100. Here, the following description will be given as an example in which the communication support system 1 supports a web conference for business negotiations by transmitting and receiving video including audio between a participant using the terminal device 100 and another participant using another terminal device 100.

[0013] The terminal device 100 is a general-purpose information terminal such as a PC (Personal Computer), a tablet terminal, or a smartphone used by a participant in a web conference. Alternatively, the terminal device 100 may be an electronic device having a web conference function such as a video conference device or an IWB (Interactive White Board). An IWB is an electronic whiteboard having a function of a blackboard that allows mutual communication, and is also called an electronic blackboard. Here, as an example, the following description will be given assuming that the terminal device 100 is a general-purpose information terminal.

[0014] Participants in a web conference join the web conference by, for example, using a web conference application installed on the terminal device 100 or a web browser, etc., by accessing the conference address provided by the conference server 20.

[0015] The communication support device 10 is, for example, an information processing device having a computer configuration or a system including a plurality of computers. The communication support device 10 provides the communication support service according to the present embodiment.

[0016] The conference server 20 is an information processing device having a computer configuration, or a system including multiple computers. The conference server 20 provides a web conference service for transmitting and receiving audio or video including audio between multiple terminal devices 100. In this embodiment, the web conference service provided by the conference server 20 may be any web conference service.

[0017] The one or more information terminals 101 are information terminals such as PCs, tablet terminals, or smartphones that are used by an administrator (manager) of an organization that uses the communication support service according to this embodiment, and by members, etc. The administrator, etc., uses the communication support service by using, for example, a web browser provided in the information terminal 101, or an application program (hereinafter referred to as an app) for the communication support system 1 executed by the information terminal 101, etc.

[0018] The system configuration of the communication support system 1 shown in FIG. 1 is an example. For example, the communication support system 1 may not have the conference server 20 as shown in FIG. 2. In this case, the terminal device 100 may hold a web conference using a web conference service external to the communication support system 1. Alternatively, the communication support device 10 may have the function of the conference server 20, and the terminal device 100 may hold a web conference using a web conference service provided by the communication support device 10. Furthermore, the communication support system 1 may support communication in a general conference in which a participant and other participants hold a conference in the same conference room using the terminal device 100a.

[0019] (Processing Overview) For example, when a business negotiation is being conducted using a web conference provided by the conference server 20, a participant in the business negotiation (e.g., a sales representative) may be asked a question by another participant in the business negotiation (e.g., a client) that is difficult to answer immediately.

[0020] In such cases, participants may want to answer the questions from the client as much as possible during the business meeting, rather than taking them home with them, so they may use messaging services or chat, for example, to check with other parties about things they don't understand.

[0021] As described above, in the conventional technology, it is difficult to obtain comments during a meeting from other participants who are not participating in the meeting.

[0022] Therefore, the present embodiment provides a communication support system 1 that makes it easy for participants in a communication to obtain comments from related parties who are not participating in the communication.

[0023] For example, the terminal device 100 executes an application program (hereinafter, referred to as an app) that transmits speech sounds of the participants participating in the conference and other participants to the communication support device 10 and accepts predetermined operations by the participants. This app causes the device 100 to display an operation screen including display elements (such as a "send SOS" button, which will be described later) that allow the participants to ask for advice from other people not participating in the conference.

[0024] When an application running on the terminal device 100 accepts a predetermined operation, the communication support device 10 notifies a preset notification destination (for example, the information terminal 101 used by the other participants) of the acceptance of the predetermined operation. In response to the notification, the communication support device 10 causes the terminal device 100 used by the participant to display a comment input at the notification destination.

[0025] Preferably, the notification sent by the communication support device 10 to the notification destination includes at least a part of a character string obtained by converting the speech of the participant and the other participants into text. This makes it easier for the other participants who receive the notification to understand the content of the comment that the participant needs. In addition, the participant can ask the other participants for advice without inputting the status of the meeting or the content about which they want advice.

[0026] In this manner, according to the present embodiment, it is possible to provide a communication support device that enables participants in a communication to easily obtain comments from related parties.

[0027] <Hardware configuration> (Computer hardware configuration) The communication support device 10, the conference server 20, and the information terminal 101 each have, for example, the hardware configuration of a computer 300 as shown in Fig. 3. Alternatively, the communication support device 10 or the conference server 20 may be configured by a plurality of computers 300. Furthermore, the terminal device 100 may have, for example, the hardware configuration of a computer 300 as shown in Fig. 3.

[0028] Fig. 3 is a diagram showing an example of a hardware configuration of a computer according to an embodiment. For example, as shown in Fig. 3, a computer 300 includes a CPU (Central Processing Unit) 301, a ROM (Read Only Memory) 302, a RAM (Random Access Memory) 303, a HD (Hard Disk) 304, a HDD (Hard Disk Drive) controller 305, a display 306, an external device connection I / F (Interface) 307, a network I / F 308, a keyboard 309, a pointing device 310, a DVD-RW (Digital Versatile Disk Rewritable) drive 312, a media I / F 314, and a bus line 315.

[0029] Furthermore, when the computer 300 is the terminal device 100, the computer 300 further includes, for example, a microphone 321, a speaker 322, an audio input / output I / F 323, a CMOS (Complementary Metal Oxide Semiconductor) sensor 324, and an imaging element I / F 325.

[0030] Of these, the CPU 301 controls the overall operation of the computer 300. The ROM 302 stores programs used to start up the computer 300, such as an IPL (Initial Program Loader). The RAM 303 is used, for example, as a work area for the CPU 301. The HD 304 stores programs such as an OS (Operating System), applications, device drivers, and various data. The HDD controller 305 controls the reading and writing of various data from and to the HD 304, for example, under the control of the CPU 301. The HD 304 and the HDD controller 305 are examples of storage devices included in the computer 300.

[0031] The display 306 displays various types of information such as a cursor, a menu, a window, characters, or an image. The display 306 may be provided outside the computer 300. The external device connection I / F 307 is an interface for connecting various external devices to the computer 300. The network I / F 308 is an interface for connecting the computer 300 to a communication network N and communicating with other devices.

[0032] The keyboard 309 is a type of input means having a plurality of keys for inputting characters, numbers, various instructions, etc. The pointing device 310 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. Note that the keyboard 309 and the pointing device 310 may be provided outside the computer 300.

[0033] The DVD-RW drive 312 controls reading and writing of various data from and to a DVD-RW 311, which is an example of a removable recording medium. The DVD-RW 311 is not limited to a DVD-RW, and may be another recording medium. The media I / F 314 controls reading and writing (storing) of data from and to a medium 313 such as a flash memory. The bus line 315 includes an address bus, a data bus, various control signals, and the like for electrically connecting the above-mentioned components.

[0034] The microphone 321 is a built-in circuit that converts sound into an electrical signal. The speaker 322 is a built-in circuit that converts the electrical signal into physical vibrations to generate sound such as music or voice. The audio input / output I / F 323 is a circuit that processes input and output of audio signals between the microphone 321 and the speaker 322 under the control of the CPU 301.

[0035] The CMOS sensor 324 is a type of built-in imaging means that captures an image of a subject (e.g., a self-portrait) and obtains image data under the control of the CPU 301. Note that the computer 300 may have an imaging means such as a CCD (Charge Coupled Device) sensor instead of the CMOS sensor 324. The imaging element I / F 325 is a circuit that controls the driving of the CMOS sensor 324.

[0036] (An example of the hardware configuration of a terminal device) 4 is a diagram showing an example of a hardware configuration of a terminal device according to an embodiment. Here, an example of the hardware configuration of the terminal device 100 in the case where the terminal device 100 is an information terminal such as a smartphone or a tablet terminal will be described. Note that the terminal device 100 may be, for example, an electronic device having a computer configuration such as a video conferencing device or an IWB and a web conferencing function. In addition, the information terminal 101 may have the hardware configuration of a computer 300 as shown in FIG. 3.

[0037] In the example of FIG. 4, the terminal device 100 includes a CPU 401, a ROM 402, a RAM 403, a storage device 404, a CMOS sensor 405, an image sensor I / F 406, an acceleration / direction sensor 407, a media I / F 409, a GPS (Global Positioning System) receiver 410, and the like.

[0038] Of these, the CPU 401 executes a predetermined program to control the overall operation of the terminal device 100. The ROM 402 stores a program used to start up the CPU 401, such as an IPL. The RAM 403 is used as a work area for the CPU 401. The storage device 404 is a large-capacity storage device that stores programs such as an OS and applications, and various data, and is realized by, for example, an SSD (Solid State Drive) or a flash ROM.

[0039] The CMOS sensor 405 is a type of built-in imaging means that captures an image of a subject (mainly a self-portrait) under the control of the CPU 401 to obtain image data. The terminal device 100 may have an imaging means such as a CCD sensor instead of the CMOS sensor 405. The imaging element I / F 406 is a circuit that controls the driving of the CMOS sensor 405. The acceleration / direction sensor 407 is various sensors such as an electronic magnetic compass or gyrocompass that detects geomagnetism, and an acceleration sensor. The media I / F 409 controls the reading or writing (storage) of data from a media (storage media) 408 such as a flash memory. The GPS receiver 410 receives a GPS signal (positioning signal) from a GPS satellite.

[0040] The terminal device 100 also includes a long-distance communication circuit 411, an antenna 411a of the long-distance communication circuit 411, a CMOS sensor 412, an image sensor I / F 413, a microphone 414, a speaker 415, an audio input / output I / F 416, a display 417, an external device connection I / F 418, a short-distance communication circuit 419, an antenna 419a of the short-distance communication circuit 419, and a touch panel 420.

[0041] Of these, the long-distance communication circuit 411 is a circuit that communicates with other devices, for example, via the communication network N. The CMOS sensor 412 is a type of built-in imaging means that captures an object under the control of the CPU 401 to obtain image data. The image sensor I / F 413 is a circuit that controls the driving of the CMOS sensor 412. The microphone 414 is a built-in circuit that converts sound into an electrical signal. The speaker 415 is a built-in circuit that converts an electrical signal into physical vibrations to generate sound such as music or voice. The audio input / output I / F 416 is a circuit that processes the input and output of audio signals between the microphone 414 and the speaker 415 under the control of the CPU 401.

[0042] The display 417 is a type of display means such as a liquid crystal or an organic EL (Electro Luminescence) display that displays an image of a subject, various icons, etc. The external device connection I / F 418 is an interface for connecting various external devices. The short-distance communication circuit 419 includes a circuit for performing short-distance wireless communication. The touch panel 420 is a type of input means that allows a user to operate the terminal device 100 by pressing the display 417.

[0043] The terminal device 100 also includes a bus line 421. The bus line 421 includes an address bus, a data bus, various control signals, and the like for electrically connecting the components such as the CPU 401 shown in FIG.

[0044] 4 is an example of the hardware configuration of the terminal device 100. The terminal device 100 may have various other hardware configurations as long as it has a computer configuration, a communication circuit, a display, a microphone, a speaker, and the like.

[0045] <Functional configuration> Next, a description will be given of the functional configuration of the communication support system 1. Fig. 5 is a diagram showing an example of the functional configuration of the communication support system according to an embodiment.

[0046] (Functional configuration of terminal device) Here, the following description will be given on the assumption that the terminal device 100 has the hardware configuration of a computer 300 as shown in FIG.

[0047] The terminal device 100 executes a predetermined program in the CPU 301 to realize functional configurations such as a communication unit 511, a conference control unit 512, an audio transmission unit 513, a display unit 514, and an operation reception unit 515. At least a part of the above functional configurations may be realized by hardware.

[0048] The communication unit 511 uses, for example, the network I / F 308 or the like to connect the terminal device 100 to the communication network N and executes communication processing for communicating with other devices.

[0049] The conference control unit 512 executes a series of processes related to a web conference, such as connecting to a conference, sending and receiving conference video (or conference audio), and inputting and outputting conference video (or conference audio). Note that the processes executed by the conference control unit 512 may be similar to those of a general web conference. Note that a conference and a web conference are examples of communication according to this embodiment.

[0050] The voice transmitting unit 513 executes a voice transmitting process of acquiring voices of participants participating in the conference and other participants, and transmitting the acquired voices (voice data) to the communication support device 10. For example, the voice transmitting unit 513 acquires voices, such as voice acquired by the microphone 321 and voice output from the speaker 322, from the sound input / output I / F 323, and transmits the acquired voices to the communication support device 10. Thereby, the communication support system 1 is not limited to web conferences, and can also support communication in normal conferences, etc.

[0051] The display unit 514 executes a display process for displaying a display screen on the display 306, for example. The operation accepting unit 515 executes an operation accepting process for accepting an operation by a participant using an input device such as the keyboard 309 or the pointing device 310, for example. For example, the operation accepting unit 515 accepts a predetermined operation on the operation screen displayed by the display unit 514, and transmits operation information, which is information on the accepted predetermined operation, to the communication support device 10 via the communication unit 511.

[0052] (Functional configuration of communication support device) The communication support device 10 realizes functional configurations such as a communication unit 501, an acquisition unit 502, a conversion unit 503, a reception unit 504, a notification unit 505, a display control unit 506, a data processing unit 507, and an information providing unit 508 by executing a predetermined program on one or more computers 300. Note that at least a part of the above functional configurations may be realized by hardware.

[0053] The communication support apparatus 10 also implements a storage unit 509 using storage devices such as the HD 304 and the HDD controller 305 .

[0054] The communication unit 501 uses, for example, the network I / F 308 or the like to connect the communication support device 10 to the communication network N and executes communication processing for communicating with other devices.

[0055] The acquisition unit 502 executes an acquisition process for acquiring the speech (speech data) of the participant participating in the conference (communication) and other participants. For example, the acquisition unit 502 acquires the speech of the participant and the speech of the other participants that are transmitted to the communication support device 10 by the voice transmission unit 513 of the terminal device 100 used by the participant, and stores the acquired speech in the storage unit 509 or the like.

[0056] The conversion unit 503 executes a conversion process to convert the speech of the participant and the other participants acquired by the acquisition unit 502 into text. For example, the conversion unit 503 converts the speech of the participant and the speech of the other participants into text, and stores the text character string in the storage unit 509 or the like.

[0057] The reception unit 504 executes a reception process for receiving a predetermined operation by a participant participating in a conference. For example, the reception unit 504 receives, from the terminal device 100, operation information indicating that a predetermined operation for requesting advice from a notification destination has been received, thereby receiving the predetermined operation by the participant.

[0058] When the receiving unit 504 receives a predetermined operation, the notification unit 505 notifies a notification destination associated with the participant in advance of a notification indicating that the predetermined operation has been received. For example, the communication support device 10 stores in advance group information, etc. in which information on managers and members belonging to the same group (organization, etc.) as the participant participating in the conference is registered. Furthermore, when the receiving unit 504 receives a predetermined operation, the notification unit 505 notifies the manager or members, etc. of the same group as the participant that the predetermined operation by the participant has been received, based on the group information, etc. stored in the storage unit 509, etc. Note that the manager of the same group is an example of a manager corresponding to the participant.

[0059] In response to the notification from the notification unit 505, the display control unit 506 executes a display control process to display the comment entered at the notification destination by transmitting it to the terminal device 100 used by the participant via the communication unit 501.

[0060] Here, the display control process also includes transmitting information such as comments displayed on the terminal device 100.

[0061] The data processing unit 507 calculates indicators (for example, speech volume, speech rate, or speech speed) indicating the speech state of the participant and other participants from the speech voice (speech data) received from the terminal device 100. In addition, the data processing unit 507 stores and manages the speech voice received from the terminal device 100, the calculated indicators, operation information received from the terminal device 100, comments corresponding to the operation information, etc. in the storage unit 509, etc.

[0062] The information providing unit 508 provides an information providing screen that displays indicators showing the speech status of the participant and other participants in chronological order based on the speech audio acquired by the acquisition unit 502, and displays a marker corresponding to the time when a specified operation is received.

[0063] The memory unit 509 stores, for example, the speech voice acquired by the acquisition unit 502, the character string converted by the conversion unit 503, information on a specified operation accepted by the acceptance unit 504, the index calculated by the data processing unit 507, and setting information.

[0064] (Functional configuration of the conference server) The conference server 20 realizes functional configurations such as a communication unit 521, a conference management unit 522, and a relay unit 523 by executing a predetermined program by one or more computers 300. Note that at least a part of the above functional configurations may be realized by hardware.

[0065] The communication unit 521 uses, for example, the network I / F 308 or the like to connect the conference server 20 to the communication network N and executes communication processing for communicating with other devices.

[0066] The conference management unit 522 executes a conference management process for allowing a plurality of terminal devices 100 accessing the same Uniform Resource Locator (URL) for a Web conference to participate in the same conference (session).

[0067] The relay unit 523 executes a relay process for relaying content data including audio and images between multiple terminal devices 100 participating in the same conference. Note that the functions of the conference management unit 522 and the relay unit 523 may be those of an existing web conference system.

[0068] In this embodiment, the conference server 20 may be any web conference service external to the communication support system 1.

[0069] (Functional configuration of administrator terminal) The information terminal 101 realizes a communication unit 531, a display unit 532, an operation receiving unit 533, and the like, for example, by the computer 300 executing a predetermined program. Note that at least a part of the above-mentioned functional configurations may be realized by hardware.

[0070] The communication unit 531 uses, for example, the network I / F 308 or the like to connect the information terminal 101 to the communication network N and executes communication processing for communicating with other devices.

[0071] The display unit 532 executes a display process for displaying, for example, a display screen provided from the information providing unit 508 or the like and a notification or the like notified from the notification unit 505 on the display 306 or the like. The operation accepting unit 533 executes an operation accepting process for accepting, for example, an input operation or a selection operation by an administrator or the like on the display screen displayed by the display unit 532. Note that the display unit 532 and the operation accepting unit 533 may be realized by, for example, a web browser or the like provided in the information terminal 101. For example, the information terminal 101 may be a general-purpose information terminal or the like provided with a web browser.

[0072] 5 is an example. For example, the function of the conference server 20 may be included in the communication support device 10, or may be included in a computer 300 external to the communication support system 1, a cloud service, or the like. At least a part of the functional configuration of the communication support device 10 may be included in the terminal device 100, the information terminal 101, or the like. At least a part of the functional configuration of the terminal device 100 may be included in the communication support device 10, the conference server 20, or the like. In short, each functional configuration of the communication support system 1 shown in FIG. 9 may be included in any of the devices included in the communication support system 1.

[0073] <Processing flow> Next, the process flow of the information processing assistance method according to this embodiment will be described.

[0074] (Terminal device processing) Fig. 6 is a flowchart showing an example of processing of a terminal device according to an embodiment. This processing shows an example of processing executed by the terminal device 100 having the functional configuration described in Fig. 5, for example. At the start of the processing shown in Fig. 6, the terminal device 100 used by a participant (for example, the terminal device 100a) participates in the same conference as the terminal device used by another participant (for example, the terminal device 100b), and is in a state where at least voice conversation is possible.

[0075] In step S601, when the operation receiving unit 515 receives an operation to start recording, the terminal device 100 executes the processes in step S602 and thereafter. For example, by executing an application corresponding to the communication support system 1 on the terminal device 100, the display unit 514 displays an operation screen 710 as shown in Fig. 7(A) on the display 306 or the like. A participant can start recording of spoken voice by selecting a "record" button 711 on the operation screen 710.

[0076] In step S602, the voice transmitting unit 513 starts a process of transmitting the spoken voice. For example, the voice transmitting unit 513 obtains the spoken voice, such as the voice acquired by the microphone 321 and the voice output from the speaker 322, from the sound input / output I / F 323, and transmits the obtained spoken voice to the communication support device 10.

[0077] In step S603, the operation receiving unit 515 judges whether or not a predetermined operation by a participant has been received. For example, the display unit 514 displays an operation screen 720 as shown in Fig. 7(B) on the display 306 or the like while recording the uttered voice. The participant can perform an operation to stop recording of the uttered voice by selecting a "Stop" button 721 on the operation screen 720 shown in Fig. 7(B). In addition, the participant can perform a predetermined operation to request advice from the notification destination by selecting a "Send SOS" button 722.

[0078] The operation of selecting the "Send SOS" button 722 is an example of a predetermined operation. For example, the display unit 514 may display an operation screen 730 as shown in FIG. 7(C) on the display 306 or the like while recording the spoken voice. In addition to the "Send SOS" button 722, a "Bookmark" button 731 is displayed on the operation screen 730. By selecting the "Bookmark" button 731, the participant can perform a predetermined operation to mark the current spoken voice. The operation of selecting the "Bookmark" button 731 is another example of a predetermined operation.

[0079] If a predetermined operation has been received, the operation receiving unit 515 shifts the process to step S604. On the other hand, if a predetermined operation has not been received, the operation receiving unit 515 shifts the process to step S605.

[0080] In step S604, the operation accepting unit 515 transmits, to the communication support device 10 via the communication unit 511, operation information indicating that a predetermined operation has been accepted.

[0081] In step S605, the display unit 514 determines whether or not a comment has been received from the communication support device 10, based on the operation information transmitted by the operation receiving unit 515. If a comment has been received, the display unit 514 shifts the process to step S606. On the other hand, if a comment has not been received, the display unit 514 shifts the process to step S607.

[0082] In step S606, the display unit 514 displays the comment received from the communication support device 10 on the display 306 or the like.

[0083] Fig. 8 is a diagram showing an example of a display screen of a terminal device according to an embodiment. This diagram shows an image of a display screen 800 that the display unit 514 displays on the display 306 or the like in step S606 in Fig. 6. In the example of Fig. 8, the operation screen 720, the conference screen 810, the comment screen 820, and the like described in Fig. 7 are displayed on the display screen 800.

[0084] The conference screen 810 is a screen for a conference system that displays, for example, a conference video of the participants and other participants, or a shared screen for sharing a screen. The display unit 514 displays, for example, a comment screen 820 superimposed on the conference screen 810, and displays comments 821 from other participants received from the communication support device 10 on the comment screen 820. This allows the participants to easily check the comments from other participants even when, for example, the operation screen 720 is minimized or is displayed behind the conference screen 810.

[0085] In step S607, operation acceptance unit 515 determines whether or not an operation to stop recording has been accepted. If an operation to stop recording has not been accepted, operation acceptance unit 515 returns the process to step S603 and executes the processes of steps S603 to S607 again. On the other hand, if an operation to stop recording has been accepted, operation acceptance unit 515 transitions the process to step S608.

[0086] In step S608, the voice transmitting unit 513 stops or ends the transmission process of the uttered voice.

[0087] Through the process of FIG. 6, the terminal device 100 can transmit the speech sounds of the participant and other participants to the communication support device 10, and can notify the communication support device 10 that a predetermined operation by the participant has been accepted.

[0088] (Speech reception and processing) 9 is a flowchart illustrating an example of a speech voice reception process according to an embodiment. This process illustrates an example of a speech voice reception process that is executed when the communication support device 10 receives the speech voice from the terminal device 100.

[0089] In step S901, when the acquiring unit 502 starts receiving the speech voices of the participant and other participants transmitted by the terminal device 100, the communication support apparatus 10 executes the process proceeding to step S902.

[0090] In step S902, the conversion unit 503 executes a conversion process to convert the speech acquired by the acquisition unit 502 into text. The process of converting speech into text may be called, for example, transcription or text conversion. The conversion unit 503 may convert the speech acquired by the acquisition unit 502 into text by using, for example, a transcription server, a text conversion server, or a text conversion server.

[0091] In step S903, the data processing unit 507 stores the character string converted by the conversion unit 503 in the storage unit 509, etc. Preferably, the data processing unit 507 adds time information or the like to the character string converted by the conversion unit 503, and stores the character string in the storage unit 509, etc. in chronological order for each conference ID.

[0092] In addition, the communication support apparatus 10 executes the processes of steps S904 and S905 in parallel with the processes of steps S902 and S903.

[0093] In step S904, the data processing unit 507 calculates the speech volume of the speech voice acquired by the acquisition unit 502. For example, the data processing unit 507 calculates the speech volume of the participant from the speech voice of the participant, and calculates the speech volume of the other participants from the speech voice of the other participants.

[0094] In step S905, the data processing unit 507 stores the speech voice acquired by the acquisition unit 502 and the speech volume calculated in step S904 in the storage unit 509, etc. Preferably, the data processing unit 507 adds time information to the acquired speech voice and the calculated speech volume, and stores them in the storage unit 509, etc. in chronological order for each conference ID.

[0095] In step S906, the communication support device 10 determines whether or not the reception of the uttered voice has been stopped. If the reception of the uttered voice has not been stopped, the communication support device 10 returns the process to step S902 and executes the processes of steps S902 to S906 again. On the other hand, if the reception of the uttered voice has been stopped, the communication support device 10 ends the process of FIG. 9.

[0096] 9, the communication support device 10 can store data such as the speech of the participant and other participants, character strings converted from the speech, and speech volume in the storage unit 509 based on the speech received from the terminal device 100. The speech volume is an example of an index indicating the speech situation of the participant and other participants. The index indicating the speech situation of the participant and other participants may be other indexes such as a speech rate, a speech ratio, or a speech speed.

[0097] <Processing of communication support system> [First embodiment] (Communication support system processing 1) Fig. 10 is a sequence diagram showing an example of a process of the communication support system according to the first embodiment. This process shows an example of a process executed by the communication support system 1 in parallel with, for example, the process of the terminal device described in Fig. 6 and the process of receiving the uttered voice described in Fig. 9.

[0098] In step S1001, it is assumed that a participant using the terminal device 100a selects the "Send SOS" button 722 on an operation screen 720 displayed on the terminal device 100, for example, as shown in FIG. 7(B).

[0099] In step S1002, when the operation acceptance unit 515 of the terminal device 100a accepts the selection operation of the “Send SOS” button 722, the operation acceptance unit 515 transmits to the communication support device 10 operation information indicating that the selection operation of the “Send SOS” button 722 has been accepted.

[0100] In step S1003, when the receiving unit 504 of the communication support device 10 receives the operation information transmitted by the terminal device 100, the notifying unit 505 collects notification information to be notified to a predetermined notification destination. For example, the notifying unit 505 of the communication support device 10 collects a conference ID for identifying a conference in which the terminal device 100 is participating, time information indicating the time when the predetermined operation was received, a character string obtained by converting a spoken voice into text (transcription result) before the predetermined operation was received, and the like.

[0101] In step S1004, the notification unit 505 specifies a notification destination to which the SOS notification is to be sent. For example, it is assumed that the storage unit 509 of the communication support device 10 stores group information 1100 as shown in Fig. 11 in advance. In the example of Fig. 11, the group information 1100 includes information such as "members", "job titles", and "email addresses" as items.

[0102] "Member" is information indicating, for example, the names of members who belong to a group such as the same organization. "Position" is information indicating the position or role of each member. "Email address" is information indicating the email address of each member.

[0103] In this case, it is assumed that the participant using the terminal device 100a is member B, and the administrator (manager) corresponding to member B is manager A. It is also assumed that each member logs in to the communication support device 10 using an email address as a login ID when using the communication support service provided by the communication support device 10.

[0104] In the first embodiment, the notification unit 505 will be described below as an example, where the notification unit 505 notifies an SOS notification to a manager (manager A) in the same group as a participant (member B) who is participating in a conference.

[0105] In step S1005, the notification unit 505 notifies the information terminal 101 used by the administrator (manager A) of an SOS notification. This SOS notification includes, for example, the conference ID collected in step S1003, time information, and at least a part of a character string (transcription result) obtained by converting a spoken voice into text before accepting a predetermined operation. For example, the notification unit 505 transmits the SOS notification to the information terminal 101 logged in to the communication support device 10 using the email address of the administrator (manager A).

[0106] In step S1006, the display unit 532 of the information terminal 101 displays the SOS notification received from the communication support device 10 on the display 306 of the information terminal 101 or the like.

[0107] Fig. 12 is a diagram (1) showing an example of a display screen of an information terminal according to an embodiment. An administrator or the like who uses the information terminal 101 can display, for example, a display screen 1200 as shown in Fig. 12 on the display 306 of the information terminal 101 by logging in to the communication support service provided by the communication support device 10. In the example of Fig. 12, the display screen 1200 displays a business negotiation list 1210 which is a list of business negotiations in which member B, member C, and member C, who belong to the same group, are participating or have participated in in the past.

[0108] Furthermore, when the display unit 532 of the information terminal 101 receives an SOS notification from the communication support device 10, it displays the received SOS notification 1221 in a notification area 1220 of the display screen 1200. In the example of Fig. 12, the SOS notification 1221 displays a title 1222 of the business negotiation, a character string 1223 in which the spoken content has been converted into text, and the like. By selecting this SOS notification 1221, the administrator can display, for example, a detailed screen 1300 of the SOS notification as shown in Fig. 13 on the display screen 1200.

[0109] For example, in step S1007, it is assumed that the administrator selects SOS notification 1221 on the display screen 1200. In response to this, in step S1008, the operation reception unit 533 of the information terminal 101 transmits operation information indicating that the SOS notification has been selected to the communication support device 10. In response to this, in step S1009, the notification unit 505 of the communication support device 10 transmits, for example, an SOS notification details screen 1300 as shown in FIG.

[0110] In step S1010, the display unit 532 of the information terminal 101 displays a details screen 1300 of the SOS notification notified from the communication support device 10 on the display screen 1200, for example, as shown in FIG.

[0111] The processes in steps S1008 to S1010 are merely examples. For example, the display unit 532 of the information terminal 101 may create and display a detailed SOS notification screen 1300 as shown in FIG. 13 based on the SOS notification received from the communication support device 10 in step S1005.

[0112] In the example of FIG. 13, the SOS notification details screen 1300 displays a character string 1301 that is a text conversion of the speech spoken by the participant and the speech spoken with other participants before the “Send SOS” button 722 was selected, and a comment field 1302.

[0113] The administrator can determine what kind of advice the participant (member B) is seeking from this character string 1301. For example, in the example of Fig. 13, when another participant (client X) uttered "Please tell me the operating speed of product A," the "Send SOS" button 722 was selected. Therefore, the administrator may determine that the participant (member B) is seeking advice on the specifications of the operating speed of product A.

[0114] In step S1011, it is assumed that the administrator inputs a comment in the comment field 1302. In the example of FIG. 13, the administrator inputs a comment indicating the movement speed of product A, such as "The movement speed is X m / s," in the comment field 1302, and selects the send button 1303.

[0115] In step S1012, the operation accepting unit 533 of the information terminal 101 transmits the comment input by the administrator to the communication support device 10.

[0116] In step S1013, the display control unit 506 of the communication support device 10 transmits the comment received from the information terminal 101 to the terminal device 100a, thereby causing the comment to be displayed on the terminal device 100a.

[0117] In step S1014, the display unit 514 of the terminal device 100a displays the comment received from the communication support device 10 on the display 306 or the like. For example, the display unit 514 displays a comment 821 received from the communication support device 10 on a comment screen 820 as shown in FIG.

[0118] 10, a participant in a conference can ask for advice from a related person such as a manager by simply selecting the "Send SOS" button 722 during the conference. Therefore, the communication support system 1 according to this embodiment makes it easy for a participant in a communication to get comments from a related person who is not participating in the communication.

[0119] (Communication support system processing 2) Fig. 14 is a sequence diagram showing another example of the processing of the communication support system according to the first embodiment. This processing shows another example of the processing executed by the communication support system 1 in parallel with, for example, the processing of the terminal device explained in Fig. 6 and the speech voice receiving processing explained in Fig. 9. Among the processing shown in Fig. 14, the processing of steps S1001 to 1003 and S1007 to S1014 are similar to the processing of steps S1001 to 1003 and S1007 to S1014 explained in Fig. 10, and therefore the explanation thereof will be omitted here.

[0120] In step S1401, the notification unit 505 transmits an SOS notification to the email address of the administrator (manager A). For example, the notification unit 505 transmits the SOS notification to the administrator by email, messaging service, etc. This SOS notification includes, for example, link information for displaying an SOS notification 1221 as shown in FIG. 12.

[0121] In step S1402, the administrator selects the link information of the SOS notification notified by email, messaging service, etc. As a result, the operation acceptance unit 533 of the information terminal 101 accesses the link destination indicated by the link information.

[0122] In step S1404, the notification unit 505 of the communication support device 10 transmits, for example, a display screen 1200 as shown in Fig. 12. As a result, in step S1405, the display unit 532 of the information terminal 101 displays, for example, an SOS notification 1221 as shown in Fig. 12 on the display 306 or the like.

[0123] In this manner, the communication support device 10 may notify the information terminal 101 of the SOS notification by using e-mail, a messaging service, or the like.

[0124] [Second embodiment] 15 is a diagram showing an example of a functional configuration of a communication support system according to the second embodiment. The communication support system 1 according to the second embodiment includes a plurality of information terminals 101a, 101c, and 101d. Here, it is assumed that the information terminal 101a is an information terminal used by a manager A included in the group information 1100 of FIG. 11. It is also assumed that the information terminals 101c and 101d are information terminals used by members C and D included in the group information 1100, respectively. It is also assumed that the information terminals 101a, 101c, and 101d have the same functional configuration as the information terminal 101 described in FIG. 5.

[0125] <Processing flow> Fig. 16 is a sequence diagram showing an example of processing of the communication support system according to the second embodiment. This processing shows another example of processing executed by the communication support system 1 in parallel with, for example, the processing of the terminal device explained in Fig. 6 and the speech voice receiving processing explained in Fig. 9. Among the processing shown in Fig. 16, the processing of steps S1001 to 1003 is similar to the processing of steps S1001 to 1003 explained in Fig. 10, and therefore the explanation thereof will be omitted here.

[0126] In step S1601, the notification unit 505 specifies a notification destination to which the SOS notification is to be sent. In the first embodiment, the notification unit 505 sends the SOS notification to a manager (manager A) in the same group as a participant (member B) who is participating in a conference, as an example.

[0127] On the other hand, in the second embodiment, the following description will be given assuming that the notification unit 505 notifies, as another example, members in the same group as a participant (member B) participating in a conference of an SOS notification.

[0128] For example, based on group information 1100 as shown in FIG. 11, the notification unit 505 identifies members (manager A, member C, and member D) in the same group as the participant (member B) attending the conference as the notification recipients of the SOS notification.

[0129] In steps S1602a and S1602b, the notification unit 505 sends an SOS notification to the information terminal 101a used by the administrator (manager A) and the information terminals 101c and 101d used by the other members (members C and D).

[0130] In step S1006, the display units 532 of the information terminals 101a, 101b, and 101c display the SOS notification received from the communication support device 10 on the display 306 or the like. For example, the display units 532 of the information terminals 101a, 101b, and 101c display an SOS notification 1221 as shown in FIG.

[0131] In step S1604, it is assumed that the administrator selects SOS notification 1221 on the display screen 1200 displayed by the information terminal 101a. In response to this, in step S1605, the operation accepting unit 533 of the information terminal 101a transmits operation information indicating that SOS notification has been selected to the communication support device 10.

[0132] In step S1606, the notification unit 505 of the communication support apparatus 10 transmits, for example, a detailed SOS notification screen 1300 as shown in FIG. 13 to the information terminal 101a.

[0133] In step S1607, the display unit 532 of the information terminal 101a displays a details screen 1300 of the SOS notification notified from the communication support device 10 on the display screen 1200, for example, as shown in FIG.

[0134] In step S1608, the administrator inputs a comment in the comment field 1302 of the SOS notification details screen 1300. In response to this, in step S1609, the operation reception unit 533 of the information terminal 101a transmits the comment input by the administrator to the communication support device 10.

[0135] In step S1610, the notification unit 505 of the communication support device 10 sends a comment notification to the information terminals 101c and 101d indicating that a comment has been input in response to the SOS notification. As a result, in step S1611, the display units 514 of the information terminals 101c and 101d display, on the displays 306 or the like, a notification indicating that a comment has been input in response to the SOS notification.

[0136] In step S1612, the display control unit 506 of the communication support device 10 transmits the comment received from the information terminal 101a to the terminal device 100a, thereby causing the comment to be displayed on the terminal device 100a.

[0137] In step S1613, the display unit 514 of the terminal device 100a displays the comment received from the communication support device 10 on the display 306 or the like. For example, the display unit 514 displays a comment 821 received from the communication support device 10 on a comment screen 820 as shown in FIG.

[0138] 16, for example, members C and D who use the information terminals 101c and 101d can easily confirm that the administrator has commented on the SOS notification. Also, if the administrator has not commented on the SOS notification, members C and D can determine that it is necessary to comment on the SOS notification on behalf of the administrator.

[0139] 16 is just an example. For example, the processes in steps S1604 to S1608 may be performed by member C using the information terminal 101c or member D using the information terminal 101d instead of the administrator.

[0140] (Modification) 16, the notification unit 505 of the communication support device 10 simultaneously performs the processes of steps S1602a and S1602b. However, this is just one example. For example, if no comment is input within a predetermined time after the notification unit 505 notifies the information terminal 101a of an SOS notification in step S1602a, the notification unit 505 may notify the information terminals 101c and 101d of an SOS notification in step S1602b.

[0141] In this way, when the administrator is able to respond to an SOS notification, it is possible to suppress the notification of the SOS notification to the other members C and D. Furthermore, when an SOS notification is sent to the information terminals 101c and 101d, the other members C and D can easily understand that since the administrator is unable to respond, they should make a comment themselves.

[0142] <About the information screen> For example, when a business negotiation is conducted using a web conference provided by the conference server 20, the participants who participated in the business negotiation or the administrator who manages the participants may want to check the contents of the business negotiation later. Therefore, the communication support device 10 according to the present embodiment has a function of providing an information provision screen 1700 as shown in FIG. 17 to the information terminal 101 used by the participants or the administrator.

[0143] Fig. 17 is a diagram (1) showing an example of an information provision screen according to an embodiment. In the example of Fig. 17, an information provision screen 1700 provided by the communication support device 10 displays a conference summary 1710 and a volume timeline 1720.

[0144] The conference summary 1710 displays, for example, information such as the names of the participants who attended the conference (business negotiation), the names of the other participants (clients) who attended the conference, the date and time the conference started, and an evaluation. Here, the evaluation is, for example, the result of the person in charge who attended the conference self-evaluating the content of the business negotiation using the number of stars or the like.

[0145] The volume timeline 1720 displays the speech volume 1721a of the participant who participated in the conference and the speech volume 1721b of the other participants who participated in the conference in chronological order, with the horizontal axis representing time. Note that the speech volume 1721a of the participant and the speech volume 1721b of the other participants are examples of indicators indicating the speech status of the participant who participated in the conference and the other participants.

[0146] Preferably, the communication support apparatus 10 calculates a moving average for a predetermined period (eg, several seconds to several tens of minutes) from the data of the speech volume, and sets it as the participant's speech volume 1721a and the other participants' speech volumes 1721b.

[0147] Furthermore, the communication support apparatus 10 displays, on the volume timeline 1720, markers corresponding to the times at which a predetermined operation was accepted, based on time information stored in the storage unit 509 or the like. In the example of FIG. 17, the volume timeline 1720 displays markers 1722a and 1722b indicating that a selection operation of the "Send SOS" button 722 has been accepted. The volume timeline 1720 also displays a marker 1723 indicating that a selection operation of the "Bookmark" button 731 has been accepted. These markers 1711a, 1722b, and marker 1723 allow the administrator, participants, or the like to easily identify the marked locations from the volume timeline 1720.

[0148] Preferably, when the marker 1722a is selected on the information providing screen 1700, the communication supporting device 10 displays, for example, a comment field 1801 as shown in Fig. 18 on the information providing screen 1700. In the example of Fig. 18, the communication supporting device 10 displays the comment field 1801 using a pop-up window 1800. However, the method of displaying the comment field 1801 is not limited to this.

[0149] 18, for example, a comment 1803 is displayed in the comment field 1801. A participant or an administrator can add a comment to the marker 1722a by inputting a comment in the comment field 1801 and selecting an "add" button 1802.

[0150] 18, the names such as "Manager A" and "Member B" may be entered by the administrator or participants themselves, or the communication support device 10 may automatically display the user name of the logged-in user. This comment field 1801 makes it easier for the administrator to communicate with the participants who have joined the communication.

[0151] Furthermore, the communication support device 10 may (or may not) display markers 1724a and 1724b at likely problematic locations based on the speech volume 1721a of the participant and the speech volume 1721b of the other participants displayed in chronological order as shown in Fig. 17. Here, the likely problematic locations include, for example, locations where the speech volume exceeds a predetermined threshold (the speech volume is loud or quiet), etc.

[0152] Furthermore, the communication support device 10 may have a function of playing back a selected part of the speech sound from the speech volume 1721a of the participant or the speech volume 1721b of another participant displayed on the information provision screen 1700 as shown in FIG. 17, for example.

[0153] (Information screen provision process) 19 is a flowchart showing an example of a process for providing an information provision screen according to an embodiment. This process shows an overview of a process in which the communication support device 10 provides an information provision screen 1700 as shown in FIG. 17 to the information terminal 101 or the like.

[0154] In step S1901, when the information providing unit 508 of the communication support apparatus 10 accepts a selection operation of a conference for which an information provision screen is to be created from a list of conferences displayed on the information terminal 101, the information providing unit 508 executes the processes in and after step S1902.

[0155] In step S1902, the information providing unit 508 acquires data on the speech voice and speech volume of a conference selected from the conference list from the storage unit 509, etc. In addition, in step S1903, the information providing unit 508 acquires marking information of the selected conference from the storage unit 509, etc.

[0156] In step S1904, the information providing unit 508 creates an information providing screen that illustrates the acquired speech volumes in chronological order. In step S1905, the information providing unit 508 displays a marker on the created information providing screen based on the acquired marking information.

[0157] Through the processing of steps S1901 to S1905, the information providing unit 508 creates, for example, an information providing screen 1700 as shown in FIG.

[0158] In step S1906, the information providing unit 508 provides the created information providing screen. For example, the information providing unit 508 displays an information providing screen 1700 as shown in Fig. 17 on the information terminal 101 used by the administrator or the terminal device 100 used by the participant.

[0159] 19, the information providing unit 508 can provide, for example, an information providing screen 1700 as shown in FIG. 17 to the administrator or the participants who have joined the conference.

[0160] (Display processing of comments section) Fig. 20 is a flowchart showing an example of a comment field display process according to an embodiment. This process shows an example of a comment field display process in which the communication support device 10 displays a comment field 1801 as shown in Fig. 18 on an information provision screen 1700 as shown in Fig. 17.

[0161] In step S2001, when the information providing unit 508 of the communication support device 10 accepts a marker selection operation, the information providing unit 508 executes the processes in and after step S2002. Note that the administrator or the like can select the marker 1722a, for example, by selecting the marker 1722a on an information providing screen 1700 as shown in Fig. 17. In addition, when the information providing unit 508 accepts, for example, operation information indicating that the selection operation of the marker 1722a has been accepted from the information terminal 101, the information providing unit 508 determines that the selection operation of the marker 1722a has been accepted.

[0162] In step S2002, the information providing unit 508 displays a comment field based on the marking information of the selected marker. For example, the information providing unit 508 displays a comment field 1801 for accepting input corresponding to the selected marker as shown in Fig. 18 on an information providing screen 1700 as shown in Fig. 17. Note that, when a comment is registered in the marking information, the information providing unit 508 displays the registered comment in the comment field 1801.

[0163] In step S2003, the information providing unit 508 judges whether a comment has been entered in the displayed comment field. If a comment has been entered, the information providing unit 508 shifts the process to step S2004. On the other hand, if no comment has been entered, the information providing unit 508 ends the process of FIG. 20.

[0164] In step S2004, the information providing unit 508 stores the comment entered in the comment field in the marking information of the selected marker.

[0165] Through the processing of FIG. 20, the information providing unit 508 stores the comment entered in the comment field in the marking information, and the next time the comment field is displayed, the information providing unit 508 can display the comment stored in the marking information in the comment field.

[0166] As described above, according to each embodiment of the present invention, it is possible to provide a communication support device that makes it easy for participants in a communication to obtain comments from related parties.

[0167] <Supplementary Information> Each function of each embodiment described above can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and a device such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), or a conventional circuit module designed to execute each function described above.

[0168] Moreover, the group of devices described in the examples merely shows one of a plurality of computing environments for implementing the embodiments disclosed herein. In one embodiment, the communication support device 10 includes a plurality of computing devices, such as a server cluster. The plurality of computing devices are configured to communicate with each other via any type of communication link, including a network, a shared memory, and the like, and perform the processes disclosed herein.

[0169] Furthermore, the communication support device 10, the conference server 20, the terminal device 100, and the information terminal 101 can be configured to share the disclosed processing steps, for example, the processes shown in Figures 6, 9, 10, 14, 16, 19, 20, etc., in various combinations. For example, a process executed by the communication support device 10 can be executed by the terminal device 100 or the information terminal 101. Also, a process executed by the terminal device 100 and a process executed by the information terminal 101 can be executed by the communication support device 10. Furthermore, each element of the communication support device 10 may be integrated into one device or may be divided into multiple devices.

[0170] <Additional Notes> This specification discloses a communication support device, a communication support system, a communication support method, and a program as described below. (Section 1) a reception unit that receives operation information indicating that a predetermined operation transmitted from a terminal device used by a participant participating in the communication has been received; a notification unit that, when the operation information is received, sends a notification indicating that the predetermined operation has been received to a notification destination that is associated with the participant in advance; a display control unit that transmits the comment input at the notification destination in response to the notification to a terminal device used by the participant so that the comment is displayed on the terminal device used by the participant; A communication support device having the above configuration. (Section 2) an acquisition unit that acquires speech sounds of the participant and other participants participating in the communication; A conversion unit that converts the spoken voice into text; having The notification includes at least a part of a character string obtained by converting the spoken voice before the predetermined operation is performed into text. 2. A communication support device as described in claim 1. (Section 3) 3. The communication support device according to claim 1, wherein the predetermined operation includes an operation of selecting a display element for requesting advice from the notification destination on an operation screen displayed by the terminal device. (Section 4) 4. The communication support device according to any one of claims 1 to 3, wherein the notification unit sends the notification to an information terminal used by an administrator corresponding to the participant. (Section 5) Group information is stored that includes information on members who belong to the same group as the participant; The notification unit notifies the notification to an information terminal used by the member. 4. A communication support device according to any one of claims 1 to 3. (Section 6) The communication support device described in claim 5, wherein, in response to the notification, when one of the members enters a comment, the notification unit notifies the other members that the one member has entered a comment. (Section 7) an information providing unit that provides a display screen that illustrates, based on the speech voice, indicators showing speech situations of the participant and the other participants in a time series and displays a marker corresponding to a time when the predetermined operation is received; 3. The communication support device according to claim 2, wherein, when the marker is selected on the display screen, the information providing unit displays on the display screen a comment field in which a comment corresponding to the marker can be displayed and input. (Section 8) A communication support system including a terminal device used by a participant participating in a communication, and a communication support device capable of communicating with the terminal device via a communication network, the terminal device has a display unit that displays an operation screen on which a predetermined operation can be performed during the communication; The communication support device includes: a reception unit that receives operation information indicating that the predetermined operation has been received, the operation information being transmitted from the terminal device; a notification unit that, when the operation information is received, sends a notification indicating that the predetermined operation has been received to a notification destination that is associated with the participant in advance; a display control unit that transmits the comment input at the notification destination in response to the notification to the terminal device used by the participant so that the comment is displayed on the display unit of the terminal device; A communication support system having the above configuration. (Section 9) A communication support device A process of receiving operation information indicating that a predetermined operation transmitted from a terminal device used by a participant participating in the communication has been received; a process of notifying a notification destination associated with the participant in advance of a notification indicating that the predetermined operation has been accepted when the operation information is accepted; a process of transmitting the comment input at the notification destination in response to the notification to a terminal device used by the participant in order to display the comment on the terminal device used by the participant; A communication support method that implements the above. (Section 10) On the computer, A process of receiving operation information indicating that a predetermined operation transmitted from a terminal device used by a participant participating in the communication has been received; a process of notifying a notification destination associated with the participant in advance of a notification indicating that the predetermined operation has been accepted when the operation information is accepted; a process of transmitting the comment input at the notification destination in response to the notification to a terminal device used by the participant in order to display the comment on the terminal device used by the participant; A program to execute.

[0171] Although an embodiment of the present invention has been described above, the present invention is not limited to such a specific embodiment, and various modifications and applications are possible within the scope of the gist of the present invention described in the claims. [Explanation of symbols]

[0172] 1. Communication Support System 10 Communication support devices 100, 100a to 100d Terminal device 101, 101c, 101d Information terminal 502 Acquisition Department 503 Conversion Unit 504 Reception 505 Notification Department 506 Display control unit 722 "Send SOS" button (an example of a display element) 1100 Group Information 1221 SOS notification (example of notification) 1700 Information provision screen 1720 Volume Timeline 1722a, 1722b, 1723 markers 1801 Comments [Prior art documents] [Patent documents]

[0173] [Patent Document 1] JP 2004-054495 A

Claims

1. a reception unit that receives operation information indicating that a predetermined operation transmitted from a terminal device used by a participant participating in the communication has been received; a notification unit that, when the operation information is received, sends a notification indicating that the predetermined operation has been received to a notification destination that is associated with the participant in advance; a display control unit that transmits the comment input at the notification destination in response to the notification to a terminal device used by the participant so that the comment is displayed on the terminal device used by the participant; A communication support device having the above configuration.

2. an acquisition unit that acquires speech sounds of the participant and other participants participating in the communication; A conversion unit that converts the spoken voice into text; having The notification includes at least a part of a character string obtained by converting the spoken voice before the predetermined operation is performed into text. The communication support device according to claim 1 .

3. The communication support device according to claim 1 , wherein the predetermined operation includes an operation of selecting a display element for requesting advice from the notification destination on an operation screen displayed by the terminal device.

4. The communication support device according to claim 1 , wherein the notification unit transmits the notification to an information terminal used by an administrator corresponding to the participant.

5. Group information is stored that includes information on members who belong to the same group as the participant; The notification unit notifies the notification to an information terminal used by the member. The communication support device according to any one of claims 1 to 3.

6. The communication support device according to claim 5 , wherein, in response to the notification, when one of the members inputs a comment, the notification unit notifies the other members that the one member has input a comment.

7. an information providing unit that provides a display screen that illustrates, based on the speech voice, indicators showing speech situations of the participant and the other participants in a time series and displays a marker corresponding to a time when the predetermined operation is received; The communication support device according to claim 2 , wherein, when the marker is selected on the display screen, the information providing unit displays, on the display screen, a comment field in which a comment corresponding to the marker can be displayed and entered.

8. A communication support system including a terminal device used by a participant participating in a communication, and a communication support device capable of communicating with the terminal device via a communication network, the terminal device has a display unit that displays an operation screen on which a predetermined operation can be performed during the communication; The communication support device includes: a reception unit that receives operation information indicating that the predetermined operation has been received, the operation information being transmitted from the terminal device; a notification unit that, when the operation information is received, sends a notification indicating that the predetermined operation has been received to a notification destination that is associated with the participant in advance; a display control unit that transmits the comment input at the notification destination in response to the notification to the terminal device used by the participant so that the comment is displayed on the display unit of the terminal device; A communication support system having the above configuration.

9. A communication support device A process of receiving operation information indicating that a predetermined operation transmitted from a terminal device used by a participant participating in the communication has been received; a process of notifying a notification destination associated with the participant in advance of a notification indicating that the predetermined operation has been accepted when the operation information is accepted; a process of transmitting the comment input at the notification destination in response to the notification to a terminal device used by the participant in order to display the comment on the terminal device used by the participant; A communication support method that implements the above.

10. On the computer, A process of receiving operation information indicating that a predetermined operation transmitted from a terminal device used by a participant participating in the communication has been received; a process of notifying a notification destination associated with the participant in advance of a notification indicating that the predetermined operation has been accepted when the operation information is accepted; a process of transmitting the comment input at the notification destination in response to the notification to a terminal device used by the participant in order to display the comment on the terminal device used by the participant; A program to execute.

Citation Information

Patent Citations

  • Business activity support system, program, and recording medium

    JP2004054495A