Information processing apparatus and information processing method

Abstract

To prepare concerning an utterance of an attendant before the attendant makes the utterance.SOLUTION: An information processing apparatus comprises: an operation receiving unit which receives an utterance request operation from an attendant when the attendant of a conference utters concerning the content of the conference; an utterance summary acquisition unit for acquiring an utterance summary of the attendant having made the utterance request operation; and a display control unit which associates the acquired utterance summary with the attendant having made the utterance request operation and displays it on a display screen for displaying information on the conference.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and an information processing method.

Background Art

[0002] When a presenter is explaining presentation materials using a PC (personal computer) during a meeting, there is a known technology that allows meeting participants to express their opinions using their own PCs (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When conducting a Web conference, participants usually keep their microphones muted except when speaking. They turn on the microphone to speak and then mute it again after finishing. At this time, the presenter may not be able to appropriately respond to sharp or unexpected questions from the participants. Therefore, it is desirable to let the presenter know in advance the information about what the participant is about to say while the participant's microphone is muted before speaking, so that the presenter can prepare for the response to the speech.

[0005] An object of the present invention is to provide an information processing apparatus and an information processing method that can prepare for a speech before a participant speaks.

Means for Solving the Problems

[0006] When a participant in a meeting makes a statement regarding the content of the meeting, the information processing apparatus of the present invention includes an operation reception unit that receives a statement request operation from the participant, a statement summary acquisition unit that acquires a statement summary of the participant who has performed the statement request operation, and a display control unit that displays the acquired statement summary on a display screen that displays information regarding the meeting, in association with the participant who has performed the statement request operation.

[0007] The information processing method of the present invention includes a step of receiving a statement request operation from a participant when the participant makes a statement regarding the content of the meeting, a step of acquiring a statement summary of the participant who has performed the statement request operation, and a step of displaying the acquired statement summary on a display screen that displays information regarding the meeting, in association with the participant who has performed the statement request operation.

Advantages of the Invention

[0008] According to the present invention, it is possible to make a participant prepare for a statement before making the statement.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

MODE FOR CARRYING OUT THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited by this embodiment, and in the following embodiments, the same parts are denoted by the same reference numerals to omit redundant explanations.

[0011] [First Embodiment] (Information Display System) The information display system according to the first embodiment will be described with reference to FIG. 1. FIG. 1 is a diagram showing a configuration example of the information display system according to the first embodiment.

[0012] The conference system 1 includes a plurality of terminal devices 10 and an information processing device 100. The terminal devices 10 and the information processing device 100 are communicably connected via a wired or wireless network N. The number of terminal devices 10 and information processing devices 100 included in the conference system 1 may be arbitrary. The conference system 1 is an online conference system in which a plurality of users can each conduct an online conference using the terminal device 10. The conference system 1 is a system that can prepare for a speech by providing a speech outline indicating the content of the speech to the speaker who is making a presentation in an online conference before the participants speak.

[0013] (Terminal device) Using FIG. 2, a configuration example of the terminal device according to the first embodiment will be described. FIG. 2 is a block diagram showing a configuration example of the terminal device according to the first embodiment.

[0014] The terminal device 10 is a device used by a user to participate in an online conference. The terminal device 10 is, for example, a notebook PC (Personal Computer), a desktop PC, a tablet terminal, a smartphone, an HMD (Head Mounted Display), an HUD (Head Up Display), etc., but is not limited thereto.

[0015] As shown in FIG. 2, the terminal device 10 includes a camera 12, an operation unit 14, an audio input unit 16, a display unit 18, an audio output unit 20, a communication unit 22, a storage unit 24, and a control unit 26.

[0016] The camera 12 is a camera that photographs the user using the terminal device 10. The camera 12 photographs, for example, the face, body, expression, and movement of the user using the terminal device 10.

[0017] The operation unit 14 receives various operations on the terminal device 10. The operation unit 14 is realized by various input devices such as a keyboard, a mouse, a switch, a button, and a touch panel.

[0018] The voice input unit 16 detects the voice of the user using the terminal device 10. The voice input unit 16 converts the detected voice into a voice signal. The voice input unit 16 is realized by a microphone.

[0019] The display unit 18 has a display surface for displaying various kinds of information. The display unit 18 is realized by a display including, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 18 displays, for example, a display screen showing information regarding an online meeting. The display unit 18 displays, for example, a list of participants in the online meeting and the shared materials. When the operation unit 14 is a touch panel, the operation unit 14 and the display unit 18 are integrally configured. Here, in the case where the terminal device 10 is a device that displays by projecting or projecting image information such as a HUD, the display unit 18 does not necessarily need to have a display surface.

[0020] The voice output unit 20 is a speaker that outputs various kinds of voices. The voice output unit 20 outputs, for example, the voice of the user himself / herself and the voices of the users of other terminal devices 10 participating in the online meeting.

[0021] The communication unit 22 is a communication interface that performs communication between the terminal device 10 and an external device. The communication unit 22 executes communication between the terminal device 10 and the information processing device 100, for example. The communication unit 22 is realized by, for example, a wireless LAN (Local Area Network), a wired LAN, Wi-Fi (registered trademark), or the like.

[0022] The storage unit 24 stores various kinds of information. The storage unit 24 stores the calculation content of the control unit 26 and information such as programs. The storage unit 24 includes at least one of, for example, a main storage device such as a RAM (Random Access Memory) and a ROM (Read Only Memory), and an external storage device such as an HDD (Hard Disk Drive).

[0023] The control unit 26 controls each part of the terminal device 10. The control unit 26 has, for example, an information processing device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), and a storage device such as a RAM or a ROM. The control unit 26 executes a program for controlling the operation of the terminal device 10 according to the present invention. The control unit 26 may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 26 may be realized by a combination of hardware and software.

[0024] The control unit 26 acquires, for example, various input information input by the user to the operation unit 14. The control unit 26 acquires, for example, a speech outline indicating the content of the opinions of the participants on the speech of the speaker. The control unit 26 transmits the acquired speech outline to the information processing device 100.

[0025] (Information processing device) With reference to FIG. 3, a configuration example of the information processing device according to the first embodiment will be described. FIG. 3 is a diagram showing a configuration example of the information processing device according to the first embodiment.

[0026] The information processing device 100 acquires and displays a speech outline for the participants in the online meeting. The information processing device 100 is, for example, a server device. The information processing device 100 may be realized by, for example, the terminal device 10 of the host of the online meeting or the speaker who is speaking in the online meeting in the online meeting. The information processing device 100 may be realized by both a server device and the terminal device 10.

[0027] As shown in FIG. 3, the information processing device 100 includes a communication unit 102, a storage unit 104, and a control unit 106.

[0028] The communication unit 102 is a communication interface that performs communication between the information processing device 100 and an external device. The communication unit 102 executes communication between, for example, the information processing device 100 and the terminal device 10. The communication unit 102 is realized by, for example, a wireless LAN or a wired LAN.

[0029] The storage unit 104 stores various kinds of information. The storage unit 104 stores the calculation content of the control unit 106 and information such as programs. The storage unit 104 includes, for example, at least one of a main storage device such as a RAM and a ROM, and an external storage device such as an HDD.

[0030] The control unit 106 controls each part of the information processing device 100. The control unit 106 has, for example, an information processing device such as a CPU or an MPU, and a storage device such as a RAM or a ROM. The control unit 106 executes a program that controls the operation of the information processing device 100 according to the present invention. The control unit 106 may be realized by an integrated circuit such as an ASIC or an FPGA. The control unit 106 may be realized by a combination of hardware and software.

[0031] The control unit 106 includes an operation reception unit 110, a speech outline acquisition unit 112, a display control unit 114, and a speech permission unit 116 as functional blocks realized by the processing by the control unit 106. Each of these functional blocks may be realized by the control unit 26 of the terminal device 10, or may be realized by the control unit 106 and the control unit 26 of the terminal device 10.

[0032] The operation reception unit 110 receives various operations input to the terminal device 10. The operation reception unit 110 receives a speech request operation of a participant in a meeting. The operation reception unit 110 receives, for example, a speech request operation from a participant before the participant speaks when the participant speaks about the content of a meeting. The speech request operation includes an operation of inputting a speech outline indicating an opinion on the speech content of the presenter.

[0033] The speech summary acquisition unit 112 acquires a speech summary indicating the opinions of the participants in the meeting who have performed the speech request operation regarding the content of the meeting. For example, the speech summary acquisition unit 112 acquires a speech summary indicating the opinions regarding the presentation content of the presenter. The speech summary acquisition unit 112 acquires, for example, the speech summary of the participant who has performed the speech request operation.

[0034] The display control unit 114 causes various types of information to be displayed on the display unit 18 of the terminal device 10. The display control unit 114 causes a display screen indicating information regarding the online meeting to be displayed on the display unit 18. The display control unit 114 causes the speech summary to be displayed on the display screen indicating information regarding the online meeting in association with the participant who has performed the speech request operation.

[0035] The speech permission unit 116 switches between permitting and not permitting the speech of the participants in the meeting. For example, the speech permission unit 116 permits the speech of the participant who has acquired the speech summary. For example, the speech permission unit 116 turns on the voice input unit 16 of the terminal device 10 used by the participant who has acquired the speech summary.

[0036] [Summary of Invention Selection Process] The summary of invention selection process according to the first embodiment will be described with reference to FIG. 4. FIG. 4 is a flowchart showing the summary of invention selection process according to the first embodiment. FIG. 4 is a process executed by the terminal device 10.

[0037] The control unit 26 causes the display unit 18 to display a display screen showing information regarding the online meeting (step S10). Specifically, the control unit 26 causes the display unit 18 to display a display screen showing information regarding the online meeting in accordance with the control of the display control unit 114. FIG. 5 is a diagram showing an example of the display screen according to the first embodiment. As shown in FIG. 5, the display screen 30 includes, for example, a camera operation button 32, a microphone operation button 34, and an exit button 36. The camera operation button 32 is a button for switching between turning on and off the operation of the camera 12 during the online meeting. The microphone operation button 34 is a button for switching between turning on (during operation) and off (during stop) the voice input unit 16 during the online meeting. The exit button 36 is a button for exiting the online meeting. In the example shown in FIG. 5, the camera 12 and the voice input unit 16 are off. In an online meeting, participants often turn off the camera 12 and the voice input unit 16 except when they are speaking. FIG. 6 is a diagram for explaining the operation when a participant according to the comparative example speaks in an online meeting. As shown in FIG. 6, when speaking, the participant uses the operation unit 14 to click the microphone operation button 34 with the pointer 40 to turn it on and then speaks. Proceed to step S12.

[0038] The control unit 26 determines whether a speech request operation has been input (step S12). FIG. 7 is a diagram for explaining a method of inputting a speech request operation according to the first embodiment. As shown in FIG. 7, in the first embodiment, when a participant uses the operation unit 14 to move the pointer 40 onto the microphone operation button 34, a speech outline selection window 50 is displayed. Here, "moving the pointer onto" means an operation of moving the pointer 40 using the operation unit 14 and superimposing it on the microphone operation button 34. Here, it may be determined that the pointer 40 is on the microphone operation button 34 only when the superimposed state continues for a certain period (for example, 100 msec) or more. Also, the condition for displaying the speech outline selection window 50 may not be moving the pointer onto, but for example, clicking on the microphone operation button 34. That is, in the first embodiment, an operation for turning on the voice input unit 16 is an example of a speech request operation. In the speech outline selection window 50, speech outlines such as "Neutral speech", "Agreeing speech", "Opposing speech", "Question (Neutral)", "Question (Positive)", and "Question (Negative)" are displayed. The speech outline represents an outline of the participant's opinion regarding the content of the meeting. The participant can input a speech request operation by operating the operation unit 14 to move the pointer 40 onto and click on the speech outline representing his or her opinion regarding the content of the meeting. If it is determined that a speech request operation has been input (step S12; Yes), the process proceeds to step S14. If it is determined that a speech request operation has not been input (step S12; No), the process proceeds to step S16.

[0039] When it is determined Yes in step S12, the control unit 26 transmits the speech outline selected by the participant to the information processing apparatus 100 via the communication unit 22 (step S14). Then, the process proceeds to step S16.

[0040] The control unit 26 determines whether to end the process (step S16). The control unit 26 determines to end the process, for example, when the online meeting ends. If it is determined to end the process (step S16; Yes), the process of FIG. 4 ends. If it is determined not to end the process (step S16; No), the process proceeds to step S10.

[0041] In the first embodiment, the operation for requesting a participant to speak is not limited to the operation of turning on the voice input unit 16. FIG. 8 is a diagram for explaining a modification example of the operation for requesting a participant to speak according to the first embodiment. As shown in FIG. 8, the operation for requesting a participant to speak may be an operation of moving the mouse cursor over a process execution button 38 that executes a predetermined process. Examples of the process execution button 38 include, but are not limited to, a raise hand button, a chat start button, and a speech outline input button.

[0042] (Speech Outline Display Process) The speech outline display process according to the first embodiment will be described with reference to FIG. 9. FIG. 9 is a flowchart showing the flow of the speech outline display process according to the first embodiment. FIG. 9 is a process executed by the information processing apparatus 100.

[0043] The display control unit 114 causes the display unit 18 of the terminal device 10 used by the participants in the meeting to display a participant list (step S20). Specifically, the control unit 26 of the terminal device 10 causes the display unit 18 to display a participant list of the participants participating in the meeting according to the control of the display control unit 114. FIG. 10 is a diagram showing an example of the participant list according to the first embodiment. As shown in FIG. 10, the participant list 60 includes, for example, a user icon 62, a user name 64, and a microphone icon 66 for each participant in the meeting.

[0044] The user icon 62 represents, for example, an icon set by the participant or an icon set by the conference system 1. The user name 64 represents the name of the participant. The microphone icon 66 indicates whether the speech of the corresponding participant is permitted. In the example shown in FIG. 10, users A, B, C, and D indicate that their speeches are not permitted.

[0045] The speech summary acquisition unit 112 determines whether or not it has acquired a speech summary from the terminal device 10 used by the participants in the meeting (step S22). If it is determined that the speech summary has been acquired (step S22; Yes), the process proceeds to step S24. If it is determined that the speech summary has not been acquired (step S22; No), the process proceeds to step S28.

[0046] When it is determined Yes in step S22, the display control unit 114 displays the speech summary (step S24). Specifically, the control unit 26 of each terminal device 10 causes the speech summary to be displayed in the participant list 60 displayed on the display unit 18 according to the control of the display control unit 114. FIG. 11 is a diagram for explaining a method of displaying the speech summary according to the first embodiment. As shown in FIG. 11, the participant list 60 includes a user icon 62, a user name 64, a microphone icon 66, and a speech summary 68. For example, in step S22, it is assumed that the speech summary acquisition unit 112 has acquired a speech summary indicating that user A will make a supportive speech from the terminal device 10 used by user A. In this case, the display control unit 114 causes the speech summary 68 indicating that a supportive speech will be made, such as "I will make a supportive speech.", to be displayed in the participant list 60 of each terminal device 10 in association with user A. Thereby, the presenter can know that user A will make a supportive speech regarding the presentation content. Then, the process proceeds to step S26.

[0047] The speech permission unit 116 permits the speech of the participants in the meeting (step S26). Then, the process proceeds to step S28. FIG. 12 is a diagram for explaining a method of permitting the speech of the participants in the meeting according to the first embodiment. In the example shown in FIG. 12, the speech permission unit 116 turns on the voice input unit 16 of the terminal device 10 used by user A. In this case, the display control unit 114 switches the state of the microphone icon 66 of user A to speech permission.

[0048] The speech permission unit 116 may set a predetermined wait time from when the speech summary 68 is displayed on the participant list 60 until the voice input unit 16 is turned on. The wait time is, for example, a fixed time of about 1 to 2 seconds, but is not limited to this. The speech permission unit 116 may change the wait time according to the type and content of the speech summary. For example, the speech permission unit 116 may set the wait time relatively short for an opinion in favor or a question, and set the wait time relatively long for an opinion in opposition or a question. The speech permission unit 116 may change the wait time according to the attributes of the participant who performed the speech request operation. For example, the speech permission unit 116 may set the wait time relatively long when the participant is the superior of the presenter.

[0049] The speech permission unit 116 may change the wait time depending on, for example, the contents of the conference. The speech permission unit 116 may change the wait time depending on, for example, the number of participants in the conference or the degree of excitement in the conference. Here, in order to determine the degree of excitement in the conference, a conference status determination unit 124 described later may be provided, and the degree of excitement may be obtained from the conference status determination unit 124. The speech permission unit 116 may set the wait time relatively short when the number of participants in the conference is large, and set the wait time relatively long when the number of participants in the conference is small. The speech permission unit 116 may set the wait time relatively short when the conference is more exciting than a predetermined level, and set the wait time relatively long when the conference is not exciting. By providing a wait time or appropriately changing the wait time, the presenter can properly prepare himself mentally before a question comes.

[0050] The control unit 106 determines whether or not to end the process (step S28). For example, when the online conference ends, the control unit 106 determines to end the process. When it is determined to end the process (step S28; Yes), the control unit 106 ends the process in FIG. 9. When it is not determined to end the process (step S28; No), the control unit 106 proceeds to step S22.

[0051] Incidentally, the speech summary displayed in step S24 may be erased, for example, after a certain period of time has elapsed. The certain period of time is, for example, 5 seconds, but is not limited to this. Also, the speech summary may be displayed until the speech of the participant associated with the speech summary ends. For example, in FIG. 11, when the speech of user A ends and the microphone icon 66 of user A is in a state where speech permission is not granted, the display of the speech summary is erased.

[0052] As described above, in the first embodiment, when a participant makes a speech, a speech summary indicating the content of the speech is displayed. Thereby, in the first embodiment, the presenter can prepare mentally by looking at the speech summary and knowing the outline of the speech of the participant who is about to speak, so that the presenter can appropriately respond to the speech of the participant and the progress of the meeting can be made smooth.

[0053] Also, in the first embodiment, when a participant is about to make a speech, not only is the mute of the microphone simply released, but also a speech summary indicating the content to be spoken and information indicating that the mute of the microphone has been released can be explicitly conveyed to each participant. Thereby, in the first embodiment, opinions can be smoothly conveyed in the meeting, so that communication can be taken more smoothly.

[0054] [Modification Example of the First Embodiment] In the first embodiment, the participant was described as selecting a predetermined speech summary, but the present disclosure is not limited to this. The speech summary may be manually inputtable by the participant who is about to speak. In this case, for example, in the example shown in FIG. 7, when the participant uses the operation unit 14 to move the pointer 40 onto the microphone operation button 34, an input field for inputting the speech summary may be displayed. The participant may input the speech summary into the displayed input field using the operation unit 14. In this case, the number of characters that can be input into the input field may be limited so that only the outline of the speech can be input.

[0055] In the first embodiment, the example of displaying the speech outline of one participant was used for explanation, but the present disclosure is not limited to this. FIG. 13 is a diagram for explaining a method of displaying a speech outline according to a modification of the first embodiment. As shown in FIG. 13, for example, the speech outline acquisition unit 112 acquires a speech outline indicating that user A makes an approval speech from the terminal device 10 used by user A, acquires a speech outline indicating that user B makes an approval speech from the terminal device 10 used by user B, and acquires a speech outline indicating that user C makes an approval speech from the terminal device 10 used by user C. In this case, the display control unit 114 may display each speech outline 68 such as "Make an approval speech." or "Make an opposing speech." in the participant list 60 in association with each participant.

[0056] In the first embodiment, it has been described that after the speech outline is displayed, the speech of the participant who has performed the speech request operation is automatically permitted, but the present disclosure is not limited to this. For example, when there is a facilitator or the like who manages the progress of the meeting in the meeting, the facilitator or the like may permit the speech of the participant. In this case, for example, the facilitator or the like may view the participant list 60 shown in FIG. 13 and permit the speech of only a predetermined participant among user A, user B, and user C. In this case, it is preferable to unmute the participant by moving the pointer to and clicking on the microphone icon 66 associated with the participant whose speech is permitted by the facilitator.

[0057] [Second Embodiment] A configuration example of the information processing apparatus according to the second embodiment will be described with reference to FIG. 14. FIG. 14 is a block diagram showing a configuration example of the information processing apparatus according to the second embodiment.

[0058] As shown in FIG. 14, the information processing apparatus 100A is different from the information processing apparatus 100 shown in FIG. 3 in that the control unit 106A includes a participant information acquisition unit 118, an emotion estimation unit 120, and a speech outline generation unit 122.

[0059] The participant information acquisition unit 118 acquires participant information regarding the participants attending the meeting. The participant information acquisition unit 118 acquires the participant information from the terminal device 10 of each participant via, for example, the communication unit 102. The participant information acquisition unit 118 acquires, for example, an image of the participant (e.g., an image of the participant's face) captured by the camera 12 of the terminal device 10. The participant information may include biometric information such as the pulse or heartbeat of the participant acquired from a sensor other than the camera 12.

[0060] The emotion estimation unit 120 estimates the emotion of each participant attending the meeting. The emotion estimation unit 120 estimates the emotion of the participant based on the participant information acquired by the participant information acquisition unit 118.

[0061] The speech summary generation unit 122 generates a speech summary of each participant attending the meeting. The speech summary generation unit 122 generates the speech summary of the participant based on the emotion of the participant estimated by the emotion estimation unit 120.

[0062] (Speech summary display process) The speech summary display process according to the second embodiment will be described with reference to FIG. 15. FIG. 15 is a flowchart showing the flow of the speech summary display process according to the second embodiment. FIG. 15 is a process executed by the information processing apparatus 100A.

[0063] The process of step S40 is the same as the process of step S20 shown in FIG. 9, so the description is omitted.

[0064] The operation reception unit 110 determines whether the terminal device 10 of the participant attending the meeting has received a predetermined operation (step S42). Specifically, for example, when the operation reception unit 110 acquires operation information indicating that the point has been moused over the microphone operation button 34 to turn on the voice input unit 16, the operation reception unit 110 determines that the terminal device 10 has received a predetermined operation. The predetermined operation is not limited to the operation of mousing over the microphone operation button 34. Then, the process proceeds to step S44.

[0065] The participant information acquisition unit 118 acquires the participant information of the participant who has performed a predetermined operation from the terminal device 10 of the participant (step S44). Specifically, the participant information acquisition unit 118 acquires an image of the participant's face as the participant information. Then, it proceeds to step S46.

[0066] The emotion estimation unit 120 estimates the emotion of the participant based on the participant information acquired by the participant information acquisition unit 118 (step S46). Specifically, the emotion estimation unit 120 estimates, for example, the emotion towards the content of the meeting based on the image of the participant's face acquired by the participant information acquisition unit 118. Here, for the estimation of emotion, known techniques such as an emotion estimation model generated in advance by machine learning processing can be used. For example, when the participant has a smiling face, the emotion estimation unit 120 estimates that the participant has a positive emotion towards the content of the meeting. For example, when the participant has an angry expression such as frowning, the emotion estimation unit 120 estimates that the participant has a negative emotion towards the content of the meeting. Then, it proceeds to step S48.

[0067] The speech summary generation unit 122 generates a summary of the participant's speech (step S48). Specifically, for example, when it is determined by the emotion estimation unit 120 that the participant has a positive emotion towards the content of the meeting, the speech summary generation unit 122 generates a speech summary indicating that the participant makes a speech of approval such as "I will make a speech of approval." For example, when it is determined by the emotion estimation unit 120 that the participant has a negative emotion towards the content of the meeting, the speech summary generation unit 122 generates a speech summary indicating that the participant makes a speech of approval such as "I will make a speech of opposition." Then, it proceeds to step S50.

[0068] The processes from step S50 to step S54 are the same as the processes from step S24 to step S28 shown in FIG. 9, respectively, so the description is omitted.

[0069] As described above, in the second embodiment, when a participant makes a statement regarding the content of a meeting, a statement summary indicating the content of the statement is automatically generated and the generated statement summary is displayed. As a result, in the second embodiment, the presenter can be mentally prepared by looking at the statement summary and knowing the outline of the statement, so that the presenter can appropriately respond to the statements of the participants. Also, by generating the statement summary without the need for the participants to input, the progress of the meeting can be smoothed.

[0070] [Modification Example of the Second Embodiment] A modification example of the second embodiment will be described. In the modification example of the second embodiment, the statement summary may be generated at predetermined intervals. In this case, the emotion estimation unit 120 estimates the emotion of the participant at predetermined intervals. Then, the statement summary generation unit 122 may update the statement summary according to the change in the emotion of the participant estimated by the emotion estimation unit 120. Here, the update of the statement summary means that the statement summary already displayed by the display control unit 114 is changed to a new statement summary and displayed.

[0071] Also, the statement summary generation unit 122 may update the statement summary according to the statement content of the participant. For example, the statement summary generation unit 122 may execute speech recognition processing on the statement of the participant to identify the statement content and may identify the statement content.

[0072] [Third Embodiment] With reference to FIG. 16, a configuration example of the information processing apparatus according to the third embodiment will be described. FIG. 16 is a block diagram showing a configuration example of the information processing apparatus according to the third embodiment.

[0073] As shown in FIG. 16, the information processing apparatus 100B is different from the information processing apparatus 100A shown in FIG. 14 in that the control unit 106B includes a meeting situation determination unit 124.

[0074] The meeting situation determination unit 124 determines the situation of the meeting based on the states of each participant in the meeting. For example, the meeting situation determination unit 124 determines the number of times each participant speaks. A threshold value is set in advance for the number of times of speaking. When the number of times of speaking is equal to or greater than the threshold value, the meeting situation determination unit 124 determines that the meeting is lively. When the number of times of speaking is less than the threshold value, the meeting situation determination unit 124 determines that the meeting is not lively. In addition, the meeting situation determination unit 124 may determine the liveliness of the meeting using other techniques such as a liveliness determination model that is learned to output liveliness by inputting conversations.

[0075] (Speech summary display process) The speech summary display process according to the third embodiment will be described with reference to FIG. 17. FIG. 17 is a flowchart showing the flow of the speech summary display process according to the third embodiment.

[0076] The processes from step S60 to step S66 are the same as the processes from step S40 to step S46 shown in FIG. 15, respectively, and thus the description thereof is omitted.

[0077] The meeting situation determination unit 124 determines whether the meeting satisfies a predetermined condition (step S68). Specifically, when the meeting is lively, the meeting situation determination unit 124 determines that the meeting satisfies the predetermined condition. When it is determined that the meeting satisfies the predetermined condition (step S68; Yes), the process proceeds to step S70. When it is determined that the meeting does not satisfy the predetermined condition (step S68; No), the process proceeds to step S74.

[0078] When it is determined as Yes in step S68, in order to automatically generate a speech summary, the display control unit 114 makes the speech summary selection window invisible (step S70). Specifically, in the example shown in FIG. 7, the display control unit 114 does not display the speech summary selection window 50. Thereby, when the meeting is lively, it is possible to prevent the progress of the meeting from being hindered by manually selecting the speech summary. Then, the process proceeds to step S72.

[0079] The process of step S72 is the same as the process of step S46 shown in FIG. 15, so the description thereof is omitted. The processes from step S74 to step S80 are the same as the processes from step S22 to step S28 shown in FIG. 9 respectively, so the descriptions thereof are omitted.

[0080] As described above, the third embodiment changes the method of obtaining the speech outline according to the meeting situation. Thereby, in the third embodiment, when the meeting is lively, etc., by manually inputting the speech outline, it is possible to prevent the progress of the meeting from being hindered.

[0081] [Other Embodiments] Other embodiments will be described. When a participant finishes speaking and mutes the voice input unit 16, other information may be inputtable. That is, when the participant uses the operation unit 14 to click the microphone operation button 34 or the like to turn the microphone operation button 34 from on to off, the participant may be able to input or select greetings such as "Thank you." or "Excuse me." In this case, greetings such as "Thank you." or "Excuse me." are displayed on the terminal device 10 of each participant. Thereby, communication in the meeting can be made smooth.

[0082] Each component of each device shown in the drawings is conceptually functional and does not necessarily have to be physically configured as shown in the drawings. That is, the specific form of the distribution and integration of each device is not limited to that shown in the drawings, and all or part of it can be functionally or physically distributed and integrated in any unit according to various loads, usage situations, etc. Note that the configuration by this distribution and integration may be performed dynamically.

[0083] The embodiments of the present invention have been described above, but the present invention is not limited by the contents of these embodiments. Further, the above-described components include those that can be easily assumed by those skilled in the art, those that are substantially the same, and those within the so-called equivalent range. Furthermore, the above-described components can be combined as appropriate. Still further, various omissions, substitutions, or changes of the components can be made without departing from the gist of the above-described embodiments.

Explanation of Reference Numerals

[0084] 1 Conference system 10 Terminal device 12 Camera 14 Operation unit 16 Voice input unit 18 Display unit 20 Voice output unit 22, 102 Communication unit 24, 104 Storage unit 26, 106, 106A, 106B Control unit 100, 100A, 100B Information processing device 110 Operation reception unit 112 Speech outline acquisition unit 114 Display control unit 116 Speech permission unit 118 Participant information acquisition unit 120 Emotion estimation unit 122 Speech outline generation unit 124 Conference situation determination unit

Claims

1. When a participant in the meeting makes a statement regarding the content of the meeting, an operation reception unit that receives a statement request operation from the participant, A statement summary acquisition unit that acquires a statement summary of the participant who has performed the statement request operation, A display control unit that associates the acquired statement summary with the participant who has performed the statement request operation and displays it on a display screen that displays information regarding the meeting, An information processing apparatus comprising:

2. The display control unit provides a wait time from the time when the statement summary is acquired until the participant is permitted to speak. The information processing apparatus according to Claim 1.

3. The display control unit changes the wait time based on the statement summary or the content of the meeting. The information processing apparatus according to Claim 2.

4. The statement summary acquisition unit changes the method of acquiring the statement summary according to the situation of the meeting. The information processing apparatus according to Claim 1.

5. When a participant in the meeting makes a statement regarding the content of the meeting, the steps of receiving a statement request operation from the participant, The step of acquiring a statement summary of the participant who has performed the statement request operation, The step of associating the acquired statement summary with the participant who has performed the statement request operation and displaying it on a display screen that displays information regarding the meeting, An information processing method comprising:

Images