Information processing device and information processing method

Abstract

To make an on-line conference progress properly.SOLUTION: An information processing device comprises: a reaction detection unit which detects reaction information indicative of a reaction of a viewer to a presentation content of a presenter in an on-line conference; a reaction place detection unit which detects reaction place information indicative of a place where the reaction information is detected; a reaction content generation unit which on the basis of the reaction information or on the basis of the reaction information and the reaction place information, generates reaction content information indicative of a reaction content to the presentation content; and a reaction content selection unit which presents the reaction content information to the viewer or a viewer other than the viewer to select.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an information processing device and an information processing method. [Background technology]

[0002] In recent years, online conferences have become widespread. For example, Patent Document 1 discloses a technique for recording a response of one user to a comment made by another user in a video conference so that the response can be played back in a synchronized manner. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2010-219733 A Summary of the Invention [Problem to be solved by the invention]

[0004] In online conferences, viewers may request an opportunity to speak in response to a speaker's remarks or convey their opinions by typing them as text in a chat, etc. In such cases, the time and effort required may affect the progress of the conference.

[0005] An object of the present invention is to provide an information processing device and an information processing method that can appropriately proceed with an online conference. [Means for solving the problem]

[0006] The information processing device of the present invention comprises a reaction detection unit that detects reaction information indicating a viewer's reaction to a presentation content made by a speaker in an online conference, a reaction location detection unit that detects reaction location information indicating the location where the reaction information was detected, a reaction content generation unit that generates reaction content information indicating a reaction content to the presentation content based on the reaction information or based on the reaction information and the reaction location information, and a reaction content selection unit that presents the reaction content information to the viewer or other viewers other than the viewer and allows them to select from the reaction content information.

[0007] The information processing method of the present invention includes the steps of detecting reaction information indicating the audience's reaction to the content of a presentation made by a speaker in an online conference, detecting reaction location information indicating the location where the reaction information was detected, and generating reaction content information indicating the reaction content to the presentation content based on the reaction information or based on the reaction information and the reaction location information. Effect of the Invention

[0008] According to the present invention, an online conference can be appropriately conducted. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an information processing system according to the first embodiment. [Diagram 2] FIG. 2 is a block diagram showing an example of the configuration of a terminal device according to the first embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of the configuration of the information processing device according to the first embodiment. [Figure 4] FIG. 4 is a flowchart showing the flow of the reaction content presentation process according to the first embodiment. [Diagram 5] FIG. 5 is a diagram showing reaction content information according to the first embodiment. [Figure 6] FIG. 6 is a diagram showing reaction content information presented to the viewer or other viewers according to the first embodiment. [Figure 7]FIG. 7 is a diagram showing an example of reaction content information presented to a speaker according to the first embodiment. [Figure 8] FIG. 8 is a diagram showing an example of reaction content presented to a speaker according to the first embodiment. [Figure 9] FIG. 9 is a flowchart showing the flow of the reaction content presentation process according to the first modified example of the first embodiment. [Figure 10] FIG. 10 is a diagram showing reaction content information presented to the viewer or other viewers according to the second modified example of the first embodiment. [Figure 11] FIG. 11 is a flowchart showing the flow of the reaction content presentation process according to the second embodiment. [Figure 12] FIG. 12 is a block diagram illustrating an example of the configuration of an information processing device according to the third embodiment. [Figure 13] FIG. 13 is a flowchart showing the reaction content presentation process according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to this embodiment, and in the following embodiments, the same components are designated by the same reference numerals, and duplicated explanations will be omitted.

[0011] [First embodiment] (Information Processing System) An example of the configuration of an information processing system according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of an information processing system according to the first embodiment.

[0012] The information processing system 1 includes a plurality of terminal devices 10 and an information processing device 100. The terminal devices 10 and the information processing devices 100 are communicatively connected via a wired or wireless network N. The number of terminal devices 10 and information processing devices 100 included in the information processing system 1 may be any number. The information processing system 1 is an online conference system in which a plurality of users can hold an online conference using each of the terminal devices 10. The information processing system 1 detects the reaction of viewers in the online conference and presents it to the speaker. Here, both the viewers and the speakers can be users who use the terminal devices 10.

[0013] (Terminal Device) An example of the configuration of a terminal device according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the configuration of a terminal device according to the first embodiment.

[0014] The terminal device 10 is a device used by a user to participate in an online conference. The terminal device 10 is, for example, a notebook PC (Personal Computer), a desktop PC, a tablet terminal, a smartphone, a HMD (Head Mounted Display), a HUD (Head Up Display), etc., but is not limited thereto. In the following description, the terminal device 10 is described as being a notebook PC.

[0015] As shown in FIG. 2, the terminal device 10 includes a sensor 12, an input unit 14, an audio input unit 16, a display unit 18, an audio output unit 20, a communication unit 22, a storage unit 24, and a control unit .

[0016] The sensor 12 detects sensor information indicating various states of the user using the terminal device 10. The sensor 12 is provided, for example, on the same surface as the surface on which the display unit 18 is provided. The sensor 12 is, for example, a sensor for detecting the reaction of the user using the terminal device 10. The sensor 12 is, for example, a camera that captures the facial expressions and actions of the user using the terminal device 10. The sensor 12 may include, for example, a biosensor for detecting bioinformation such as the heart rate and blood pressure of the user using the terminal device 10. The sensor 12 may include multiple types of sensors.

[0017] The input unit 14 accepts various operations for the terminal device 10. The input unit 14 is realized by various input devices such as a keyboard, a mouse, a switch, a button, and a touch panel.

[0018] The voice input unit 16 detects the voice of the user who is using the terminal device 10. The voice input unit 16 converts the detected voice into a voice signal. The voice input unit 16 is realized by a microphone.

[0019] The display unit 18 has a display surface that displays various information. The display unit 64 is realized by a display including, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 18 displays, for example, explanatory materials shared in an online conference. The shared explanatory materials are also called a shared screen. When the input unit 14 is a touch panel, the input unit 14 and the display unit 18 are integrally configured. Here, when the terminal device 10 is a device that displays image information by projecting or projecting, such as a HUD, the display unit 18 does not need to have a display surface.

[0020] The audio output unit 20 is a speaker that outputs various types of audio. For example, the audio output unit 20 outputs the audio of users of other terminal devices 10 participating in the online conference.

[0021] The communication unit 22 is a communication interface that performs communication between the terminal device 10 and an external device. The communication unit 22 executes communication between the terminal device 10 and the information processing device 100, for example. The communication unit 22 is realized by, for example, a wireless LAN (Local Area Network), a wired LAN, Wi-Fi (registered trademark), or the like. Specifically, the communication unit 22 performs communication that is distributed by a speaker to viewers in an online conference.

[0022] The storage unit 24 stores various types of information. The storage unit 24 stores information such as the contents of calculations performed by the control unit 26 and programs. The storage unit 24 includes at least one of a RAM (Random Access Memory), a main storage device such as a ROM (Read Only Memory), and an external storage device such as an HDD (Hard Disk Drive).

[0023] The control unit 26 controls each unit of the terminal device 10. The control unit 26 has, for example, an information processing device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), and a storage device such as a RAM or a ROM. The control unit 26 executes a program that controls the operation of the terminal device 10 according to the present invention. The control unit 26 may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 26 may be realized by a combination of hardware and software.

[0024] The control unit 26, for example, controls the sensor 12 to detect a reaction of the user who uses the terminal device 10. The control unit 26 transmits the reaction of the user detected by the sensor 12 to the information processing device, for example, via the communication unit 22. When the sensor 12 is, for example, a camera, the control unit 26 transmits image data of the user who uses the terminal device 10 to the information processing device 100 via the communication unit 22.

[0025] (Information processing device) An example of the configuration of the information processing device according to the first embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of the configuration of the information processing device according to the first embodiment.

[0026] The information processing device 100 allows viewers of an online conference to select a reaction to a statement made by a speaker. The information processing device 100 is, for example, a server device. The information processing device 100 may be realized, for example, in an online conference, by a terminal device 10 of an online conference organizer or a speaker who is speaking in the online conference. The information processing device 100 may be realized, for example, by both the server device and the terminal device 10. In this case, specifically, for example, the processes of a reaction detection unit 110, a reaction location detection unit 112, a reaction content generation unit 114, and a reaction content selection unit 116, which will be described later, are realized by the terminal device 10, and the processes of a display control unit 118, which will be described later, are realized by the server device.

[0027] As shown in FIG. 3, the information processing device 100 includes a communication unit 102, a storage unit 104, and a control unit 106.

[0028] The communication unit 102 is a communication interface that performs communication between the information processing device 100 and an external device. The communication unit 102 executes communication between the information processing device 100 and the terminal device 10, for example. The communication unit 102 is realized by, for example, a wireless LAN or a wired LAN.

[0029] The storage unit 104 stores various types of information. The storage unit 104 stores information such as the contents of calculations performed by the control unit 106 and programs. The storage unit 104 includes at least one of a RAM, a main storage device such as a ROM, and an external storage device such as a HDD.

[0030] The control unit 106 controls each unit of the information processing device 100. The control unit 106 has, for example, an information processing device such as a CPU or an MPU, and a storage device such as a RAM or a ROM. The control unit 106 executes a program that controls the operation of the information processing device 100 according to the present invention. The control unit 106 may be realized by an integrated circuit such as an ASIC or an FPGA. The control unit 106 may be realized by a combination of hardware and software.

[0031] The control unit 106 includes a reaction detection unit 110 , a reaction site detection unit 112 , a reaction content generation unit 114 , a reaction content selection unit 116 , and a display control unit 118 .

[0032] The reaction detection unit 110 detects reaction information indicating the reaction of the audience to the content of the speaker's presentation in the online conference. The reaction includes, but is not limited to, a change in the audience's facial expression, a head nodding or shaking action, a hand movement action, a change in the volume of the voice, and the like. For example, the reaction detection unit 110 detects the reaction information based on the photographed data of the user's facial expression acquired from the terminal device 10. For example, the reaction detection unit 110 detects the reaction information based on the voice data of the user acquired from the terminal device 10. The reaction detection unit 110 may detect the speed of the reaction. The speed of the reaction may be, for example, the time from the timing when the speaker speaks in the online conference to the timing when the user's reaction is detected.

[0033] The reaction part detection unit 112 detects reaction part information indicating a part where reaction information was detected by the reaction detection unit 110. For example, when the content of a speaker's speech is converted into text in real time, the reaction part detection unit 112 detects a part where the user of the terminal device 10 reacted in the text immediately before the reaction information was detected by the reaction detection unit 110 as reaction part information.

[0034] The reaction part detection unit 112 detects, as reaction part information, a part where the user of the terminal device 10 reacted, based on, for example, the content displayed on the shared screen at the time when the reaction information is detected by the reaction detection unit 110. The content displayed on the shared screen includes, for example, slides of presentation materials, URLs (Uniform Resource Locators), branches of a mind map, and the like, but is not limited to these.

[0035] The reaction point detection unit 112 detects, as reaction point information, a part where the speaker is pointing on the shared screen at the time when the reaction information is detected by the reaction detection unit 110. The point includes, but is not limited to, a laser pointer, a mouse cursor, a cursor, and the like.

[0036] The reaction content generation unit 114 generates reaction content information indicating the reaction content to the presentation content based on the reaction information detected by the reaction detection unit 110 and the reaction site information detected by the reaction site detection unit 112. The reaction content generation unit 114 may generate reaction content information based only on the reaction information without using the reaction site information. In other words, the reaction content generation unit 114 generates reaction content information based on the reaction information, or based on the reaction information and the reaction site information. Here, the reaction content generation unit 114 generates reaction content information using only the reaction information when, for example, the reaction site detection unit 112 cannot detect a reaction site.

[0037] The reaction content generation unit 114 may determine whether the reaction of the viewer to the presentation content is positive or negative based on the reaction information detected by the reaction detection unit 110 and the reaction part information detected by the reaction part detection unit 112. The reaction content generation unit 114 may determine whether the reaction of the viewer to the presentation content is positive or negative by analyzing, for example, a change in the viewer's facial expression, a change in the volume of the voice, or the speed of the reaction. The reaction content generation unit 114 may generate reaction content information indicating whether the reaction of the viewer to the presentation content is positive or negative.

[0038] The reaction content generating unit 114 may determine how interested the viewer is in the presentation content based on the reaction information detected by the reaction detecting unit 110 and the reaction part information detected by the reaction part detecting unit 112. The reaction content generating unit 114 may generate reaction content information indicating whether the viewer is interested in the presentation content.

[0039] The reaction content generation unit 114 may determine whether or not there is a question about the presentation content based on the reaction information detected by the reaction detection unit 110 and the reaction part information detected by the reaction part detection unit 112. The reaction content generation unit 114 may generate reaction content information indicating whether or not the viewer has a question about the presentation content.

[0040] The reaction content generation unit 114 generates one or more pieces of reaction content information for the reaction site detected by the reaction site detection unit 112. That is, the reaction content generation unit 114 generates reaction content information indicating whether the reaction site detected by the reaction site detection unit 112 is positive, negative, interested, or a question.

[0041] For example, when the content of a speaker's speech is converted into text in real time, the reaction content generation unit 114 may analyze the content of the text and generate text indicating an expected reaction of the audience based on the content of the text and the reaction information detected by the reaction detection unit 110. For example, the reaction content generation unit 114 generates pro-opinions, anti-opinions, questions, and the like as text indicating an expected reaction of the audience. For example, the reaction content generation unit 114 may generate text using a machine learning method.

[0042] The reaction content generation unit 114 may, for example, analyze the content displayed on the shared screen and generate text indicating the expected reaction of the viewers based on the content displayed on the shared screen and the reaction information detected by the reaction detection unit 110.

[0043] The reaction content generation unit 114 may, for example, analyze the content surrounding the location where the speaker is pointing on the shared screen, and generate text indicating an expected reaction based on the content surrounding the pointed location and the reaction information detected by the reaction detection unit 110.

[0044] The reaction content selection unit 116 presents the reaction content information generated by the reaction content generation unit 114 to the viewer who is the target of the reaction content information or other viewers different from the viewer, and allows them to select. Here, the other viewers are viewers who are not the target of the reaction content information, and may be, for example, the boss or colleagues of the target viewer. The other viewers may be the speakers. For example, the reaction content selection unit 116 displays a combination of the reaction content information generated by the reaction content generation unit 114 and the reaction location information detected by the reaction location detection unit 112 on the display unit 18 of the viewer's terminal device 10 as candidates for the viewer's reaction, and allows them to select. The viewer can select a desired reaction from the reaction content information displayed on the display unit of the terminal device 10 by the input unit 14.

[0045] The display control unit 118 displays the reaction content information selected by the viewer or other viewers on the display unit 18 of the terminal device 10 of the speaker. The display control unit 118 may also display the reaction content information selected by the viewer or other viewers on the display unit 18 of the terminal device 10 of the host of the online conference, such as a facilitator. Note that there may not be a facilitator in an online conference. For example, the display control unit 118 may display the comment content information in chronological order on the display unit 18 of the terminal device 10, but is not limited to this.

[0046] For example, the display control unit 118 may display reaction content information having similar meanings together, or may display the information in order of priority. The display control unit 118 may determine the priority according to the type of the content of the online conference or the content of the presentation.

[0047] For example, when the online conference is an approval conference for obtaining approval, the display control unit 118 may increase the priority of the reaction content information of important stakeholders who need to obtain approval. For example, when the online conference is a progress meeting, the display control unit 118 may increase the priority of opposing opinions. For example, when the online conference is an idea conference, the display control unit 118 may increase the priority of reaction content information for an agenda related to the idea conference.

[0048] (Reaction content presentation processing) The reaction content presentation process according to the first embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the flow of the reaction content presentation process according to the first embodiment.

[0049] The reaction detection unit 110 judges whether reaction information indicating the reaction of the viewers of the online conference is detected (step S10). If it is judged that the reaction information of the viewers is detected (step S10; Yes), the process proceeds to step S12 and step S14. In this embodiment, the processes of step S12 and step S14 are executed in parallel. If it is not judged that the reaction information of the viewers is detected (step S10; No), the process proceeds to step S22.

[0050] If the answer to step S10 is Yes, the reaction part detection unit 112 detects the reaction part where the viewer reacted (step S12). Specifically, the reaction part detection unit 112 detects the reaction part based on the reaction information detected by the reaction detection unit 110. Here, in the process of Fig. 4, the process of step S12 may be omitted. Then, the process proceeds to step S16.

[0051] Furthermore, if the answer to step S10 is Yes, the reaction content generation unit 114 generates reaction content information indicating the reaction content (step S14). Specifically, the reaction content generation unit 114 generates reaction content information based on the reaction information detected by the reaction detection unit 110. FIG. 5 is a diagram showing reaction content information according to the first embodiment. As shown in FIG. 5, reaction content information 201 includes items such as "type" and "content".

[0052] "Type" indicates the type of reaction made by the viewer. Examples of "Type" include, but are not limited to, "Agree", "Disagree", "Question", and "Interest". "Type" may indicate, for example, partial agreement or partial disagreement, such as "Partial agreement" or "Partial disagree".

[0053] The "content" indicates the specific content of the viewer's reaction. The "content" indicates the specific content in the form of a sentence, such as, for example, "It is better not to execute that process," "Where is the data that is the reason for that?", or "Either is fine." The reaction content generation unit 114 may create a sentence indicating the specific content using, for example, a machine learning method based on the reaction information detected by the reaction detection unit 110. The reaction content generation unit 114 may use, for example, a sentence prepared in advance according to the reaction information detected by the reaction detection unit 110. In this case, it is preferable that the storage unit 104 stores information in which the reaction information is associated with the sentence.

[0054] One example shown in FIG. 5 is "against" the speaker's statement, indicating that "it would be better not to carry out that process."

[0055] The reaction detection unit 110 judges whether or not the detection of the reaction information has ended (step S16). Here, the reaction detection unit 110 judges that the detection of the reaction information has ended, for example, when the speaker has finished speaking. The reaction detection unit 110 may judge that the detection of the reaction information has ended based on a preset position of a break in the presentation content. For example, when the presentation content uses a presentation file, the position of a break in the presentation content is when any slide in the presentation file switches to the next slide. If it is judged that the detection of the reaction information has ended (step S16: Yes), the process proceeds to step S18. If it is not judged that the detection of the reaction information has ended (step S16; No), the process proceeds to step S10.

[0056] If the answer is Yes in step S16, the reaction content selection unit 116 presents the reaction content information to the viewer or other viewers (step S18). The reaction content selection unit 116 displays the reaction content information on the display unit 18 of the terminal device 10 used by the viewer or other viewers, for example. The reaction content selection unit 116 allows the viewer or other viewers to select the reaction content information, for example. FIG. 6 is a diagram showing reaction content information presented to the viewer or other viewers according to the first embodiment. As shown in FIG. 6, the reaction content information 202 includes items such as "reaction location", "type", and "content". The reaction content selection unit 116 generates the reaction content information 202 based on the reaction content information 201 generated by the reaction content generation unit 114 and the reaction location detected by the reaction location detection unit 112. However, the reaction content selection unit 116 may present the reaction content information 201 without the item of the reaction location as shown in FIG. 5 to the viewer or other viewers.

[0057] "Reaction location" indicates the location where the target viewer's reaction was detected. "Reaction location" indicates, for example, which page and which location of the shared document the target viewer's reaction was detected. In the example shown in FIG. 6, the "reaction location" is written as "Page 1: Proposal to execute process," "Page 2: Reason for executing process," and "Page 1: Proposal to execute process." "Page 1: Proposal to execute process" indicates that the target viewer's reaction was detected in the part of the first page of the shared document that suggested executing the process.

[0058] 6, the word "Against" is displayed in the section "Page 1: Processing Execution Proposal", indicating that "It is better not to execute that processing." The viewer or another viewer can refer to the reaction content information 202 displayed on the display unit 18 and use the input unit 14 to select a desired reaction content.

[0059] The display control unit 118 presents the reaction content information selected by the viewer or another viewer to the speaker (step S20). The display control unit 118 displays the reaction content information selected by the viewer or another viewer on the display unit 18 of the terminal device 10 used by the speaker, for example.

[0060] Fig. 7 is a diagram showing an example of reaction content information presented to a speaker according to the first embodiment. As shown in Fig. 7, reaction content information 203 includes items such as "timing", "reaction location", and "type".

[0061] "Timing" indicates the timing at which the viewer selected the reaction content information. "Timing" is indicated by time, for example, "1 minute before," "30 seconds before," "5 seconds before," etc.

[0062] For example, one example shown in FIG. 7 indicates that the reaction content information indicating "agree" was selected at "Page 1: 'Of X and Y, adopt X'" "1 minute ago."

[0063] Fig. 8 is a diagram showing an example of reaction content presented to a speaker according to the first embodiment. As shown in Fig. 8, reaction content information 204 includes items such as "viewer", "reaction location", and "type".

[0064] The "viewer" indicates the viewer who selected the reaction content information. The "viewer" indicates the name of the viewer, for example, "Mr. A," "Mr. B," "Mr. C," etc.

[0065] For example, one example shown in FIG. 8 indicates that "Mr. A" "agrees" with the passage "Page 1: 'Of X and Y, adopt X.'"

[0066] The display control unit 118 may, for example, display either the reaction content information 203 or the reaction content information 204 on the display unit 18 of the terminal device 10 used by the speaker, or may display both. The speaker can properly grasp the audience's reactions by visually checking the reaction content information 203 and the reaction content information 204. This allows the speaker to properly proceed with the presentation, such as by answering questions from the audience.

[0067] The control unit 106 determines whether or not to end the process (step S22). Specifically, the control unit 106 determines to end the process when, for example, the online conference ends. If it is determined to end the process (step S22; Yes), the control unit 106 ends the process in FIG. 4. If it is not determined to end the process (step S22; No), the control unit 106 returns to step S10.

[0068] As described above, the first embodiment can easily convey the reactions of the viewers of an online conference to the speaker. As a result, the first embodiment allows the speaker to properly grasp the reactions of the viewers, and therefore allows the conference to proceed properly.

[0069] [First Modification of the First Embodiment] A description will be given of a modification of the first embodiment. In the first embodiment, the detection of reaction site information and the generation of reaction details information are executed in parallel, but the present invention is not limited to this.

[0070] (Reaction content presentation processing) The reaction content presentation process according to the first modified example of the first embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart showing the flow of the reaction content presentation process according to the first modified example of the first embodiment.

[0071] The processes in steps S30 and S32 are the same as those in steps S10 and S12 shown in FIG. 4, respectively, and therefore will not be described.

[0072] The reaction details generation unit 114 generates reaction details information based on the reaction information detected by the reaction detection unit 110 and the reaction site information detected by the reaction site detection unit 112 (step S34). That is, in the first modified example of the first embodiment, the reaction details information is generated after the reaction site information is detected. The reaction details generation unit 114 can generate reaction details information with higher accuracy by generating reaction details information based on the reaction site information in addition to the reaction information. The reaction details generation unit 114 can generate reaction site information shown in FIG. 6, for example.

[0073] The processes from step S36 to step S42 are the same as the processes from step S16 to step S22 shown in FIG. 4, respectively, and therefore will not be described.

[0074] As described above, in the first modification of the first embodiment as well, the speaker can properly grasp the reaction of the viewers, and therefore can properly proceed with the conference.

[0075] [Second Modification of the First Embodiment] A second modified example of the first embodiment will be described. When generating reaction content information, the reaction content generation unit 114 may determine the detection accuracy of the reaction information detected by the reaction detection unit 110. For example, when the detection accuracy of the reaction information is equal to or lower than a predetermined value, the reaction content generation unit 114 may generate reaction content information indicating multiple reactions for one reaction site.

[0076] Reaction content information to be presented to the viewer or other viewers according to the second modified example of the first embodiment will be described with reference to Fig. 10. Fig. 10 is a diagram showing reaction content information to be presented to the viewer or other viewers according to the second modified example of the first embodiment.

[0077] As shown in FIG. 10, the reaction content information 205 includes items such as "reaction location," "type," and "content." In the reaction content information 205, two reactions, "Agree" and "Disagree," are associated with "Page 1: Processing Execution Proposal." In this case, the viewer or another viewer can select either "Agree" or "Disagree." As a result, in the second modified example of the first embodiment, even if an erroneous determination by the reaction detection unit is expected, the speaker can properly grasp the reaction of the viewer.

[0078] [Second embodiment] A second embodiment will now be described. In the second embodiment, when reaction information is detected, it is determined whether or not the reaction made by the viewer is a reaction to the content of the speech made by the speaker.

[0079] (Reaction content presentation processing) The reaction content presentation process according to the second embodiment will be described with reference to Fig. 11. Fig. 11 is a flowchart showing the flow of the reaction content presentation process according to the second embodiment.

[0080] The process of step S50 is the same as the process of step S10 shown in FIG. 4, and therefore a description thereof will be omitted.

[0081] If the answer to step S50 is Yes, the reaction detection unit 110 judges whether the detected reaction information is reaction information to the presentation content (step S52). Specifically, for example, when the time between the timing of the speaker's speech and the timing of the detection of the reaction information is within a predetermined time, the reaction detection unit 110 judges that the reaction information is reaction information to the presentation content. For example, when the reaction detection unit 110 detects a change in the viewer's facial expression in response to the speaker's speech, the reaction detection unit 110 judges that the reaction information is reaction to the presentation content. For example, when the reaction detection unit 110 detects an utterance from the viewer in response to the speaker's speech, the reaction detection unit 110 judges that the reaction information is reaction information to the presentation content. For example, when the reaction detection unit 110 detects a state in which the viewer makes an interjection in response to the speaker's speech, the reaction detection unit 110 judges that the reaction information is reaction information to the presentation content. When the reaction detection unit 110 detects that the viewer's face is facing the display unit 18 at the timing of the speaker's speech, the reaction detection unit 110 judges that the reaction information is reaction information to the presentation content. If it is determined that the information is reaction information to the presentation content (step S52; Yes), the process proceeds to step S54 and step S56. If it is not determined that the information is reaction information to the presentation content (step S52; No), the process proceeds to step S64.

[0082] The processes from step S54 to step S64 are the same as the processes from step S12 to step S14 shown in FIG. 4, respectively, and therefore will not be described.

[0083] As described above, in the second embodiment, when the reaction information of the audience is detected, it is determined whether the reaction information is related to the presentation content. In this way, the second embodiment can prevent erroneous detection, so that the speaker can more appropriately grasp the reaction of the audience, and therefore the conference can be more appropriately conducted.

[0084] [Third embodiment] A third embodiment will now be described. In the third embodiment, when secondary communication is carried out between terminal devices of a plurality of viewers, reaction information is detected from the contents of the secondary communication.

[0085] The communication unit 22 in the third embodiment further performs communication between multiple terminal devices 10 used by multiple viewers. In the following description, the communication between multiple terminal devices 10 performed by the communication unit 22 may be referred to as secondary communication. The secondary communication is communication for viewers to talk with each other, separate from the communication performed by the speaker to the viewers in the online conference. Specifically, the communication unit 22 transmits and receives audio information between the multiple terminal devices 10, thereby enabling the multiple viewers to talk with each other. At this time, the communication unit 22 may output to the audio output unit 20, for example, a mixture of the voice of the speaker of the online conference and the voice of the secondary communication.

[0086] An example of the configuration of an information processing device according to the third embodiment will be described with reference to Fig. 12. Fig. 12 is a block diagram showing an example of the configuration of an information processing device according to the third embodiment.

[0087] As shown in FIG. 12, an information processing device 100A according to the third embodiment differs from the information processing device 100 shown in FIG.

[0088] The analysis unit 120 analyzes information regarding the content of communication between multiple terminal devices 10 connected via the communication unit 22. For example, when the communication unit 22 is transmitting and receiving voice signals, the analysis unit 120 analyzes the voice signals between the multiple terminal devices 10 using a known technique such as a trained model that realizes a voice recognition function, and converts the voice signals into text information representing the content of the conversation.

[0089] (Reaction content presentation processing) The reaction content presentation process according to the third embodiment will be described with reference to Fig. 13. Fig. 13 is a flowchart showing the reaction content presentation process according to the third embodiment.

[0090] The process of step SY6 is the same as the process of step S16 shown in FIG. 4, and the process of step SZ0 is the same as the process of step S20, so their explanation will be omitted.

[0091] The communication unit 22 starts secondary communication, which is communication between multiple terminal devices 10 used by multiple viewers (step SX8). For example, secondary communication is started by any viewer pressing a soft key for starting secondary communication displayed on the display unit 18 of the terminal device 10. At this time, it is assumed that the any viewer can select one or more viewers with whom to start secondary communication. Then, the process proceeds to step SY0.

[0092] The reaction detection unit 110 determines whether or not reaction information of the secondary communication has been detected based on the result of the analysis by the analysis unit 120 (step SY0). When the communication unit 22 is transmitting or receiving an audio signal, the reaction detection unit 110 detects reaction information of the secondary communication based on text information that represents the content of the conversation analyzed by the analysis unit 120.

[0093] For example, the reaction detection unit 110 determines that reaction information of the secondary communication has been detected when it detects, from the content of the conversation analyzed by the analysis unit 120, that a conversation between multiple viewers who are engaged in secondary communication has started. For example, the reaction detection unit 110 determines that reaction information of the secondary communication has been detected when it detects, from the content of the conversation analyzed by the analysis unit 120, that the opinions of the multiple viewers who are engaged in secondary communication agree. Here, the reaction detection unit 110 detects whether the opinions of the viewers agree by using a publicly known technique such as a trained model that determines whether opinions agree by inputting a conversation.

[0094] If it is determined that the reaction information of the secondary communication is detected (step SY0; Yes), proceed to step SY2 and step SY4. In this embodiment, the processes of step SY2 and step SY4 are executed in parallel. If it is determined that the reaction information of the secondary communication is not detected (step SY0; No), proceed to step SZ2.

[0095] If the answer to step SY0 is Yes, the reaction part detection unit 112 detects reaction parts at which the multiple viewers who are engaged in the secondary communication reacted (step SY2), and then the process proceeds to step SY6.

[0096] If the answer to the question in step SY0 is Yes, the reaction content generating unit 114 generates reaction content information indicating the reaction content of the multiple viewers who are engaged in the secondary communication (step SY4), and then the process proceeds to step SY6.

[0097] If the answer to step SY6 is Yes, the reaction content selection unit 116 presents the reaction content information to the multiple viewers who are engaged in the secondary communication (step SY8). For example, the reaction content selection unit 116 displays the reaction content information on the display unit 18 of the terminal device 10 used by the multiple viewers who are engaged in the secondary communication. Here, the reaction content selection unit 116 may present the reaction content information to all of the multiple viewers who are engaged in the secondary communication, or may present the reaction content information to some of the multiple viewers, or to only one viewer. For example, the multiple viewers who are engaged in the secondary communication can select a desired reaction content using the input unit 14 by referring to the reaction content information 202 displayed on the display unit 18. Here, when the reaction content selection unit 116 presents the reaction content information to all of the multiple viewers who are engaged in the secondary communication, it may adopt only the reaction content information selected by the most viewers as the desired reaction content. Then, the process proceeds to step SZ0.

[0098] The communication unit 22 determines whether or not to end the secondary communication (step SZ2). Specifically, the communication unit 22 determines to end the secondary communication when, for example, any viewer presses a soft key for ending the secondary communication displayed on the display unit 18 of the terminal device 10. If it is determined to end the secondary communication (step SZ2; Yes), the processing of Fig. 13 is terminated. If it is not determined to end the secondary communication (step SZ2; No), the processing returns to step SY0.

[0099] As described above, in the third embodiment, when secondary communication is performed between terminal devices of multiple viewers, reaction information is detected from the contents of the secondary communication. This allows the third embodiment to appropriately proceed with a conference even in a wider variety of types of conference.

[0100] Each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc. This distribution and integration configuration may be performed dynamically.

[0101] Although the embodiments of the present invention have been described above, the present invention is not limited to the contents of these embodiments. The above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-described components can be appropriately combined. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the above-described embodiments. [Explanation of symbols]

[0102] 1. Information Processing Systems 10 Terminal Equipment 12 Sensors 14 Input section 16 Audio input section 18 Display 20 Audio output section 22,102 Communications Department 24,104 storage section 26,106 Control section 100 Information processing device 110 Reaction detection unit 112 Reaction site detection unit 114 Reaction Content Generation Unit 116 Reaction Content Selection Section 118 Display control section 120 Analysis Department

Claims

1. A reaction detection unit that detects reaction information indicating a reaction of a viewer to a presentation content of a speaker in an online conference; A reaction site detection unit that detects reaction site information indicating a site where the reaction information is detected; a reaction content generating unit that generates reaction content information indicating a reaction content to the presentation content based on the reaction information or based on the reaction information and the reaction site information; a reaction content selection unit that presents the reaction content information to the viewer or other viewers different from the viewer and allows them to select a reaction content; An information processing device comprising:

2. The reaction content information includes information on whether the reaction of the viewer is a question, and if the reaction of the viewer is a question, includes text information indicating the content of the question. The information processing device according to claim 1 .

3. A display control unit is further provided that displays the reaction content information selected by the viewer or the other viewer on a terminal device used by the speaker, the display control unit determines a priority of displaying the reaction content information according to a type of the presentation content.

3. The information processing device according to claim 1 or 2.

4. An analysis unit that analyzes information regarding secondary communication between terminal devices used by a plurality of the viewers, The reaction detection unit detects the reaction information based on the result of the analysis by the analysis unit.

3. The information processing device according to claim 1 or 2.

5. The reaction detection unit detects the reaction information when it detects that opinions of a plurality of viewers coincide from the analysis result by the analysis unit. The information processing device according to claim 4.

6. Detecting reaction information indicating a reaction of a viewer to a presentation content of a speaker in an online conference; detecting reaction site information indicating a site where the reaction information was detected; generating reaction content information indicating a reaction content to the presentation content based on the reaction information or based on the reaction information and the reaction site information; An information processing method comprising:

Images