Information processing device and program

Abstract

To appropriately give an opportunity for speech to a person who is expected to speak, in a web conference.SOLUTION: An information processing apparatus includes: a display control unit which causes display screens of information terminal devices used by participants to display participant information indicating a plurality of participants participating in a web conference; a line-of-sight detection unit which detects lines of sight of the participants with respect to the display screens displaying the participant information; and an identification unit which identifies participant information at which the participants gazed for a predetermined time or longer. The display control unit causes the participant information identified by the identification unit, to be displayed in preference to other participant information on the display screen.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an information processing device and an information processing method. [Background technology]

[0002] Due to the widespread use of web conferences, various conferences are held in the form of web conferences. In many cases, participants in a conference make voluntary remarks, but there are also cases where a person who is expected to speak does not speak voluntarily, or does not have the opportunity to speak. Patent Document 1 discloses a technology that displays a message encouraging a user who speaks less than a certain frequency to speak. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-113066 A Summary of the Invention [Problem to be solved by the invention]

[0004] When the technology described in Patent Document 1 is used, a message is displayed to encourage participants who do not speak often to speak, but it does not identify people who are expected to speak.

[0005] An object of the present invention is to provide an information processing device and an information processing method that can provide an appropriate opportunity to speak to a person who is expected to speak in a web conference. [Means for solving the problem]

[0006] The information processing device of the present invention comprises a display control unit that displays participant information indicating multiple participants participating in a WEB conference on a display screen of an information terminal device used by the participants, a gaze detection unit that detects the participant's gaze toward the display screen displaying the participant information, and an identification unit that identifies participant information to which the participant's gaze is directed for a predetermined period of time or longer, and the display control unit displays the participant information identified by the identification unit on the display screen in priority to other participant information.

[0007] The information processing method of the present invention is performed by an information processing device, which performs the following steps: displaying participant information indicating multiple participants participating in a WEB conference on a display screen of an information terminal device used by the participants; detecting the participant's gaze toward the display screen displaying the participant information; identifying participant information to which the participant's gaze is directed for a predetermined period of time or longer; and displaying the identified participant information on the display screen in priority over other participant information. Effect of the Invention

[0008] According to the present invention, in a Web conference, it is possible to provide an appropriate opportunity to speak to a person who is expected to speak. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an information processing system according to the first embodiment. [Diagram 2] FIG. 2 is a block diagram showing an example of the configuration of the information terminal device according to the first embodiment. [Diagram 3] FIG. 3 is a block diagram illustrating an example of the configuration of a server device according to the first embodiment. [Figure 4] FIG. 4 is a block diagram illustrating an example of the configuration of the information processing device according to the first embodiment. [Diagram 5] FIG. 5 is a flowchart showing an information processing method executed by the information processing device according to the first embodiment. [Figure 6]FIG. 6 is a diagram showing an example of a display screen of a web conference according to the first embodiment. [Figure 7] FIG. 7 is a diagram for explaining a method for increasing the display priority of participant information and displaying it according to the first embodiment. [Figure 8] FIG. 8 is a flowchart showing an information processing method according to the second embodiment. [Figure 9] FIG. 9 is a block diagram showing an example of the configuration of an information processing device according to the third embodiment. [Figure 10] FIG. 10 is a flowchart showing an information processing method executed by the information processing device according to the third embodiment. [Figure 11] FIG. 11 is a flowchart showing an information processing method executed by the information processing device according to the fourth embodiment. [Figure 12] FIG. 12 is a flowchart showing an information processing method executed by the information processing device according to the fifth embodiment. [Figure 13] FIG. 13 is a flowchart showing an information processing method executed by the information processing device according to the sixth embodiment. [Figure 14] FIG. 14 is a flowchart showing an information processing method executed by the information processing device according to the seventh embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to this embodiment, and in the following embodiments, the same components are designated by the same reference numerals, and duplicated explanations will be omitted.

[0011] [First embodiment] (Information Processing System) An example of the configuration of an information processing system according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of an information processing system according to the first embodiment.

[0012] 1, an information processing system 1 includes a plurality of information terminal devices 10 and a server device 12. Each information terminal device 10 can hold a WEB conference with the other information terminal devices 10 via the server device 12 by starting a WEB conference application.

[0013] (Information terminal device) An example of the configuration of an information terminal device according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the configuration of an information terminal device according to the first embodiment.

[0014] 2, the information terminal device 10 includes a camera 20, a microphone 22, an operation unit 24, a communication unit 26, a display unit 28, and a terminal control device 30. Examples of the information terminal device 10 include information terminal devices such as a desktop personal computer (PC), a notebook PC, and a tablet terminal.

[0015] The camera 20 captures an image of a user standing in front of the information terminal device 10. The camera 20 captures an image of a user using the information terminal device 10. The camera 20 is, for example, an in-camera provided on the display unit 28 side. The camera 20 may be an external camera connected to the information terminal device 10.

[0016] The microphone 22 picks up the voice spoken by the user of the information terminal device 10. The microphone 22 outputs voice information relating to the picked up voice to the voice acquisition unit 44 of the terminal control device 30.

[0017] The operation unit 24 accepts various input operations for the information terminal device 10. The operation unit 24 outputs an operation signal corresponding to the accepted input operation to the terminal control device 30. The operation unit 24 includes, for example, a keyboard, a mouse, a touch panel, buttons, switches, etc. When a touch panel is used as the operation unit 24, the operation unit 24 is disposed on the display unit 28.

[0018] The communication unit 26 communicates with an external device via a wired or wireless network. The communication unit 26 communicates with the server device 12, for example, via a wired or wireless network.

[0019] The display unit 28 displays various types of video including characters and images on a display screen. The display unit 28 is, for example, a display including a liquid crystal display (LCD) or an organic EL (Electro-Luminescence) display.

[0020] The terminal control device 30 controls each part of the information terminal device 10. The terminal control device 30 has, for example, an information processing device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), and a storage device such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The terminal control device 30 executes a program that controls the operation of the information terminal device 10 according to the present invention. The terminal control device 30 may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The terminal control device 30 may be realized by a combination of hardware and software.

[0021] The terminal control device 30 has functional blocks realized by circuits or executed programs, including a shooting control unit 40, a communication control unit 42, an audio acquisition unit 44, a display control unit 46, an operation control unit 48, an application control unit 50, a memory unit 52, and a gaze detection unit 54.

[0022] The photographing control unit 40 controls the camera 20 to photograph an image. The photographing control unit 40 acquires image data relating to the image photographed by the camera 20. Specifically, the photographing control unit 40 acquires image data photographed by the camera 20 of a user using the information terminal device 10. The image data acquired by photographing with the camera 20 is image data of a moving image.

[0023] The communication control unit 42 controls the communication unit 26 to execute communication between the information terminal device 10 and an external device. Specifically, the communication control unit 42 controls, for example, the communication unit 26 to control communication between the information terminal device 10 and the server device 12. The communication control unit 42 controls, for example, the communication unit 26 to transmit and receive information about a WEB conference with another information terminal device 10 to and from the server device 12. The communication control unit 42 controls, for example, the communication unit 26 to transmit to the server device 12 voice information about the speech of a user using the information terminal device 10 acquired by the voice acquisition unit 44 and image data of a user using the information terminal device 10 acquired by the image capture control unit 40. The communication control unit 42 controls, for example, the communication unit 26 to receive information necessary for a WEB conference, such as speech and image data of a user of another information terminal device 10, from the server device 12.

[0024] The voice acquiring unit 44 acquires voice information related to the voice detected by the microphone 22. The voice acquiring unit 44 acquires voice information related to the speech of the user who uses the information terminal device .

[0025] The display control unit 46 controls the display unit 28 to display various videos including characters and images. For example, the display control unit 46 controls the display unit 28 to display information obtained by executing a WEB conference application (hereinafter, a WEB conference screen). The display control unit 46 may generate a WEB conference screen and display it on the display unit 28, or the display control unit 46 may display a WEB conference screen generated by the server device 12 on the display unit 28.

[0026] The operation control unit 48 acquires an operation signal corresponding to an operation accepted by the operation unit 24 from the operation unit 24. The operation control unit 48 outputs a control signal corresponding to the acquired operation signal to each unit.

[0027] The application control unit 50 controls the operation of various applications that run on the information terminal device 10. The application control unit 50 controls the operation of applications such as a WEB conference tool and a chat tool. The application control unit 50 may be configured to run an application such as a WEB conference tool in the server device 12 instead of running an application such as a WEB conference tool in the information terminal device 10. The application control unit 50 may also run an application such as a WEB conference tool in a distributed manner in the information terminal device 10 and the server device 12. In other words, the application control unit 50 may be a functional block realized only in the information terminal device 10, as well as a functional block realized in the server device 12, or a functional block realized in the information terminal device 10 and the server device 12.

[0028] The storage unit 52 stores information such as the contents of calculations and programs of the terminal control device 30. The storage unit 52 is composed of, for example, a RAM, a main storage device such as a ROM, and a storage device such as an SSD (Solid State Drive).

[0029] The gaze detection unit 54 detects the gaze of the user of the information terminal device 10. For example, the gaze detection unit 54 detects the gaze direction of the user of the information terminal device 10 from an image captured by the camera 20 provided on the display unit 28 side. For example, the gaze detection unit 54 recognizes the user's face from the image capture data captured by the camera 20 acquired by the image capture control unit 40, and detects the gaze direction from the detected eye image. The gaze detection unit 54 transmits the detection result of the gaze of the user of the information terminal device 10 to the server device 12.

[0030] (Server device) An example of the configuration of the server device according to the first embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing an example of the configuration of the server device according to the first embodiment.

[0031] 3, the server device 12 includes a communication unit 60 and a server control device 62. The server device 12 is a general-purpose server device realized by a computer. The server device 12 is a server of a company that provides a web conference tool or a web conference application.

[0032] The communication unit 60 communicates with an external device via a wired or wireless network. The communication unit 60 communicates with the information terminal device 10, for example, via a wired or wireless network.

[0033] The server control device 62 controls each unit of the server device 12. The server control device 62 has, for example, an information processing device such as a CPU or an MPU, and a storage device such as a RAM or a ROM. The server control device 62 executes a program that controls the operation of the server device 12 according to the present invention. The server control device 62 may be realized by an integrated circuit such as an ASIC or an FPGA. The server control device 62 may be realized by a combination of hardware and software.

[0034] The server control device 62 includes, as functional blocks realized by circuits or programs executed, a connection control unit 70, a communication control unit 72, and a storage unit 74. As described above, the server control device 62 may include an application control unit that operates an application such as a web conference tool.

[0035] The connection control unit 70 controls the connection between the server device 12 and the information terminal device 10. The connection control unit 70 establishes a connection between the server device 12 and the information terminal device 10.

[0036] The communication control unit 72 controls the communication unit 60 to execute communication between the server device 12 and an external device. Specifically, the communication control unit 72 controls, for example, the communication unit 60 to control communication between the server device 12 and the information terminal device 10. The communication control unit 72 controls, for example, the communication unit 60 to acquire information indicating a detection result of the line of sight of a user using the information terminal device 10 from the information terminal device 10. The communication control unit 72 controls, for example, the communication unit 60 to acquire information indicating a detection result of an utterance of a user using the information terminal device 10 from the information terminal device 10. The communication control unit 72 transmits, for example, information indicating a detection result of an utterance of a user using the information terminal device 10 to another information terminal device 10. The communication control unit 72 transmits, for example, information indicating a detection result of an utterance of a user using the information terminal device 10 to another information terminal device 10.

[0037] The storage unit 74 stores information such as programs and the contents of calculations performed by the connection control unit 70 and the communication control unit 72. The storage unit 74 is composed of a main storage device such as a RAM and a ROM, and a storage device such as an SSD or a hard disk drive (HDD).

[0038] (Information processing device) An example of the configuration of the information processing device according to the first embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram showing an example of the configuration of the information processing device according to the first embodiment.

[0039] 4, the information processing device 80 includes a display control unit 82, a gaze detection unit 84, and a determination unit 86. The information processing device 80 is a conceptual device realized by the cooperation of either one or both of the terminal control device 30 and the server control device 62. Therefore, either the information terminal device 10 or the server device 12 may be referred to as the information processing device 80, and the information processing system 1 consisting of the information terminal device 10 and the server device 12 may be referred to as the information processing device 80. Therefore, the display control unit 82, the gaze detection unit 84, and the determination unit 86 included in the information processing device 80 are realized by either one or both of the terminal control device 30 and the server control device 62.

[0040] The display control unit 82 displays participant information indicating a plurality of participants participating in the WEB conference on the display unit 28 of each information terminal device 10 used by the participants. For example, based on the participant information specified by the specifying unit 86, the display control unit 82 displays the participant information of the participant who has been specified to have directed his / her gaze toward the display unit 28 for a predetermined period of time or more on the display unit 28 of each information terminal device 10 of the participants participating in the WEB conference preferentially over other participant information. In other words, the display control unit 82 displays the participant information of the participant who has directed his / her gaze toward the display unit 28 for a predetermined period of time or more in the WEB conference prominently on the display unit 28 of each information terminal device 10 of the participants participating in the WEB conference. For example, the display control unit 82 may transmit instruction information indicating a display form of information to each information terminal device 10, and the display control unit 46 of each information terminal device 10 may display the participant information of the participant who has directed his / her gaze toward the display unit 28 for a predetermined period of time or more on the display unit 28 in accordance with the instruction information. For example, the gaze detection unit 84 may transmit detection results indicating the results detected by the gaze detection unit 84 to each information terminal device 10, and the display control unit 46 of each information terminal device 10 may configure the display form to display on the display unit 28 the participant information of participants who directed their gaze at the display unit 28 for a predetermined period of time or more.

[0041] The gaze detection unit 84 detects which participant information displayed on the display unit 28 is being looked at by the user of each information terminal device 10, based on the coordinate range of each participant information displayed on the display unit 28 of the information terminal device 10 by the display control unit 82 and the coordinate range of the display screen to which the gaze is directed. The gaze detection unit 84 acquires information indicating which participant information displayed on the display unit 28 is being looked at by the user of each information terminal device 10, and information on the time the user is looking, and transmits the information to the server device 12. Alternatively, when the gaze detection unit 84 detects that the user of each information terminal device 10 has looked at the participant information displayed on the display unit 28 for two seconds or more, the gaze detection unit 84 may transmit the participant information and information on the time the user is looking to the server device 12.

[0042] The identifying unit 86 identifies participant information of participants who have directed their gaze toward the display unit 28 for a predetermined period of time or more based on the detection result of the gaze of each participant by the gaze detection unit 84. The identifying unit 86 identifies, for example, participant information of a predetermined number of participants out of the multiple participants who have directed their gaze toward the display unit 28 for a predetermined period of time or more. The identifying unit 86 identifies participant information of the participant for which the cumulative total time that the multiple participants have directed their gaze is the longest.

[0043] (Information processing method) The information processing method according to the first embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the information processing method executed by the information processing device according to the first embodiment.

[0044] The process shown in FIG. 5 is started when a Web conference in which multiple participants participate is started.

[0045] When the WEB conference is started, the display control unit 82 starts displaying information indicating each participant of the WEB conference, that is, displaying a WEB conference screen (step S10). Then, the process proceeds to step S12. Specifically, the display control unit 82 displays information of each participant on each information terminal device 10. FIG. 6 is a diagram showing an example of a display screen of the WEB conference according to the first embodiment. FIG. 6 shows an example of a screen displayed on the display unit 28 of the information terminal device 10 of each participant participating in the WEB conference. As shown in FIG. 6, for example, when twelve people, participants P1 to P12, are participating in the WEB conference, a window W is displayed for each participant. In the example shown in FIG. 6, participant P6 is speaking, and in this case, the display control unit 82 explicitly displays the participant information of participant P6. For example, the window W of the participant who is speaking is displayed with a frame or larger than the other windows W, like the window of participant P6. In each window W, information indicating each participant, such as the camera image of each participant and the initials of each participant, is randomly displayed. For example, the window W of participant P1 may be displayed in a large window, or the window W of participant P7 may be displayed in a small window, and the display order may be changed arbitrarily. Depending on the number of participants in the WEB conference, the window W may not be displayed for some participants. In other words, the display format of the information of the participants other than the participant P6 who is speaking may be arbitrary.

[0046] The gaze detection unit 84 starts detecting the gaze of each participant (step S12). Specifically, the gaze detection unit 84 detects which participant information the user of each information terminal device 10 is looking at among the participant information displayed on the display unit 28. Then, the process proceeds to step S14.

[0047] The identification unit 86 judges whether participant information seen by a participant for a predetermined time or more has been detected based on the gaze detection result by the gaze detection unit 84 (step S12). Specifically, the identification unit 86 judges whether participant information seen by a predetermined number of participants for a predetermined time or more has been detected based on the gaze detection result by the gaze detection unit 84 (step S14). The predetermined time is, for example, 2 seconds, but is not limited thereto. The predetermined number is, for example, 3 people when there are 10 to 20 participants, but is not limited thereto. For example, the identification unit 86 detects participant information that 3 or more participants simultaneously or almost simultaneously direct their gaze at for 2 seconds or more. If it is judged that participant information seen by a predetermined number of participants for a predetermined time or more has been detected (step S14; Yes), the process proceeds to step S16. If it is not judged that participant information seen by a predetermined number of participants for a predetermined time or more has been detected (step S14; No), the process proceeds to step S18.

[0048] If the answer is Yes in step S14, the display control unit 82 increases the display priority of the participant information viewed by a predetermined number of participants or more for a predetermined time or more (step S16). Specifically, the display control unit 82 displays the participant information viewed by a predetermined number of participants or more for a predetermined time or more in a more prominent position. FIG. 7 is a diagram for explaining a method for increasing the display priority of the participant information according to the first embodiment. In the example shown in FIG. 7, for example, a display screen in which the participant information of the participant P11 is viewed by four participants for two seconds or more is shown. Here, the four participants may or may not include the participant P6 who is speaking. In this case, the display control unit 82, for example, moves the participant information of the participant P11 next to the participant information of the participant P6 who is speaking and enlarges the window W to make the participant P11 prominent.

[0049] The window W that was displayed with the increased display priority in step S16 may be displayed with the increased display priority for a predetermined time, such as five seconds. In other words, when the predetermined time has elapsed, the display with the increased display priority is terminated.

[0050] The information processing device 80 judges whether the WEB conference has ended (step S18). If it is judged that the WEB conference has ended (step S18; Yes), the process in FIG. 5 ends. If it is not judged that the WEB conference has ended (step S18; No), the process proceeds to step S12.

[0051] As described above, in the first embodiment, in a Web conference in which multiple participants participate, participant information that has been seen by a predetermined number of participants for a predetermined time or more can be displayed in a prominent position. As a result, the first embodiment can provide participants who are expected to speak with an appropriate opportunity to speak, making it easier for the participants who are expected to speak to speak. In addition, in the first embodiment, the participant information can be displayed in a prominent position by directing the gaze toward the participant information, making it easier for other participants to encourage a specific participant to speak.

[0052] [Second embodiment] The information processing method according to the second embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the information processing method according to the second embodiment.

[0053] The processing from step S20 to step S24 shown in FIG. 8 is the same as the processing from step S10 to step S14 shown in FIG. 5, and therefore the description thereof will be omitted.

[0054] If the answer to step S24 is Yes, the display control unit 82 increases the display priority of the most viewed participant information among the participant information viewed by a predetermined number of participants or more for a predetermined time or more (step S26). The method of increasing the display priority is the same as that of step S16 shown in FIG. 5, and therefore the explanation will be omitted.

[0055] The process of step S28 is the same as the process of step S18 shown in FIG. 5, and therefore a description thereof will be omitted.

[0056] As described above, in the second embodiment, in a Web conference in which multiple participants participate, the most viewed participant information among the participant information viewed for a predetermined time or more and in the participant information of a predetermined number of participants or more can be displayed in a prominent position. As a result, the second embodiment can provide an appropriate speaking opportunity to the participant who is most expected to speak, making it easier for the participant who is most expected to speak to speak. In addition, in the second embodiment, the participant information can be displayed in a prominent position by directing the gaze to the participant information, making it easier for other participants to encourage a specific participant to speak.

[0057] [Third embodiment] An example of the configuration of an information processing device according to the third embodiment will be described with reference to Fig. 9. Fig. 9 is a block diagram showing an example of the configuration of an information processing device according to the third embodiment.

[0058] 9, the information processing device 80A includes a display control unit 82, a gaze detection unit 84, a specification unit 86A, and an utterance detection unit 88. The information processing device 80A differs from the information processing device 80 shown in FIG. 4 in that the information processing device 80A includes the utterance detection unit 88. The information processing device 80A is a conceptual device that is realized by cooperation of either or both of the terminal control device 30 and the server control device 62, similar to the information processing device 80 shown in FIG. 4. Therefore, the utterance detection unit 88 included in the information processing device 80A is also realized by either or both of the terminal control device 30 and the server control device 62.

[0059] The speech detection unit 88 detects speech in the WEB conference of each participant who is using the information terminal device 10 to participate in the WEB conference. The speech detection unit 88 detects the speech state of the user of each information terminal device 10 based on the voice acquired by the voice acquisition unit 44 of each information terminal device 10. Specifically, the speech detection unit 88 detects, for example, a speaker who is speaking for a predetermined time or more, or detects a break in the speaker's speech. A break in speech refers to a blank period of speech detected following a state in which continuous speech is being made. The blank period of speech is, for example, 2 seconds, but is not limited to this. For example, the speech detection unit 88 detects speech from other participants other than the speaker while the speaker is speaking.

[0060] (Information processing method) An information processing method according to the third embodiment will be described with reference to Fig. 10. Fig. 10 is a flowchart showing the information processing method executed by the information processing device according to the third embodiment.

[0061] The processes from step S30 to step S32 are the same as the processes from step S10 to step S12 shown in FIG. 5, respectively, and therefore will not be described.

[0062] The speech detection unit 88 judges whether or not a participant who has been speaking for a predetermined time or longer has been detected (step S34). If it is judged that a participant who has been speaking for a predetermined time or longer has been detected (step S34; Yes), the process proceeds to step S36. If it is not judged that a participant who has been speaking for a predetermined time or longer has been detected (step S34; No), the process proceeds to step S42.

[0063] If it is determined that the answer is Yes in step S34, the speech detection unit 88 determines whether or not a speech boundary has been detected (step S36). If it is determined that a speech boundary has been detected (step S36; Yes), the process proceeds to step S38. If it is determined that a speech boundary has not been detected (step S36; No), the process proceeds to step S42.

[0064] If the answer to step S36 is Yes, the identification unit 86A determines whether or not participant information seen by a predetermined number of participants for a predetermined time or more was detected when a speech break was detected based on the gaze detection result by the gaze detection unit 84 (step S38). If it is determined that participant information seen by a predetermined number of participants for a predetermined time or more was detected when a speech break was detected (step S38; Yes), the process proceeds to step S40. If it is not determined that participant information seen by a predetermined number of participants for a predetermined time or more was detected when a speech break was detected (step S38; No), the process proceeds to step S42.

[0065] The processes in steps S40 and S42 are the same as those in steps S16 and S18 shown in FIG. 5, respectively, and therefore will not be described.

[0066] As described above, in the third embodiment, when a break in speech is detected in a Web conference in which multiple participants participate, participant information that has been seen by a predetermined number of participants for a predetermined time or more can be displayed in a prominent position. This allows the third embodiment to provide an appropriate opportunity for a participant who is expected to speak, making it easier for the participant who is expected to speak when a break in speech is detected.

[0067] [Fourth embodiment] An information processing method according to the fourth embodiment will be described with reference to Fig. 11. Fig. 11 is a flowchart showing an information processing method executed by an information processing device according to the fourth embodiment.

[0068] The information processing device according to the fourth embodiment is the same as the information processing device 80A shown in FIG. 9, and therefore a description thereof will be omitted.

[0069] The processes in steps S50 and S52 are the same as those in steps S10 and S12 shown in FIG. 5, respectively, and therefore will not be described.

[0070] The speech detection unit 88 determines whether or not there has been a speech from each participant (step S54). If it is determined that there has been a speech (step S54; Yes), the process proceeds to step S56. If it is determined that there has not been a speech (step S54; No), the process proceeds to step S62.

[0071] If it is determined that step S54 is Yes, the identification unit 86A determines whether or not participant information that was looked at by the speaker for a predetermined time or longer while the speaker was speaking was detected based on the gaze detection result by the gaze detection unit 84 (step S56). The speaker here is the speaker determined to have spoken in step S54. If it is determined that participant information that was looked at by the speaker for a predetermined time or longer while the speaker was speaking was detected (step S56; Yes), proceed to step S58. If it is not determined that participant information that was looked at by the speaker for a predetermined time or longer while the speaker was speaking was detected (step S56; No), proceed to step S60.

[0072] If the answer is Yes in step S56, the display control unit 82 increases the display priority of the participant information that was viewed by the speaker for a predetermined period of time or longer while the speaker was speaking (step S58). The method of increasing the display priority is the same as that in step S16 shown in Fig. 5, and therefore the explanation will be omitted. Then, the process proceeds to step S60.

[0073] The speech detection unit 88 determines whether the speaker has finished speaking (step S60). Specifically, the speech detection unit 88 determines that the speech has ended when it detects a blank period longer than the time determined to be a break in the speech by a predetermined amount or more. If it is determined that the speech has ended (step S60; Yes), the process proceeds to step S62. If it is not determined that the speech has ended (step S60; No), the process proceeds to step S56.

[0074] The process of step S62 is the same as the process of step S18 shown in FIG. 5, and therefore a description thereof will be omitted.

[0075] As described above, in a Web conference in which multiple participants participate, the fourth embodiment can display participant information that has been viewed by a speaker for a predetermined period of time or more in a prominent position while the speaker is speaking. This allows the fourth embodiment to provide an appropriate opportunity to a participant who is expected to speak by the speaker while the speaker is speaking. In addition, in the fourth embodiment, the speaker can direct his or her gaze toward the participant information to display the participant information in a prominent position, making it easier for the speaker to encourage a specific participant to speak.

[0076] [Fifth embodiment] An information processing method according to the fifth embodiment will be described with reference to Fig. 12. Fig. 12 is a flowchart showing the information processing method executed by the information processing device according to the fifth embodiment.

[0077] The information processing device according to the fifth embodiment is the same as the information processing device 80A shown in FIG. 9, and therefore a description thereof will be omitted.

[0078] The processes from step S70 to step S76 are the same as the processes from step S50 to step S56 shown in FIG. 11, respectively, and therefore will not be described.

[0079] If the answer is Yes in step S76, the display control unit 82 increases the display priority of the participant information that was most viewed by the speaker while the speaker was speaking (step S78). The method of increasing the display priority is the same as that in step S16 shown in Fig. 5, so the explanation will be omitted. Then, the process proceeds to step S80.

[0080] The processes in steps S80 and S82 are the same as those in steps S60 and S62 shown in FIG. 11, respectively, and therefore will not be described.

[0081] As described above, in the fifth embodiment, in a Web conference in which multiple participants participate, the participant information most viewed by the speaker can be displayed in a prominent position while the speaker is speaking. As a result, the fifth embodiment can provide an appropriate opportunity to speak to the participant most expected to speak by the speaker while the speaker is speaking. In addition, in the fifth embodiment, the speaker can direct his / her gaze to the participant information to display the participant information in a prominent position, making it easier for the speaker to encourage a specific participant to speak.

[0082] [Sixth embodiment] An information processing method according to the sixth embodiment will be described with reference to Fig. 13. Fig. 13 is a flowchart showing an information processing method executed by an information processing device according to the sixth embodiment.

[0083] The information processing device according to the sixth embodiment is the same as the information processing device 80A shown in FIG. 9, and therefore a description thereof will be omitted.

[0084] The processes from step S90 to step S94 are the same as the processes from step S50 to step S54 shown in Fig. 11, respectively, and therefore their explanation will be omitted. The process from step S96 is the same as the process from step S36 shown in Fig. 10, and therefore its explanation will be omitted.

[0085] If the answer to step S96 is Yes, the identification unit 86A determines whether participant information that was looked at by the speaker for a predetermined time or longer was detected when a speech break was detected based on the gaze detection result by the gaze detection unit 84 (step S98). If it is determined that participant information that was looked at by the speaker for a predetermined time or longer was detected when a speech break was detected (step S98; Yes), the process proceeds to step S100. If it is not determined that participant information that was looked at by the speaker for a predetermined time or longer was detected when a speech break was detected (step S98; No), the process proceeds to step S102.

[0086] If the answer to step S98 is Yes, the display control unit 82 increases the display priority of the participant information that was viewed by the speaker for a predetermined period of time or longer when a break in the speech was detected (step S100). The method of increasing the display priority is the same as that of step S16 shown in FIG. 5, and therefore the explanation will be omitted. Then, the process proceeds to step S102.

[0087] The processes in steps S102 and S104 are the same as those in steps S60 and S62 shown in FIG. 11, respectively, and therefore will not be described.

[0088] As described above, in the sixth embodiment, when a break in speech is detected in a Web conference in which multiple participants participate, the participant information most viewed by the speaker can be displayed in a prominent position. As a result, in the sixth embodiment, when a break in speech is detected, the participant most expected by the speaker to speak can be given an appropriate opportunity to speak. In addition, in the sixth embodiment, the speaker can direct his / her gaze toward the participant information to display the participant information in a prominent position, making it easier for the speaker to encourage a specific participant to speak.

[0089] [Seventh embodiment] An information processing method according to the seventh embodiment will be described with reference to Fig. 14. Fig. 14 is a flowchart showing an information processing method executed by an information processing device according to the seventh embodiment.

[0090] The information processing device according to the seventh embodiment is the same as the information processing device 80A shown in FIG. 9, and therefore a description thereof will be omitted.

[0091] The processes from step S110 to step S116 are the same as the processes from step S50 to step S56 shown in FIG. 11, respectively, and therefore will not be described.

[0092] If the answer is Yes in step S116, the speech detection unit 88 determines whether or not the speech of the participant in the detected participant information is less than a predetermined amount while the speaker is speaking (step S118). Specifically, the speech detection unit 88 determines whether or not the number of times or the duration of speech of the participant in the detected participant information is less than a predetermined amount while the speaker is speaking. If it is determined that the speech of the participant in the detected participant information is less than a predetermined amount while the speaker is speaking (step S118; Yes), the process proceeds to step S120. If it is not determined that the speech of the participant in the detected participant information is less than a predetermined amount while the speaker is speaking (step S118; No), the process proceeds to step S122.

[0093] If the answer is Yes in step S118, the display control unit 82 increases the display priority of the participant information of the participant who is seen by the speaker for a predetermined time or more while the speaker is speaking and who speaks less than a predetermined amount (step S120). The method of increasing the display priority is the same as step S16 shown in FIG. 5, so the explanation is omitted. Then, the process proceeds to step S122.

[0094] The processes in steps S122 and S124 are the same as those in steps S60 and S62 shown in FIG. 11, respectively, and therefore will not be described.

[0095] As described above, in the seventh embodiment, in a Web conference in which multiple participants participate, the participant information of a user who is seen by a speaker for a predetermined time or more while the speaker is speaking and who speaks less than a predetermined amount can be displayed in a prominent position. As a result, the seventh embodiment can provide an appropriate opportunity to a participant who is expected to speak by the speaker while the speaker is speaking but who does not speak much. In addition, in the seventh embodiment, the speaker can direct his / her gaze toward the participant information to display the participant information in a prominent position, making it easier for the speaker to encourage a specific participant to speak.

[0096] Each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various additions and usage conditions. This distribution and integration configuration may be performed dynamically.

[0097] Although the embodiments of the present invention have been described above, the present invention is not limited to the contents of these embodiments. The above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-described components can be appropriately combined. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the above-described embodiments. [Explanation of symbols]

[0098] 1. Information Processing Systems 10 Information terminal equipment 12 Server equipment 20 Camera 22 Microphone 24 Control section 26,60 Communications Department 28 Display section 30 Terminal control device 40 Shooting control section 42,72 Communication control section 44 Voice Acquisition Unit 46 Display control section 48 Operation control section 50 Application control section 52,74 Storage part 54 Gaze detection unit 62 Server control device 70 Connection control section 80, 80A Information processing equipment 82 Display control section 84 Gaze detection unit 86,86A Specific part 88 Speech detection unit

Claims

1. a display control unit that displays participant information indicating a plurality of participants participating in the web conference on a display screen of an information terminal device used by the plurality of participants; a gaze detection unit that detects the gaze of the participant using the information terminal device toward a display screen that displays the participant information; an identification unit that identifies participant information that the gazes of the plurality of participants are directed to for a predetermined period of time or more from among the participant information displayed on the display screens used by the plurality of participants; Equipped with the display control unit causes the participant information identified by the identification unit to be displayed on the display screen with priority over other participant information. Information processing device.

2. the identifying unit identifies participant information to which a predetermined number or more of the participants among the plurality of participants have directed their gaze for a predetermined period of time or more. The information processing device according to claim 1 .

3. the identification unit identifies participant information for which the cumulative total of the gazes of the plurality of participants is directed is the longest. The information processing device according to claim 1 .

4. A display control unit that displays participant information indicating multiple participants participating in a web conference on the display screen of an information terminal device used by the multiple participants; an utterance detection unit that identifies a speaker who is speaking among the plurality of participants; a gaze detection unit that detects the speaker's gaze toward a display screen that displays the participant information; an identification unit that identifies participant information, to which the speaker's gaze is directed for a predetermined period of time or more, from the participant information displayed on the display screen of the information terminal device used by the speaker while the speaker is speaking; Equipped with the display control unit causes the participant information identified by the identification unit to be displayed on the display screen with priority over other participant information. Information processing device.

5. The identification unit identifies participant information to which the speaker's gaze is directed for the longest period of time while the speaker is speaking. The information processing device according to claim 4 .

6. The speech detection unit detects a speech state of the speaker, the identifying unit identifies participant information to which the speaker's gaze is directed for a predetermined period of time or longer when a break in the speaker's speech is detected. The information processing device according to claim 4 .

7. displaying participant information indicating a plurality of participants participating in the web conference on display screens of information terminal devices used by the plurality of participants; detecting a line of sight of the participant using the information terminal device with respect to a display screen displaying the participant information; identifying participant information, to which the gazes of the plurality of participants are directed for a predetermined period of time or more, from among the participant information displayed on the display screens used by the plurality of participants; displaying the identified participant information on the display screen in preference to other participant information; A program that causes an information processing device to execute the above.

8. A step of displaying participant information indicating multiple participants participating in the web conference on a display screen of an information terminal device used by the multiple participants; identifying a speaker who is speaking from among the plurality of participants; detecting a line of sight of the speaker with respect to a display screen displaying the participant information; identifying participant information, to which the speaker's gaze is directed for a predetermined period of time or more, from among the participant information displayed on the display screen of the information terminal device used by the speaker while the speaker is speaking; displaying the identified participant information on the display screen in preference to other participant information; A program that causes an information processing device to execute the above.

Images