Information processing device and information processing method

Abstract

To facilitate smooth meetings in web conferences and web seminars by appropriately controlling the muting of participants' audio. [Solution] The information processing device 80 includes a mute control unit 84 that, when it detects an unmuting operation by a participant whose voice is muted, unmutes the voice of the participant whose voice unmuting operation was detected on the information terminal device used by the speaker or facilitator among the multiple participants, and, based on a predetermined input from the speaker or facilitator after unmuting the voice on the information terminal device used by the speaker or facilitator, unmutes the voice of the remaining participants whose voice unmuting operation was detected on the information terminal devices of the remaining participants.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and an information processing method.

Background Art

[0002] In a web conference held online, a technique is disclosed in which a speaking right is granted to a participant terminal that has requested to speak, and an instruction to switch to a state where the participant can speak in an application for the web conference is output to the participant terminal to which the speaking right has been granted (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In web conferences and web seminars, especially when a large number of people participate, participants other than the speaker often mute their voices. Depending on the timing of解除 muting when a participant speaks, it may cause an abrupt interruption to the speaker's speech or a voice collision, which may interfere with the progress of the meeting.

[0005] The present disclosure aims to appropriately control the muting of participants' voices in web conferences, web seminars, etc., and assist in smooth meeting progress.

Means for Solving the Problems

[0006] To solve the above-mentioned problems and achieve the objectives, the information processing device according to this disclosure is an information processing device that processes information in an online meeting or seminar in which multiple participants participate using information terminal devices, and includes a mute control unit that, when it detects an operation to unmute the audio of a participant whose audio is muted, unmutes the audio of the participant whose audio unmuting operation was detected on the information terminal device used by the speaker or facilitator among the multiple participants, and, based on a predetermined input from the speaker or facilitator after unmuting the audio on the information terminal device used by the speaker or facilitator, unmutes the audio of the participants whose audio unmuting operation was detected on the information terminal devices of the remaining participants.

[0007] The information processing method relating to this disclosure is an information processing method performed by an information processing device that performs information processing in an online meeting or seminar in which multiple participants participate using information terminal devices, wherein when an operation to unmute the audio by a participant whose audio is muted is detected, the information processing device performs the following actions: unmutes the audio of the participant whose audio unmuting operation was detected on the information terminal device used by the speaker or facilitator among the multiple participants; and, based on a predetermined input from the speaker or facilitator after unmuting the audio on the information terminal device used by the speaker or facilitator, unmutes the audio of the participants whose audio unmuting operation was detected on the information terminal devices of the remaining participants. [Effects of the Invention]

[0008] According to this disclosure, it is possible to appropriately control the muting of participants' audio during web conferences and web seminars, thereby assisting in the smooth progress of meetings. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a schematic diagram showing an example configuration of an information processing system according to the first embodiment. [Figure 2] Figure 2 is a block diagram showing an example configuration of an information terminal according to the first embodiment. [Figure 3] Figure 3 is a block diagram showing an example configuration of a server device according to the first embodiment. [Figure 4] Figure 4 is a block diagram showing an example configuration of an information processing device according to the first embodiment. [Figure 5] Figure 5 is a flowchart showing the information processing method according to the first embodiment. [Modes for carrying out the invention]

[0010] Embodiments of the information processing apparatus and information processing method according to the present invention will be described in detail below with reference to the attached drawings. However, the present invention is not limited to the following embodiments.

[0011] [First Embodiment] (Information Processing System) An example of the configuration of the information processing system according to the first embodiment will be described using Figure 1. Figure 1 is a schematic diagram showing an example of the configuration of the information processing system according to the first embodiment.

[0012] Information processing system 1 includes multiple information terminal devices 10 and a server device 30. Information processing system 1 is used in online meetings (hereinafter referred to as "web conferences") or web seminars with multiple participants.

[0013] In web conferences or web seminars, participants other than the speaker or facilitator often mute their microphones. The microphone muting operation is performed using the participant's information terminal device 10.

[0014] The speaker or facilitator, for example in a web conference, would be the participant currently speaking or the participant acting as the meeting facilitator.

[0015] The speaker or facilitator is, for example, in a web seminar, the seminar presenter or the seminar facilitator, etc.

[0016] In the following description, the case where the information processing system 1 is used for a web conference will be described as an example.

[0017] (Information terminal) Using FIG. 2, a configuration example of the information terminal according to the first embodiment will be described. FIG. 2 is a block diagram showing a configuration example of the information terminal according to the first embodiment. Each information terminal device 10 starts a web conference application and conducts a web conference or web seminar, etc. with other information terminal devices 10 via the server device 30. The information terminal device 10 is, for example, an information terminal such as a desktop or notebook personal computer, a smartphone, or a tablet terminal used when participating in an online web conference or web seminar.

[0018] The information terminal device 10 includes a camera 11, a microphone 12, an operation unit 13, a communication unit 18, a display unit 19, and a terminal control device 20.

[0019] The camera 11 photographs the participants facing the information terminal device 10. The camera 11 photographs the participants using the information terminal device 10. The camera 11 is, for example, an in-camera provided on the display unit 19 side. The camera 11 may also be an external camera connected to the information terminal device 10. The camera 11 outputs the photographed video to the photographing control unit 21 of the terminal control device 20.

[0020] The microphone 12 picks up the voices spoken by the participants using the information terminal device 10. The microphone 12 is, for example, a built-in microphone arranged in the information terminal device 10. The microphone 12 may also be an external microphone connected to the information terminal device 10. The microphone 12 outputs the voice related to the picked-up voice to the voice acquisition unit 23 of the terminal control device 20.

[0021] The operation unit 13 is an input device capable of receiving various operations for the information terminal device 10. The operation unit 13 outputs an operation signal to the terminal control device 20 corresponding to the received input operation. The operation unit 13 includes, for example, a keyboard, mouse, touch panel arranged on top of the display unit 19, buttons, switches, etc. The operation unit 13 can receive, for example, key input operations, click operations, or touch operations.

[0022] In this embodiment, the operation unit 13 receives muting and unmuting operations for the audio of participants using the information terminal device 10 in the web conferencing application.

[0023] In this embodiment, the operation unit 13 receives requests in the web conferencing application to allow or deny the unmuting of the audio of a speaker or facilitator using the information terminal device 10.

[0024] The communication unit 18 is a communication unit for wired or wireless communication. The communication unit 18 communicates using methods such as Wi-Fi (registered trademark) or mobile phone lines, and connects to an intranet or the internet. The communication unit 18's communication is controlled by the communication control unit 22 of the terminal control device 20. The communication unit 18 communicates with the server device 30. More specifically, the communication unit 18 transmits the participant's voice picked up by the microphone 12, the participant's video captured by the camera 11, mute and unmute operations received via the operation unit 13, and unmute permission and voice unmute denial operations to the server device 30. The communication unit 18 receives video of screen-shared materials, video and audio of other participants from the server device 30.

[0025] The display unit 19 displays various types of images, including text and pictures. The display unit 19 is a display including, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 19 displays images of participants and other participants, as well as screen-shared materials.

[0026] (Terminal control device) The terminal control device 20 controls each part of the information terminal device 10. The terminal control device 20 includes, for example, a processing unit such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), and a storage device such as RAM (Random Access Memory) or ROM (Read Only Memory). The terminal control device 20 executes a program that controls the operation of the information terminal device 10 according to this disclosure. The terminal control device 20 may be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The terminal control device 20 loads the stored program into memory and executes the instructions contained in the program. The terminal control device 20 includes internal memory such as the aforementioned RAM, and the internal memory is used for temporary storage of data in the terminal control device 20. The terminal control device 20 may be implemented by a combination of hardware and software.

[0027] The terminal control device 20 includes, as functional blocks realized by hardware and the programs it executes, an image capture control unit 21, a communication control unit 22, an audio acquisition unit 23, a display control unit 24, an operation control unit 25, an application control unit 26, and a storage unit 29.

[0028] The shooting control unit 21 controls the camera 11 to capture video. The shooting control unit 21 acquires the video captured by the camera 11. Specifically, the shooting control unit 21 acquires video captured by the camera 11 of participants using the information terminal device 10. The video acquired by the camera 11 is a moving image.

[0029] The communication control unit 22 controls the communication unit 18 to perform communication between the information terminal device 10 and external devices. Specifically, the communication control unit 22 controls, for example, the communication unit 18 to control communication between the information terminal device 10 and the server device 30. The communication control unit 22 controls, for example, the communication unit 18 to send and receive information regarding web conferences with other information terminal devices 10 to and from the server device 30. More specifically, the communication control unit 22 controls the transmission of participant audio picked up by the microphone 12, participant video captured by the camera 11, mute and unmute operations received via the operation unit 13, and unmute permission and audio unmute denial operations to the server device 30 via the communication unit 18. The communication control unit 22 controls the transmission of video of screen-shared materials, video and audio of other participants from the server device 30 via the communication unit 18.

[0030] The voice acquisition unit 23 acquires audio related to the voice detected by the microphone 12. The voice acquisition unit 23 also acquires audio related to the speech of participants using the information terminal device 10.

[0031] The display control unit 24 controls the display of video on the display unit 19. For example, the display control unit 24 controls the display unit 19 to display information (hereinafter referred to as the web conferencing screen) that has been executed using a web conferencing application. The web conferencing screen may be generated by the display control unit 24 and displayed on the display unit 19 by the display control unit 24. Alternatively, the web conferencing screen may be generated by the server device 30, received as video, and displayed on the display unit 19 by the display control unit 24.

[0032] The operation control unit 25 acquires operation signals from the operation unit 13 in response to operations performed on the operation unit 13. The operation control unit 25 outputs control signals corresponding to the acquired operation signals to each unit.

[0033] The application control unit 26 controls the operation of various applications running on the information terminal device 10. For example, the application control unit 26 controls the operation of applications used for web conferencing, such as web conferencing tools. The application control unit 26 may also operate applications such as web conferencing tools on the server device 30 using the application control unit 42, which will be described later. The application control unit 26 may also operate applications such as web conferencing tools in a distributed manner by having the information terminal device 10 and the server device 30 cooperate. In other words, the application control unit 26 may be a functional block implemented only on the information terminal device 10, a functional block implemented only on the server device 30, or a functional block implemented through the cooperation of the information terminal device 10 and the server device 30.

[0034] The memory unit 29 stores information such as the calculation contents and programs of the terminal control device 20. The memory unit 29 is composed of, for example, RAM, a main memory such as ROM, and a storage device such as an SSD (Solid State Drive).

[0035] (Server device) An example of the configuration of a server device according to the first embodiment will be described using Figure 3. Figure 3 is a block diagram showing an example of the configuration of a server device according to the first embodiment. The server device 30 is, for example, a server of a company that provides a web conferencing tool. The server device 30 comprises a communication unit 31 and a server control device 40.

[0036] The communication unit 31 is a communication unit for wired or wireless communication. The communication unit 31 communicates using methods such as Wi-Fi or mobile phone lines and connects to an intranet or the internet. The communication unit 31's communication is controlled by the communication control unit 41 of the server control device 40. The communication unit 31 communicates with the information terminal device 10. More specifically, the communication unit 31 receives video and audio from the information terminal device 10, mute and unmute operations received by the information terminal device 10, and unmute permission operations and audio unmute denial operations. The communication unit 31 transmits video of screen-shared materials, video and audio of other participants to the information terminal device 10.

[0037] (Server control unit) The server control device 40 controls various parts of the server device 30. The server control device 40 includes, for example, an information processing device such as a CPU or MPU, and a storage device such as RAM or ROM. The server control device 40 may be implemented by an integrated circuit such as an ASIC or FPGA. The server control device 40 may be implemented by a combination of hardware and software. The server control device 40 executes a program that controls the operation of the server device 30 according to this disclosure.

[0038] The server control device 40 comprises a communication control unit 41, an application control unit 42, and a storage unit 49 as functional blocks realized by circuits and programs.

[0039] The communication control unit 41 controls the communication unit 31 to perform communication between the server device 30 and the information terminal device 10. Specifically, the communication control unit 41 controls, for example, the communication unit 31 to control communication between the server device 30 and the information terminal device 10. The communication control unit 41 controls, for example, the communication unit 31 to receive video and audio of participants, mute and unmute operations received by the information terminal device 10, and unmute permission operations and audio unmute denial operations from the information terminal device 10. The communication control unit 41 controls, for example, the communication unit 31 to transmit video of the shared screen materials, video and audio of participants to the information terminal device 10.

[0040] The application control unit 42 controls the operation of various applications running on the server device 30. For example, the application control unit 42 controls the operation of applications used for web conferencing, such as web conferencing tools.

[0041] The memory unit 49 stores, for example, the calculation contents of the application control unit 42 and the communication control unit 41, as well as information such as programs. The memory unit 49 is composed of, for example, RAM and a main memory device such as ROM, an SSD, an HDD (Hard Disk Drive), or other storage devices.

[0042] (Information processing device) An example of the configuration of the information processing device according to the first embodiment will be described using Figure 4. Figure 4 is a block diagram showing an example of the configuration of the information processing device according to the first embodiment.

[0043] As shown in Figure 4, the information processing device 80 processes information in a web conference or web seminar in which multiple participants participate using information terminal devices 10. The information processing device 80 comprises a display control unit 82 and a mute control unit 84. The information processing device 80 is a conceptual device realized through the cooperation of either or both of the terminal control devices 20 in multiple information terminal devices 10 connected to the server control device 40 and the server device 30 and used for web conferences, etc. Therefore, either the information terminal device 10 or the server device 30 may be referred to as the information processing device 80, or the information processing system 1 consisting of the information terminal device 10 and the server device 30 may be referred to as the information processing device 80. Each part of the information processing device 80 is realized by either or both of the terminal control devices 20 and the server control devices 40.

[0044] The display control unit 82 causes the display units 19 of multiple information terminal devices 10 to display images showing various information, such as web conference screens. The display control unit 82 may also display shared materials to multiple information terminal devices 10. Such a function of the display control unit 82 is implemented as follows: For example, if shared materials are displayed on the display unit 19 by the display control unit 24 of the information terminal device 10 used by the speaker, the server device 30 acquires the shared materials as display content from the information terminal device 10 used by the speaker via the communication unit 31. The server device 30 then causes the display control units 24 of other information terminal devices 10, other than the information terminal device 10 used by the speaker, to display them on the display unit 19.

[0045] When the mute control unit 84 detects that a participant whose voice is muted has unmuted their voice, it unmutes the voice of the participant whose voice unmuting operation was detected on the information terminal device 10 used by the speaker or facilitator among the multiple participants. Furthermore, based on a predetermined input from the speaker or facilitator after unmuting their voice on the information terminal device 10 used by the speaker or facilitator, the mute control unit 84 unmutes the voice of the remaining participants whose voice unmuting operation was detected on the information terminal device 10.

[0046] The "audio muted" state refers to a state in a web conferencing application where a participant's audio is muted. The audio muted state may also be determined, for example, when a participant's audio mute operation is detected on their information terminal device 10.

[0047] When the speaker or moderator detects that a participant has unmuted their audio on the information terminal device 10 used by the speaker or moderator, the state in which the participant's audio is unmuted is referred to as the "partially unmuted audio" state.

[0048] In a state where audio is partially unmuted, only the speaker or facilitator can hear the audio of the participant whose unmuting attempt was detected. Other participants remain unable to hear the audio of the participant whose unmuting attempt was detected. In this state, the participant whose unmuting attempt was detected can only inform the speaker or facilitator that they have a question or wish to speak.

[0049] The state in which the audio of participants who have detected an audio unmute operation has been unmuted on the remaining participants' information terminal devices 10 is referred to as the "total audio unmuted state."

[0050] When all audio mutes are unmuted, all participants can hear the audio of any participant whose audio has been unmuted. In this state, the participant whose audio has been unmuted can ask questions or speak.

[0051] The mute control unit 84 may notify the information terminal device 10 used by the participant that the audio is partially unmuted or completely unmuted by changing the display of the icon image of the operation button that accepts the operation to unmute the audio.

[0052] An example of an icon image for an operation button that accepts an unmute command for audio is a microphone icon.

[0053] If the audio is not fully muted after partially unmuting it, the mute control unit 84 may notify the participant's information terminal device 10, which detected the audio unmuting operation, that the audio will not be fully muted, by displaying an icon image of an operation button that accepts audio unmuting operations.

[0054] The predetermined input is based on at least one of the following: whether or not an operation to allow unmuting audio has been performed, whether or not audio input containing a predetermined keyword has been entered, whether or not audio input has been entered within a predetermined time period, and whether or not a predetermined action (gesture) has been performed. The predetermined input is capable of determining whether or not the timing is such that transitioning from a partially unmuted state to a fully unmuted state of audio will not disrupt the progress of the meeting. The predetermined input is capable of determining whether or not the timing is such that outputting the audio of a participant whose unmuting operation has been detected from the information terminal device 10 of the remaining participants will not disrupt the progress of the meeting. The predetermined input can also be rephrased as a predetermined operation, predetermined audio input, or predetermined action by the speaker or facilitator.

[0055] If the mute control unit 84 detects, for example, an operation to unmute audio on the information terminal device 10 used by the speaker or facilitator, it may unmute the audio of the participant whose audio unmuting operation was detected on the information terminal device 10 of the remaining participants.

[0056] If the mute control unit 84 receives audio from a speaker or facilitator containing predetermined keywords such as "Please ask a question" or "XX, please," it may unmute the audio of the participant whose unmuting operation has been detected for the remaining participants' information terminal devices 10.

[0057] If the mute control unit 84 detects, for example, that no voice input has been received from the speaker or facilitator for a predetermined period of several seconds, it may unmute the voices of the remaining participants' information terminal devices 10 if an unmuting operation has been detected.

[0058] The mute control unit 84 may, for example, unmute the audio of the participant whose audio unmuting operation was detected on the remaining participants' information terminal devices 10 when a predetermined action is performed by the speaker or facilitator.

[0059] The mute control unit 84 may, for example, refrain from unmuting the audio on the information terminal devices 10 of the remaining participants if it detects an operation that does not permit the unmuting of audio on the information terminal device 10 used by the speaker or facilitator.

[0060] The mute control unit 84 may not unmute the audio for the remaining participants' information terminal devices 10 if the audio input from the speaker or facilitator does not contain a predetermined keyword for unmuting the audio, such as "Please ask a question" or "Mr. / Ms. XX, please." The mute control unit 84 may also not unmute the audio for the remaining participants' information terminal devices 10 if the audio input from the speaker or facilitator contains a predetermined keyword that does not permit unmuting the audio, such as "We will address this separately" or "We have received your feedback."

[0061] The mute control unit 84 may, for example, not unmute the audio for the remaining participants' information terminal devices 10 if it detects that the speaker's or facilitator's audio input has continued for a predetermined period of time, such as several seconds.

[0062] If the mute control unit 84 detects that multiple participants have unmuted their audio within a predetermined period of a few seconds, it may partially unmute the audio in the order in which the unmuting operations were detected. More specifically, if the mute control unit 84 detects that multiple participants have unmuted their audio within a predetermined period of a few seconds, it may unmute the audio of the participant whose unmuting operation was detected first on the information terminal device 10 used by the speaker or facilitator. Furthermore, if the mute control unit 84 does not unmute the audio of the participant whose unmuting operation was detected first on the information terminal device 10 of the remaining participants, it may unmute the audio of the participants on the information terminal device 10 used by the speaker or facilitator in the order in which the unmuting operations were detected.

[0063] (Information processing methods) The information processing method according to the first embodiment will be explained using Figure 5. Figure 5 is a flowchart of the information processing method according to the first embodiment. The process shown in Figure 5 is initiated when a web conference is started. When a web conference is started, the information processing device 80, via the display control unit 82, starts displaying the web conference screen on each information terminal device 10. In Figure 5, "information terminal device" is abbreviated as "terminal". In the following explanation, "terminal" in Figure 5 will be referred to as "information terminal device". Figure 5 illustrates an information processing method in a web conference as an example.

[0064] As a prerequisite for the process shown in Figure 5, multiple individuals are participating in the web conference using information terminal devices 10, and among these individuals, the speaker or facilitator has been identified. In this case, the speaker is the moderator or primary speaker in the web conference, and the facilitator is the moderator or the person who convened the web conference, while everyone other than the speaker or facilitator is a participant. Furthermore, the process shown in Figure 5 is repeatedly executed for the duration of the web conference, or for the duration of the web conference as long as the above prerequisite conditions are met.

[0065] The information processing device 80 determines whether or not it has detected an unmute operation on the participant's information terminal device 10 (step S101). More specifically, the information processing device 80 determines, with the mute control unit 84, whether or not it has detected an unmute operation on the information terminal device 10 used by the participant. If the information processing device 80 determines that it has detected an unmute operation (Yes in step S101), it proceeds to step S102. If the information processing device 80 does not determine that it has detected an unmute operation (No in step S101), it repeats the process in step S101.

[0066] If the information processing device 80 determines that an unmute operation has been detected (Yes in step S101), it unmutes the audio from the participant's information terminal device 10 on the speaker's or facilitator's information terminal device 10 (step S102). More specifically, the information processing device 80, using the mute control unit 84, unmutes the audio of the participant whose unmuting operation has been detected on the information terminal device 10 used by the speaker or facilitator among the multiple participants. The information processing device 80 then proceeds to step S103.

[0067] The information processing device 80 determines whether or not to unmute all information terminal devices 10 (step S103). More specifically, the information processing device 80, using the mute control unit 84, determines whether or not to unmute the audio of the remaining participants' information terminal devices 10 based on a predetermined input from the speaker or facilitator when the audio is partially unmuted. If the information processing device 80 determines to unmute (Yes in step S103), it proceeds to step S104. If the information processing device 80 does not determine to unmute (No in step S103), it proceeds to step S105.

[0068] If it is determined that the mute should be unmuted (Yes in step S103), the information processing device 80 unmutes the audio from the participant's terminal on all terminals (step S104). More specifically, the information processing device 80, via the mute control unit 84, unmutes the audio of the participant whose audio unmuting operation has been detected on the remaining participants' information terminal devices 10. The information processing device 80 then proceeds to step S106.

[0069] If it is not determined that the mute should be unmuted (No in step S103), the information processing device 80 mutes the audio from the participant's terminal on the speaker's or facilitator's terminal (step S105). More specifically, the information processing device 80 does not unmute the audio for the remaining participants' information terminal devices 10 using the mute control unit 84. The information processing device 80 proceeds to step S106.

[0070] The information processing device 80 determines whether or not it has detected a mute operation on the participant's information terminal device 10 (step S106). More specifically, the information processing device 80 determines, using the mute control unit 84, whether or not it has detected an audio mute operation on the information terminal device 10 used by the participant. If the information processing device 80 determines that it has detected a mute operation (Yes in step S106), it terminates the process in this flowchart. If the information processing device 80 does not determine that it has detected a mute operation (No in step S106), it repeats the process in step S106.

[0071] In the case of a web conference, in step S102, the audio from the participant's information terminal that performed the unmute operation first is unmuted on the information terminal device 10 used by the current speaker (partially unmuted audio state). Then, in step S105, based on the speaker's predetermined input regarding the first participant's statement, it is determined whether or not to unmute the audio for all participants, including the remaining participants.

[0072] In a web conference, the speaker is not fixed; the person speaking at that moment becomes the speaker. For example, if the information processing device 80 detects that a speaker's utterance has continued for a predetermined period of time, such as 3 minutes, the speaker may be changed to the person speaking at that time. For example, if the information processing device 80 constantly recognizes from the content of participants' statements and detects that the topic has changed, the speaker may be changed. For example, if the information processing device 80 detects that a speaker or facilitator has authorized a change of speaker while some participants are unmuted, the speaker may be changed.

[0073] For example, let's explain the information processing method in a web seminar. In step S102, the speaker or facilitator unmutes the audio from the information terminal of the participant who performed the unmuting operation first (partially unmuted audio state). Then, in step S105, based on the predetermined input from the speaker or facilitator regarding the first participant's statement, it is determined whether or not to unmute the audio for all participants, including the remaining participants.

[0074] (effect) As described above, in this embodiment, when a participant unmutes their audio, the audio is partially unmuted on the information terminal device 10 used by the speaker or facilitator. In this embodiment, based on a predetermined input from the speaker or facilitator in the partially unmuted state, the audio can be fully unmuted, including the information terminal devices 10 of the remaining participants. This embodiment allows for the audio to be fully unmuted, including the information terminal devices 10 of the remaining participants, regardless of the timing of the participant's unmuting operation, by partially unmuting the audio, for example, when it does not interfere with the progress of the meeting. In this way, according to this embodiment, the muting of participants' audio can be appropriately controlled in web conferences and web seminars, etc., to assist in the smooth progress of meetings.

[0075] In this embodiment, if multiple participants are detected to have unmuted their audio, the audio of the participant whose unmuting operation was detected first will be partially unmuted. In this embodiment, if the audio does not transition from partially unmuted to fully unmuted, the audio can be partially unmuted in the order in which the unmuting operations were detected.

[0076] In this embodiment, the predetermined input is based on at least one of the following: whether or not an operation to allow unmuting the audio has been performed, whether or not an audio input containing a predetermined keyword has been performed, and whether or not an audio input has been performed within a predetermined time. In this embodiment, it is possible to appropriately determine whether or not to transition from a partially unmuted state to a fully unmuted state.

[0077] Each component of the illustrated information processing system is a functional concept and does not necessarily have to be physically configured as shown. In other words, the specific form of each device is not limited to that shown in the illustration, and all or part of them may be functionally or physically distributed or integrated in any unit depending on the processing load and usage of each device.

[0078] The configuration of each device in an information processing system is realized, for example, as software, such as a program loaded into memory. In the above embodiment, these were described as functional blocks realized through the cooperation of hardware or software. That is, these functional blocks can be realized in various forms using hardware alone, software alone, or a combination thereof.

[0079] The components described above include those that are easily conceivable by those skilled in the art, and those that are substantially identical. Furthermore, the components described above can be combined as appropriate. In addition, various omissions, substitutions, or modifications of the components are possible without departing from the spirit of the present invention. [Explanation of Symbols]

[0080] 1. Information Processing System 10 Information terminal device 11 Cameras 12 microphones 13 Control section 18 Communications Department 19 Display section 20 Terminal control device 21. Image capture control unit 22 Communication Control Unit 23. Voice acquisition unit 24 Display Control Unit 25 Operation Control Unit 26 Application Control Unit 29 Memory section 30 Server Devices 31 Communications Department 40 Server Control Units 41 Communication Control Unit 42 Application Control Unit 49 Memory section 80 Information Processing Device 82 Display Control Unit 84 Mute Control Unit

Claims

1. An information processing device for processing information in an online meeting or seminar in which multiple participants participate using information terminal devices, If a participant whose audio is muted attempts to unmute their audio, For the information terminal device used by the speaker or facilitator among the multiple participants, the system will unmute the audio of the participant whose audio unmuting operation has been detected. Based on a predetermined input from the speaker or facilitator after the speaker or facilitator has unmuted their audio on the information terminal device used by the speaker or facilitator, a mute control unit unmutes the audio of the remaining participants' information terminal devices for which the unmuting operation has been detected. An information processing device equipped with the following features.

2. The mute control unit, If multiple participants are detected unmuting their audio within a specified period, The information terminal device used by the speaker or the facilitator will unmute the audio of the participant whose audio unmuting operation is detected first. If the audio unmuting operation of the remaining participants on the information terminal device is not performed first, the audio of the participant whose audio unmuting operation is detected will be unmuted on the information terminal device used by the speaker or the facilitator in the order in which the audio unmuting operations are detected. The information processing apparatus according to claim 1.

3. The predetermined input is based on at least one of the following: whether or not an operation to allow unmuting the audio is performed, whether or not an audio input containing a predetermined keyword is performed, and whether or not an audio input is performed within a predetermined time period. The information processing apparatus according to claim 1 or 2.

4. An information processing method performed by an information processing device that processes information in an online meeting or seminar in which multiple participants participate using information terminal devices, If a participant who is muted attempts to unmute their audio, To unmute the audio of the participant whose audio unmuting operation has been detected on the information terminal device used by the speaker or facilitator among the multiple participants, Based on a predetermined input from the speaker or facilitator after unmuting the audio on the information terminal device used by the speaker or facilitator, the remaining participants' information terminal devices will be configured to unmute the audio of the participants for whom the audio unmuting operation was detected. An information processing method performed by an information processing device.

Images