Information processing system, information processing device, information processing method and voice input control program

Abstract

To provide an information processing system which can prevent erroneous unmuting by utterance unrelated to a conference and utterance of a person other than an attendant of the conference.SOLUTION: An information processing system 1 includes an information processing device 100 and a plurality of terminal devices 200. Each terminal device 200 includes: a voice input unit 250; a first communication unit 211d; a voice data control unit 211e; a mute state detection unit 211f; and a mute state control unit 211g. The information processing device 100 includes: a second communication unit 140; an intention analysis processing unit 113; a speaker determination unit 114; and a sound volume determination unit 115. The second communication unit 140 transmits a first determination result of the intention analysis processing unit 113 and a second determination result of the speaker determination unit 114 to the terminal device 200. The mute state control unit 211g performs unmuting in a first case in which the first determination result indicates that voice S has an intention that a conference is a target and the second determination result indicates that the number of speakers Ps is one.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system, an information processing device, an information processing method, and a voice input control program. [Background technology]

[0002] In a conference system for holding a conference using the communication terminals described in Patent Document 1, the mute state is released when the strength of the input audio is equal to or greater than a first threshold and the strength of the audio indicated by the audio data transmitted from another communication terminal is equal to or less than a second threshold, or when the input audio contains a predetermined word that indicates an intention to speak. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-013073 Summary of the Invention [Problem to be solved by the invention]

[0004] In a conventional conference system where a conference (sometimes called an "electronic conference") is held using communication terminals, the microphones of each participant may be muted in advance. The muting is automatically released when there is voice input into the microphone, or when there is voice input but no other participants are speaking. Therefore, there are cases where the muting is released by mistake even when a comment unrelated to the conference or a loud external noise is heard. Furthermore, if the input voice contains certain words that indicate the intention to speak, the muting may be released by mistake even when the voice is from someone other than a conference participant.

[0005] The present disclosure aims to provide an information processing system, an information processing device, an information processing method, and a voice input control program for a terminal device that can prevent mute from being accidentally released due to comments unrelated to the meeting or comments made by people other than the meeting participants. [Means for solving the problem]

[0006] An information processing system according to a first aspect of the present disclosure includes a plurality of terminal devices and an information processing device. The terminal device includes a voice input unit, a first communication unit, a voice data control unit, a mute state detection unit, and a mute state control unit. Voices of conference participants are input to the voice input unit. The first communication unit communicates with the information processing device. The voice data control unit transmits voice data corresponding to the voices input to the voice input unit to the information processing device. The mute state detection unit detects whether a running conference application is in a mute state. The mute state control unit controls switching of the mute state of the conference application. The information processing device includes a second communication unit, an intention analysis processing unit, and a speaker determination unit. The second communication unit communicates with the first communication unit of the terminal device. The intention analysis processing unit analyzes the voice data received by the second communication unit from the first communication unit of the terminal device in the mute state, and determines whether the voice has an intention to target the conference. The speaker determination unit determines the number of speakers of the voice data. The second communication unit transmits the first determination result of the intention analysis processing unit and the second determination result of the speaker determination unit to the terminal device. The mute state control unit cancels the mute state in a first case where the first determination result indicates that the voice has an intention to target the conference and the second determination result indicates that the number of speakers is one.

[0007] An information processing device according to a second aspect of the present disclosure includes a second communication unit, an intention analysis processing unit, and a speaker determination unit. The second communication unit has an audio input unit to which audio of conference participants is input, and receives audio data output corresponding to the audio from a terminal device in which a running conference application is muted. The intention analysis processing unit analyzes the audio data and determines whether the audio has an intention to target the conference. The speaker determination unit determines the number of speakers of the audio data. The second communication unit transmits a first determination result of the intention analysis processing unit and a second determination result of the speaker determination unit to the terminal device. When the first determination result indicates that the audio has an intention to target the conference and the second determination result indicates that the number of speakers is one, the mute state is canceled.

[0008] An information processing method according to a third aspect of the present disclosure includes the steps of: receiving audio data output in response to audio from a terminal device having an audio input unit into which the audio of conference participants is input and in which a running conference application is in a muted state; analyzing the audio data to make a first determination as to whether the audio is intended to be directed to the conference; making a second determination as to the number of speakers of the audio data; transmitting the results of the first determination and the second determination; and canceling the mute state when the result of the first determination indicates that the audio is intended to be directed to the conference and the result of the second determination indicates that the number of speakers is one.

[0009] A fourth aspect of the present disclosure provides a voice input control program that causes a computer to execute voice input control of a terminal device, and includes a step of detecting whether a running conference application is in a muted state, a step of analyzing input voice data corresponding to the voices of the conference participants, and a step of canceling the muted state when it is determined that the voices are intended to be directed at the conference and the number of speakers in the voice data is determined to be one.

[0010] An information processing system according to a fifth aspect of the present disclosure includes a plurality of terminal devices and an information processing device. The terminal devices include an audio input unit, an audio data control unit, a mute state detection unit, and a mute state control unit. The audio input unit receives audio from conference participants. The audio data control unit outputs audio data corresponding to the audio input to the audio input unit. The mute state detection unit detects whether a running conference application is in a mute state. The mute state control unit controls switching of the mute state of the conference application. The information processing device includes an intention analysis processing unit and a speaker determination unit. The intention analysis processing unit analyzes the audio data of the terminal devices in the mute state and determines whether the audio is intended to be directed to the conference. The speaker determination unit determines the number of speakers of the audio data. The mute state control unit cancels the mute state when the intention analysis processing unit determines that the audio is intended to be directed to the conference and the speaker determination unit determines that the number of speakers is one. [Effects of the Invention]

[0011] According to the present disclosure, it is possible to prevent erroneous unmuting due to a comment unrelated to the conference or a comment made by a person other than a conference participant. [Brief explanation of the drawings]

[0012] [Figure 1] 2 is a block diagram showing in detail the configuration of the information processing device 100 and the terminal device 200 of the information processing system 1 according to the first embodiment of the present disclosure. FIG. [Figure 2] 1 is a block diagram showing the connection relationship between the information processing device 100 and each terminal device 200 of the information processing system 1, the schedule management service device 10, and the electronic conference service device 20. FIG. [Figure 3] 1 is a block diagram showing the configuration of a schedule management service device 10. FIG. [Figure 4] FIG. 2 is a block diagram showing the configuration of an electronic conference service device 20. [Figure 5]1 is a flowchart showing the basics of an information processing method by the information processing system 1. [Figure 6] 10 is a flowchart showing an application example of an information processing method by the information processing system 1. [Figure 7] FIG. 10 is an explanatory diagram illustrating an example of a login screen of the conference support application 211. [Figure 8] FIG. 10 is an explanatory diagram illustrating an example of a screen for selecting a desired schedule using a schedule management service. [Figure 9A] FIG. 10 is an explanatory diagram showing a display example when the application microphone 212b of the electronic conference application 212 is enabled. [Figure 9B] FIG. 9B is an enlarged view of the icon of the application microphone 212b in FIG. 9A. [Figure 10A] FIG. 10 is an explanatory diagram showing a display example when the application microphone 212b is in a muted state. [Figure 10B] FIG. 10B is an enlarged view of the icon of the application microphone 212b in FIG. 10A. [Figure 11] FIG. 1 is an explanatory diagram illustrating a situation in which a terminal user U is having a conversation with a person H who is not involved in the conference. [Figure 12] FIG. 10 is an explanatory diagram illustrating another situation in which a terminal user U is having a conversation with a person H who is not involved in the conference. [Figure 13] 10 is a flowchart of a voice input control process in the conference support application 211. DETAILED DESCRIPTION OF THE INVENTION

[0013] Embodiments of the present disclosure will be described with reference to the drawings. In the drawings, the same or corresponding parts are designated by the same reference numerals, and description thereof will not be repeated.

[0014] First Embodiment 1.1 Overall configuration of information processing system 1 First, with reference to FIGS. 1 to 4, the overall configuration of an information processing system 1 capable of holding a conference with multiple participants will be described. The "conference" referred to here may also be referred to as an electronic conference or a web conference. FIG. 1 is a block diagram showing in detail the configuration of an information processing device 100 and a terminal device 200 of the information processing system 1 according to the first embodiment of the present disclosure. FIG. 2 is a block diagram showing the connection relationship between the information processing device 100 and each terminal device 200 of the information processing system 1 and the schedule management service device 10 and the electronic conference service device 20. FIG. 3 is a block diagram showing the configuration of the schedule management service device 10. FIG. 4 is a block diagram showing the configuration of the electronic conference service device 20.

[0015] 1 and 2, the information processing system 1 includes an information processing device 100 and a plurality of terminal devices 200. The information processing device 100 and each terminal device 200 are connected to each other by wire or wirelessly. Note that only one terminal device 200 is shown in FIG. 1. In FIG. 2, one of the plurality of terminal devices 200 is designated as a first terminal device 200-1.

[0016] The information processing device 100 and each terminal device 200 are also connected to the schedule management service device 10 and the electronic conference service device 20 by wire or wirelessly.

[0017] The schedule management service device 10 is a device that provides a schedule management service. Various schedule management services are managed and operated by external companies, but they can also be provided in-house. As shown in FIG. 3, the schedule management service device 10 includes a schedule management unit 11, an authentication unit 12, a communication unit 13, a schedule management database D11, and a user management database D12. The schedule management unit 11 manages the schedule management database D11. The authentication unit 12 manages the user management database D12. Since schedule management services are well known, a detailed description of the schedule management service device 10 will be omitted.

[0018] The electronic conference service device 20 is a device that provides an electronic conference service. Various electronic conference services are managed and operated by external companies, but they can also be provided in-house. As shown in FIG. 4, the electronic conference service device 20 includes an electronic conference management unit 21, an authentication unit 22, a communication unit 23, an electronic conference management database D21, and a user management database D22. The electronic conference management unit 21 manages the electronic conference management database D21. The authentication unit 22 manages the user management database D22. Since electronic conference services are well known, a detailed description of the electronic conference service device 20 will be omitted.

[0019] As shown in FIG. 1, the information processing device 100 is a server device that supports each participant P in using an electronic conference service. The information processing device 100 includes a CPU (Central Processing Unit) 110, a storage unit 120, a display unit 130, and a second communication unit 140. The CPU 110 includes a conference management unit 111, an authentication unit 112, an intention analysis processing unit 113, a speaker determination unit 114, and a volume determination unit 115. The intention analysis processing unit 113 includes a voice recognition unit 113a, an analysis unit 113b, and an intention determination unit 113c. The storage unit 120 stores a conference management database D121, an authentication result management database D122, a voice recognition result database D123, and keywords D124.

[0020] The terminal device 200 is a client device used by each participant P of a conference to use the teleconference service provided by the teleconference service device 20. The terminal device 200 includes a CPU 210, a storage unit 220, a display unit 230, an operation unit 240, an audio input unit 250, a speaker 260, a camera 270, and a first communication unit 290. The CPU 210 executes a conference support application 211 and a teleconference application 212. The conference support application 211 includes a conference control unit 211a, a display control unit 211b, a device control unit 211c, a first communication control unit 211d, an audio data control unit 211e, a mute status detection unit 211f, and a mute status control unit 211g. The teleconference application 212 includes a second communication control unit 212a and an application microphone 212b, which is a microphone for this application. The conference support application 211 communicates with each terminal device 200, the schedule management service device 10, and the electronic conference service device 20 via the first communication unit 290 using the first communication control unit 211d. Similarly, the electronic conference application 212 communicates with each terminal device 200, the schedule management service device 10, and the electronic conference service device 20 via the first communication unit 290 using the second communication control unit 212a. The conference support application 211 and the electronic conference application 212 executed in the CPU 210 can communicate directly with each other.

[0021] 1.2 Main parts of the information processing device 100 and each terminal device 200 The main components of the information processing device 100 and each terminal device 200 will be described first.

[0022] In each terminal device 200, the voice S of a conference participant P is input to the voice input unit 250. The first communication unit 290 communicates with the information processing device 100. The voice data control unit 211e transmits voice data Ds corresponding to the voice S input to the voice input unit 250 to the information processing device 100. The mute state detection unit 211f detects whether the currently running electronic conference application 212 is in a mute state. The mute state control unit 211g controls switching of the mute state of the electronic conference application 212.

[0023] In the information processing device 100, the second communication unit 140 communicates with the first communication unit 290 of the terminal device 200. The intention analysis processing unit 113 analyzes the voice data Ds received by the second communication unit 140 from the first communication unit 290 of the terminal device 200 in a muted state, and determines whether the voice S has an intention to be directed to the conference. The speaker determination unit 114 determines the number of speakers Ps of the voice data Ds.

[0024] The second communication unit 140 transmits the first determination result of the intention analysis processing unit 113 and the second determination result of the speaker determination unit 114 to the terminal device 200. The mute state control unit 211g cancels the mute state in a first case where the first determination result indicates that the voice S has an intention to target the conference and the second determination result indicates that the number of speakers Ps is one.

[0025] Therefore, even if the voice S of a conference participant P is input, the mute state is maintained as it is if the voice S is not intended to be directed to the conference or includes voice S from someone other than the user of the terminal device 200. As a result, it is possible to prevent the mute state from being accidentally released, causing the unintended voice S to be heard by other participants P and disrupting the progress of the conference.

[0026] As a method for the information processing device 100 to recognize whether each terminal device 200 is in a mute state, for example, the conference support application 211 may transmit the detection result of the mute state detection unit 211f to the information processing device 100 when necessary (for example, when the mute state changes). If the received detection result is stored in the storage unit 120 for each terminal device 200, the information processing device 100 can always recognize whether each terminal device 200 is in a mute state. However, the method is not limited to this.

[0027] At least one of the information processing device 100 and the terminal device 200 may further include a volume determination unit 115 that determines whether the volume V corresponding to the audio data Ds is equal to or greater than a predetermined volume Vt. In this case, the mute state control unit 211g does not cancel the mute state even in the first case unless the volume V is equal to or greater than the predetermined volume Vt. Note that FIG. 1 illustrates an example in which the information processing device 100 includes the volume determination unit 115.

[0028] Therefore, even if the voice S of a conference participant P is input, if the volume V is not equal to or greater than the predetermined volume Vt (for example, when the participant P has removed the headset 280), the mute state is maintained as is. As a result, it is further prevented that the mute state is accidentally released and the unintended voice S is heard by other participants P, which would disrupt the progress of the conference.

[0029] The intention analysis processing unit 113 may have, for example, a voice recognition unit 113a that converts voice data Ds into character data Dc, an analysis unit 113b that analyzes the character data Dc and extracts features, and an intention determination unit 113c that determines whether the voice S has an intention to be directed to a meeting based on the extracted features.

[0030] Therefore, based on the characteristics extracted by analyzing the character data Dc converted from the voice data Ds, it is determined whether the voice S is intended to be directed to the conference. As a result, it is further prevented that the mute state is accidentally released and the unintended voice S is heard by other participants P, thereby disrupting the progress of the conference.

[0031] The information processing device 100 may further include a storage unit 120 that stores keywords D124 extracted from the character data Dc and materials used in the meeting. In this case, the intention determination unit 113c determines whether the speech S has an intention to target the meeting, based on the extracted features and the keywords D124 stored in the storage unit 120.

[0032] Therefore, whether the audio S is intended for the conference is determined based on the features extracted by analyzing the character data Dc converted from the audio data Ds and the keywords D124 extracted from the character data Dc and materials used in the conference. As a result, it is possible to prevent the mute state from being erroneously released for audio S that is not intended for the conference.

[0033] The storage unit 120 may store in advance the voice data Ds of the participant P as identification information for identifying the speaker Ps. In this case, the speaker determination unit 114 identifies the speaker Ps based on the identification information and determines whether the speaker Ps includes anyone other than the participant P. Even in the first case, if the speaker Ps includes anyone other than the participant P, the mute state control unit 211g does not cancel the mute state.

[0034] Therefore, in addition to determining whether the voice S is directed to the conference, the speaker Ps is identified based on the voice data Ds of the conference participants P. As a result, if the speakers Ps include people other than the participants P, the mute state is prevented from being erroneously released.

[0035] The terminal device 200 may further include a display unit 230 and a display control unit 211b that controls the display unit 230. It is assumed that the multiple terminal devices 200 include a first terminal device 200-1. In this case, if the volume V corresponding to the audio data Ds received by the second communication unit 140 of the information processing device 100 from the first terminal device 200-1, which is in a muted state, is equal to or greater than a predetermined volume Vt, the second communication unit 140 transmits a notification indicating that audio S has been input to the terminal devices 200 other than the first terminal device 200-1. When the first communication unit 290 receives the notification, the display control unit 211b of the terminal device 200 causes the display unit 230 to display the content of the notification.

[0036] Therefore, even if the voice S of a conference participant P is input but the mute state is not released, the fact that the voice S has been input is displayed on the other terminal devices 200. As a result, the other participants P notice that the voice S has been input, and can subsequently make corrections to the progress of the conference if necessary.

[0037] If the volume V corresponding to the audio data Ds received by the second communication unit 140 of the information processing device 100 from the first terminal device 200-1 in a muted state is equal to or greater than a predetermined volume Vt, the second communication unit 140 may transmit a notification indicating that audio S has been input to the first terminal device 200-1. When the first communication unit 290 receives the notification, the display control unit 211b of the first terminal device 200-1 causes the display unit 230 to display the content of the notification.

[0038] Therefore, if a participant P of the conference inputs voice S but does not disable the voice input, the fact that voice S has been input is also displayed on the participant P's own terminal device 200. As a result, the participant P can manually disable the voice input if necessary, and can subsequently correct the progress of the conference.

[0039] 1.3 Configuration of the information processing device 100 As described above, the information processing device 100 is a server device that supports the use of electronic conference services by each participant P. The following describes configurations and supplementary matters that have not yet been mentioned.

[0040] The CPU 110 controls each unit of the information processing device 100. The CPU 110 may be, for example, a processor or an MPU (Micro Processing Unit), but is not limited to these.

[0041] For example, if an OS (Operating System, sometimes called "basic software") that runs on the CPU 110 and programs for the functions corresponding to the conference manager 111, the authentication unit 112, the intention analysis processor 113, the speaker determiner 114, and the volume determiner 115 are stored in the storage unit 120, the CPU 110 executes the OS and programs to realize the above-mentioned units. Examples of OS include, but are not limited to, Microsoft Windows (registered trademark) and Android (registered trademark). The above-mentioned units may be configured with, but are not limited to, electronic circuits, DSPs (Digital Signal Processors), FPGAs (Field Programmable Gate Arrays), ASICs (Application Specific Integrated Circuits), etc.

[0042] The conference management unit 111 manages conference information such as conference schedules of the schedule management service.

[0043] The authentication unit 112 authenticates a user (hereinafter referred to as "terminal user U") participating in a conference using the terminal device 200. When login is performed using single sign-on (SSO) provided by another company's service, the authentication unit 112 associates the used other company's service with user information. Note that single sign-on is a mechanism that allows a user to log in to multiple services with a single set of ID and password.

[0044] As described above, the intention analysis processing unit 113 analyzes the voice data Ds received by the second communication unit 140 from the first communication unit 290 of the terminal device 200 in a muted state, and determines whether the voice S is intended to be about a conference. The intention analysis processing unit 113 may include, for example, a voice recognition unit 113a that converts the voice data Ds into character data Dc, an analysis unit 113b that analyzes the character data Dc to extract features, and an intention determination unit 113c that determines whether the voice S is intended to be about a conference based on the extracted features. The determination of whether the voice S is intended to be about a conference may be made based on, for example, the extracted features and keywords D124 extracted from the character data Dc or materials used in the conference. Alternatively, the determination may be made by AI (Artificial Intelligence). However, the present invention is not limited to these examples.

[0045] As described above, the speaker determination unit 114 determines the number of speakers Ps of the voice data Ds. If the number of speakers Ps is one, the terminal user U has already been authenticated, and therefore it can be assumed that the voice data Ds corresponds to the voice of the terminal user U. The terminal user U is also one of the participants P of the conference. If there are two or more speakers Ps, there is a possibility that all speakers P other than the terminal user U are also participants P of the conference, or that participants other than the conference participants P are included. An example of a case in which participants other than the conference participants P are included is when the terminal user U is wearing the headset 280 and talking with someone other than those involved in the conference. However, this is not a limited example.

[0046] As described above, the volume determination unit 115 determines whether the volume V corresponding to the audio data Ds is equal to or greater than a predetermined volume Vt. A case in which the volume V is less than the predetermined volume Vt may occur, for example, when the terminal user U wears the headset 280, which serves as both the audio input unit 250 and the speaker 260, during a conference, but removes the headset 280 only at that time. Here, the headset 280 includes a microphone 250A instead of the audio input unit 250, and an ear speaker 260A instead of the speaker 260. When the headset 280 is removed, the distance between the terminal user U's mouth and the microphone 250A increases compared to when the headset 280 is normally worn, and the volume of the terminal user U's voice picked up by the microphone 250A decreases. The predetermined volume Vt may be, for example, the average volume when the terminal user U is speaking in a conference while wearing the headset 280, or a value obtained by multiplying the average volume by a certain ratio (e.g., 50%). However, the present invention is not limited to these examples.

[0047] As described above, the storage unit 120 stores the keywords D124 extracted from the character data Dc and materials used in the conference. The storage unit 120 may also store in advance the voice data Ds of the participant P as identification information for identifying the speaker Ps.

[0048] The storage unit 120 stores information and data necessary for controlling each unit of the information processing device 100. The storage unit 120 may store or include an OS, programs, etc. executed by the CPU 110. The storage unit 120 includes memory, specifically, volatile memory and nonvolatile memory. Volatile memory includes, for example, dynamic random access memory (DRAM) and static random access memory (SRAM), but is not limited to these. Nonvolatile memory includes, for example, read-only memory (ROM), flash memory, solid state drive (SSD), and hard disk, but is not limited to these.

[0049] The display unit 130 includes both those with and without memory properties. Examples of display units without memory properties include, but are not limited to, liquid crystal and organic electroluminescence (EL). Examples of display units with memory properties include, but are not limited to, electronic paper.

[0050] As described above, the second communication unit 140 communicates with the first communication unit 290 of the terminal device 200. The second communication unit 140 also communicates with the schedule management service device .

[0051] 1.4 Configuration of each terminal device 200 As described above, the terminal device 200 is a client device used by each participant P of the conference to use the electronic conference service provided by the electronic conference service device 20. The following describes configurations and supplementary matters that have not yet been mentioned. However, the CPU 210, storage unit 220, and display unit 230 have the same configurations as the CPU 110, storage unit 120, and display unit 130 described above, respectively, and therefore their description will be omitted.

[0052] The conference control unit 211 a of the conference support application 211 manages information received from the electronic conference application 212 .

[0053] The display control unit 211b controls the display and display content on the display unit 230 by the electronic conference application 212.

[0054] The device control unit 211c manages the device information and input / output states of the devices included in the terminal device 200, that is, the storage unit 220, the display unit 230, the operation unit 240, the audio input unit 250, the speaker 260, and the camera 270.

[0055] As described above, the first communication unit 290 communicates with the second communication unit 140 of the information processing device 100. Furthermore, the first communication control unit 211d also communicates with the second communication control unit 212a of the electronic conference application 212.

[0056] The application microphone 212 b is interposed between the electronic conference application 212 and the audio input unit 250 .

[0057] The operation unit 240 accepts operations on the terminal device 200. Examples of the operation unit 240 include, but are not limited to, a mouse and a keyboard. The display unit 230 and the operation unit 240 may be integrated using a touch panel that allows touch operations and the like.

[0058] As described above, the voice S of a conference participant P is input to the voice input unit 250. An example of the voice input unit 250 is a microphone that inputs voice, but is not limited to this. In FIG. 1 , the voice input unit 250 is built into the terminal device 200. However, the voice input unit 250 may be arranged outside the terminal device 200 and connected to the terminal device 200.

[0059] The speaker 260 outputs the voice S of another participant P of the conference. In FIG. 1, the speaker 260 is built into the terminal device 200. However, the speaker 260 may be arranged outside the terminal device 200 and connected to the terminal device 200. Furthermore, the voice input unit 250 may be built into the speaker 260. Alternatively, the voice input unit 250 and the speaker 260 may be replaced by a headset 280 that integrates these functions. The headset 280 is extremely suitable in that it can reliably pick up the voice of the terminal user U and can improve the determination accuracy of the volume determination unit 115 by maintaining a substantially constant distance between the mouth of the terminal user U and the voice input unit 250.

[0060] The camera 270 captures an image of the face of the terminal user U. In FIG. 1, the camera 270 is built into the terminal device 200. However, the camera 270 may be disposed outside the terminal device 200 and connected to the terminal device 200. Such a camera 270 may have a built-in audio input unit 250.

[0061] 1.5 Basics of information processing method by information processing system 1 Next, the most basic information processing method by the information processing system 1 will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the basics of the information processing method by the information processing system 1.

[0062] 5, in step S1, the second communication unit 140 of the information processing device 100 receives audio data Ds output corresponding to audio S from a terminal device 200 in which the currently running electronic conference application 212 is muted. Here, this terminal device 200 has an audio input unit 250 to which the audio S of a conference participant P is input. Note that step S1 is an example of the "step of receiving audio data" of the present disclosure.

[0063] In step S2, the intention analysis processing unit 113 analyzes the voice data Ds and determines whether or not the voice S has an intention to be about a conference (first determination). Note that step S2 is an example of the "step of making a first determination" in the present disclosure.

[0064] In step S3, the speaker determination unit 114 determines the number of speakers Ps of the voice data Ds (second determination). Note that step S3 is an example of the "step of making a second determination" of the present disclosure.

[0065] In step S4, the second communication unit 140 transmits the result of the first determination and the result of the second determination to the terminal device 200. Note that step S4 is an example of the "step of transmitting the result of the first determination and the result of the second determination" of the present disclosure.

[0066] In step S5, the mute state control unit 211g of the terminal device 200 first determines the result of the first determination. If the result of the first determination indicates that the voice S is intended to be directed to the conference, the process proceeds to the next step S6; otherwise, the process ends.

[0067] In step S6, the mute state control unit 211g also determines the result of the second determination. If the result of the second determination indicates that the number of speakers Ps is one, the process proceeds to the next step S7; if not, the process ends.

[0068] In step S7, the mute state control unit 211g also cancels the mute state of the electronic conference application 212 running on the terminal device 200. Note that these steps S5 to S7 are an example of the "step of canceling the mute state" of the present disclosure.

[0069] Therefore, even if the voice S of a conference participant P is input, the mute state is maintained as it is if the voice S is not intended to be directed to the conference or includes voice S from someone other than the user of the terminal device 200. As a result, it is possible to prevent the mute state from being accidentally released, causing the unintended voice S to be heard by other participants P and disrupting the progress of the conference.

[0070] 1.6 Application examples of the information processing method by the information processing system 1 Next, with reference to FIGS. 6 to 12, an application example of the information processing method by the information processing system 1 will be described. FIG. 6 is a flowchart showing an application example of the information processing method by the information processing system 1. FIG. 7 is an explanatory diagram illustrating an example of a login screen of the conference support application 211. FIG. 8 is an explanatory diagram illustrating an example of a screen for selecting a desired schedule using a schedule management service. FIG. 9A is an explanatory diagram illustrating an example of a display when the application microphone 212b of the electronic conference application 212 is enabled (the mute state is canceled). FIG. 9B is an enlarged view of the icon of the application microphone 212b in FIG. 9A. FIG. 10A is an explanatory diagram illustrating an example of a display when the application microphone 212b is in a muted state. FIG. 10B is an enlarged view of the icon of the application microphone 212b in FIG. 10A. FIG. 11 is an explanatory diagram illustrating a situation in which a terminal user U is having a conversation with a person H who is unrelated to the conference. FIG. 12 is an explanatory diagram illustrating another situation in which a terminal user U is having a conversation with a person H who is unrelated to the conference.

[0071] As shown in FIG. 6, in step S11, the authentication unit 112 of the information processing device 100 authenticates a user (hereinafter referred to as "terminal user U") who is attempting to participate in a conference using the terminal device 200. Prior to this, the terminal user U launches the conference support application 211 on the terminal device 200 that he or she is using. For example, a login screen such as that shown in FIG. 7 appears, and the terminal user U logs in using registered account information (user ID and password). This account information at the time of authentication allows the information processing device 100 to directly identify the terminal user U without relying on voice recognition or the like. Note that the terminal user U is also a participant P of the conference.

[0072] For example, single sign-on provided by another company's service can be used to log in to the conference support application 211. In this case, since it is possible to log in to the conference support application 211 using account information for the other company's service, there is no need to create a new account dedicated to the conference support application 211. During the login process, the user information is linked between the other company's service used and the conference support service provided by the information processing device 100.

[0073] Furthermore, the terminal user U can access the schedule management service provided by the schedule management service device 10 from a web browser or the like on the terminal device 200. The schedule management service displays a schedule list or the like as shown in FIG. 8 on the display unit 230, and the terminal user U selects a desired schedule by operating the operation unit 240. If the selected schedule is an electronic conference, the terminal user U can start the electronic conference application 212 with a simple operation. This allows the terminal user U to participate in the electronic conference of the selected schedule.

[0074] 9A and 10A, the display unit 230 displays the electronic conference application 212 and the conference support application 211 side by side. In the case of Fig. 9A, the application microphone 212b is enabled, so the icon Mon indicating that the mute state is canceled is displayed, as also shown in Fig. 9B. On the other hand, in the case of Fig. 10A, the application microphone 212b is muted, so the icon Moff indicating that the application microphone 212b is muted is displayed, as also shown in Fig. 10B.

[0075] In step S12, similar to step S1 described above, the second communication unit 140 of the information processing device 100 receives audio data Ds output corresponding to the audio S from the terminal device 200 in which the running electronic conference application 212 is in a muted state.

[0076] In step S13, the intention analysis processing unit 113 determines whether or not an actual voice S has been input. If a voice has been input, the process proceeds to the next step S14; if not, the process returns to step S12.

[0077] In step S14, similarly to step S2 described above, the intention analysis processing unit 113 analyzes the voice data Ds and determines whether or not the voice S has an intention to be directed to a conference.

[0078] In step S15, if the determination result in step S14 is that the voice S has an intention to be directed to the conference, the process proceeds to the next step S16, and if not, the process returns to step S12.

[0079] In step S16, similarly to step S3 described above, the speaker determination unit 114 determines the number of speakers Ps of the voice data Ds.

[0080] In step S17, if the result of the determination in step S16 is that the number of speakers Ps is one, the process proceeds to step S20, and if not, the process proceeds to the next step S18.

[0081] In step S18, the speaker determination unit 114 compares the voice data Ds with the voice data of all the conference participants P (or invitees to the conference) to identify all the speakers Ps. It is assumed that the voice data of all the conference participants P (or invitees to the conference) is stored in advance in the storage unit 120 as identification information for identifying the speakers Ps.

[0082] In step S19, the speaker determination unit 114 determines whether or not all speakers Ps identified in step S18 include anyone other than the conference participant P. If no speakers Ps include anyone other than the conference participant P, the process proceeds to the next step S20; otherwise, the process returns to step S12.

[0083] An example of a situation where there are two or more speakers Ps and the speakers P are people other than the conference participants P is the situation shown in Fig. 11. That is, in this situation, a person H unrelated to the conference approaches the seat of a terminal user U during a conference and starts talking to the terminal user U, so the terminal user U responds while wearing headset 280. This is because microphone 250A of headset 280 picks up the voices of not only terminal user U but also unrelated person H, so the number of speakers Ps determined from the voice data Ds will be two or more.

[0084] In step S20, the volume determination unit 115 determines whether the volume V corresponding to the audio data Ds is equal to or greater than a predetermined volume Vt.

[0085] In step S21, if the determination result in step S20 is that the volume V is equal to or greater than the predetermined volume Vt, the process proceeds to the next step S22, and if not, the process returns to step S12.

[0086] Note that, when the determination result in step S20 is that the volume V is not equal to or greater than the predetermined volume Vt, that is, when the volume V is less than the predetermined volume Vt, it is possible to cite a situation such as that shown in Fig. 12. That is, when a person H who is unrelated to the conference approaches the seat of a terminal user U during a conference and starts talking to the terminal user U, the terminal user U takes off the headset 280 to respond.

[0087] In step S22, the second communication unit 140 of the information processing device 100 transmits to the terminal device 200 an instruction to cancel the mute state of the electronic conference application 212.

[0088] In step S23, the mute state control unit 211g of the terminal device 200 cancels the mute state of the electronic conference application 212 that is currently running.

[0089] 1.7 Information Processing System According to Modification of First Embodiment Next, a description will be given of an information processing system according to a modification of the first embodiment. In this information processing system, the configuration relating to communication between the information processing device 100 and the terminal device 200 is simplified.

[0090] The terminal device 200 has an audio input unit 250, an audio data control unit 211e, a mute state detection unit 211f, and a mute state control unit 211g. The audio input unit 250 receives input of audio S from conference participants. The audio data control unit 211e outputs audio data Ds corresponding to the audio S input to the audio input unit 250. The mute state detection unit 211f detects whether the currently running electronic conference application 212 is in a mute state. The mute state control unit 211g controls switching of the mute state of the electronic conference application 212.

[0091] The information processing device 100 has an intention analysis processing unit 113 and a speaker determination unit 114. The intention analysis processing unit 113 analyzes the voice data Ds of the terminal device 200 in a muted state and determines whether or not the voice S has an intention to be directed to the conference. The speaker determination unit 114 determines the number of speakers of the voice data Ds.

[0092] The mute state control unit 211g cancels the mute state when the intention analysis processing unit 113 determines that the voice S has an intention to be directed to the conference and the speaker determination unit 114 determines that the number of speakers Ps is one.

[0093] Therefore, even if the voice S of a conference participant P is input, the mute state is maintained as it is if the voice S is not intended to be directed to the conference or includes voice S from someone other than the user of the terminal device 200. As a result, it is possible to prevent the mute state from being accidentally released, causing the unintended voice S to be heard by other participants P and disrupting the progress of the conference.

[0094] Second Embodiment The information processing device 100 included in the above-described information processing system 1 is within the scope of the present disclosure even when used alone. Therefore, the following description will be given with reference to FIG.

[0095] 1, the information processing device 100 includes a second communication unit 140, an intention analysis processing unit 113, and a speaker determination unit 114. The second communication unit 140 receives audio data Ds output in response to audio S from a terminal device 200 in which an active electronic conference application 212 is muted. Here, this terminal device 200 has an audio input unit 250 to which audio S of a participant P of the conference is input. The intention analysis processing unit 113 analyzes the audio data Ds and determines whether the audio S has an intention to target the conference. The speaker determination unit 114 determines the number of speakers Ps of the audio data Ds.

[0096] The second communication unit 140 transmits the first determination result of the intention analysis processing unit 113 and the second determination result of the speaker determination unit 114 to the terminal device 200. If the first determination result indicates that the voice S has an intention to target the conference and the second determination result indicates that the number of speakers Ps is one, the mute state is canceled.

[0097] Therefore, even if the voice S of a conference participant P is input, the mute state is maintained as it is if the voice S is not intended to be directed to the conference or includes voice S from someone other than the user of the terminal device 200. As a result, it is possible to prevent the mute state from being accidentally released, causing the unintended voice S to be heard by other participants P and disrupting the progress of the conference.

[0098] <Third embodiment> The scope of the present disclosure also includes a voice input control program that causes a computer to execute voice input control, such as detecting and controlling the mute state of the application microphone 212b of the electronic conference application 212, in the conference support application 211 of the terminal device 200 included in the information processing system 1. Therefore, the following description will be given with reference to Fig. 1 and Fig. 13. Fig. 13 is a flowchart of the voice input control process in the conference support application 211.

[0099] 13, in step S31, the mute state detection unit 211f of the conference support application 211 detects whether the currently running electronic conference application 212 is in a mute state. Note that step S31 is an example of the "process of detecting whether the device is in a mute state" in the present disclosure.

[0100] In step S32, if it is detected in step S31 that the electronic conference application 212 is in a muted state, the process proceeds to the next step S33, and if not, the process returns to step S31.

[0101] In step S33, the first communication unit 290 transmits voice data Ds corresponding to the input voice S of the conference participant P to the information processing device 100. Note that step S33 is an example of the "step of transmitting voice data" of the present disclosure.

[0102] In step S34, the first communication unit 290 receives the determination result from the information processing device 100.

[0103] In step S35, if the determination result received in step S34 indicates that the voice S is intended to be directed to a conference, the process proceeds to the next step S36, otherwise the process ends.

[0104] In step S36, if the determination result received in step S34 indicates that the number of speakers Ps is one, the process proceeds to the next step S37, and if not, the process ends.

[0105] In step S37, the mute state control unit 211g cancels the mute state. Note that steps S35 to S37 are an example of the "step of canceling the mute state" of the present disclosure.

[0106] Therefore, even if the voice S of a conference participant P is input, the mute state is maintained as it is if the voice S is not intended to be directed to the conference or includes voice S from someone other than the user of the terminal device 200. As a result, it is possible to prevent the mute state from being accidentally released, causing the unintended voice S to be heard by other participants P and disrupting the progress of the conference.

[0107] The present invention can be embodied in various other forms without departing from its spirit or main features. Therefore, the above-described embodiments are merely illustrative in all respects and should not be interpreted as limiting. The scope of the present invention is defined by the claims and is not limited to the text of the specification. Furthermore, all modifications and variations within the equivalent range of the claims are within the scope of the present invention. [Industrial Applicability]

[0108] The present disclosure is applicable to an information processing system, an information processing device, an information processing method, and a voice input control program. [Explanation of symbols]

[0109] 1. Information Processing Systems 10 Schedule management service device 20. Electronic conference service equipment 100 Information processing device 113 Intention analysis processing unit 113a Voice Recognition Unit 113b Analysis Department 113c Intention determination part 114 Speaker Determination Unit 115 Volume determination unit 120 Storage section 130 Display section 140 Second Communications Department 200 Terminal Device 200-1 First terminal device 211 Meeting Support Application 211d First communication control unit 211e Audio data control unit 211f Mute state detection unit 211g Mute status control section 212 Electronic Conferencing Applications 212a Second communication control unit 212b App Microphone 220 Storage section 230 Display section 240 Operation section 250 Audio input section 260 speakers 270 Camera 280 Headset 290 First Communications Department

Claims

1. An information processing system including a plurality of terminal devices and an information processing device, The terminal device a voice input unit to which voices of participants in a conference are input; a first communication unit that communicates with the information processing device; a voice data control unit that transmits voice data corresponding to the voice input to the voice input unit to the information processing device; a mute state detection unit that detects whether a running conference application is in a mute state; a mute state control unit that controls switching of the mute state of the conference application; and The information processing device includes: a second communication unit that communicates with the first communication unit of the terminal device; an intention analysis processing unit that analyzes the voice data received by the second communication unit from the first communication unit of the terminal device in the muted state and determines whether the voice has an intention to be directed to the conference; a speaker determination unit that determines the number of speakers of the voice data; and the second communication unit transmits the first determination result of the intention analysis processing unit and the second determination result of the speaker determination unit to the terminal device; The mute state control unit cancels the mute state in a first case in which the first determination result indicates that the audio is intended to be directed at the conference and the second determination result indicates that the number of speakers is one.

2. At least one of the information processing device and the terminal device further includes a volume determination unit that determines whether a volume corresponding to the audio data is equal to or greater than a predetermined volume; The information processing system according to claim 1 , wherein the mute state control unit does not cancel the mute state even in the first case unless the volume is equal to or greater than the predetermined volume.

3. The intention analysis processing unit a voice recognition unit that converts the voice data into character data; an analysis unit that analyzes the character data and extracts features; an intention determination unit that determines whether the voice has an intention to be directed to the conference based on the extracted features; 3. The information processing system according to claim 1, further comprising:

4. the information processing device further includes a storage unit that stores keywords extracted from the character data and materials used in the conference; The information processing system according to claim 3 , wherein the intention determination unit determines whether the speech has an intention to be directed to the conference based on the extracted features and the keywords stored in the storage unit.

5. the storage unit pre-stores the voice data of the participants as identification information for identifying the speaker; the speaker determination unit identifies the speaker based on the identification information and determines whether the speakers include anyone other than the participants; The information processing system according to claim 4 , wherein the mute state control unit does not cancel the mute state even in the first case if the speakers include a person other than the participant.

6. The terminal device A display unit; a display control unit that controls the display unit; and the plurality of terminal devices includes a first terminal device, when the volume corresponding to the audio data received by the second communication unit of the information processing device from the first terminal device in the mute state is equal to or greater than the predetermined volume, the second communication unit transmits a notification indicating that the audio has been input to the terminal device other than the first terminal device; The information processing system according to claim 2 , wherein the display control unit of the terminal device, in which the first communication unit has received the notification, causes the display unit to display content of the notification.

7. when the volume corresponding to the audio data received by the second communication unit of the information processing device from the first terminal device in the mute state is equal to or greater than the predetermined volume, the second communication unit transmits the notification indicating that the audio has been input to the first terminal device; The information processing system according to claim 6 , wherein the display control unit of the first terminal device, of which the first communication unit has received the notification, causes the display unit to display content of the notification.

8. a second communication unit having an audio input unit to which the voices of the conference participants are input, and which receives audio data output in response to the voices from a terminal device in which a conference application currently being executed is in a muted state; an intention analysis processing unit that analyzes the voice data and determines whether the voice has an intention to be directed to the conference; a speaker determination unit that determines the number of speakers of the voice data; Equipped with the second communication unit transmits the first determination result of the intention analysis processing unit and the second determination result of the speaker determination unit to the terminal device; An information processing device, wherein the mute state is canceled when the first determination result indicates that the audio is intended to be directed to the conference and the second determination result indicates that the number of speakers is one.

9. receiving audio data output from a terminal device having an audio input unit for inputting audio of a conference participant and running a conference application in a muted state, the audio data corresponding to the audio; analyzing the audio data to make a first determination of whether the audio is intended to be directed to the conference; performing a second determination of the number of speakers of the voice data; transmitting a result of the first determination and a result of the second determination; canceling the mute state when the result of the first determination indicates that the voice is intended to be directed to the conference and the result of the second determination indicates that the number of speakers is one; An information processing method, including:

10. A voice input control program that causes a computer to execute voice input control of a terminal device, detecting whether a running conferencing application is in a muted state; analyzing input voice data corresponding to voices of conference participants; canceling the mute state when it is determined that the voice has an intention to be directed to the conference and the number of speakers of the voice data is determined to be one; A voice input control program, including:

11. An information processing system including a plurality of terminal devices and an information processing device, The terminal device a voice input unit to which voices of participants in a conference are input; a voice data control unit that outputs voice data corresponding to the voice input to the voice input unit; a mute state detection unit that detects whether a running conference application is in a mute state; a mute state control unit that controls switching of the mute state of the conference application; and The information processing device includes: an intention analysis processing unit that analyzes the voice data of the terminal device in the muted state and determines whether the voice has an intention to be directed to the conference; a speaker determination unit that determines the number of speakers of the voice data; and The mute state control unit cancels the mute state when the intention analysis processing unit determines that the voice has an intention to be directed to the conference and the speaker determination unit determines that the number of speakers is one.

Images