Information processing device and program

Abstract

To prevent the occurrence of howling due to the presence of two or more information processing devices in close proximity when a remote conference is held by transmitting and receiving audio data between a plurality of information processing devices via a network.SOLUTION: A CPU receives advertisement signals each including a conference ID, and information about the state of a microphone or information about the state of a speaker, from other terminal devices through short-range wireless communication channels. The CPU identifies, among the terminal devices that have transmitted the received advertisement signals, as a proximity device, a terminal device whose advertisement signal contains the same conference ID as the conference ID of a web conference which its own device is scheduled to participate in and whose distance from the own device is shorter than a preset distance. The CPU compares the state of the microphone of the identified proximity device with the state of a speaker of the own device, compares the state of the speaker of the identified proximity device with the state of a microphone of the own device, and determines the possibility of occurrence of howling.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an information processing device and a program. [Background technology]

[0002] Patent document 1 discloses a transmission management device that, when a video conference is held among multiple transmission terminals, acquires location information of the transmission terminals, and, if the distance between one transmission terminal and another transmission terminal is equal to or less than a threshold, instructs the audio output of the other transmission terminals to be muted.

[0003] Patent document 2 discloses a system in which multiple information processing devices used in a web conference each transmit an audio signal representing the picked-up audio to a server device, and the server device transmits an audio signal transmitted from one information processing device to other information processing devices, and each information processing device does not output the audio represented by the received audio signal if the device that transmitted the received audio signal is a device in the same room. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-143534 [Patent Document 2] Japanese Patent Application Publication No. 2019-140517 Summary of the Invention [Problem to be solved by the invention]

[0005] The object of the present invention is to provide an information processing device and a program that can suppress the occurrence of howling caused by two or more information processing devices being located in close proximity when conducting a remote conference by sending and receiving audio data between multiple information processing devices via a network. [Means for solving the problem]

[0006] A program according to a first aspect of the present invention includes a receiving step of receiving, via a wireless communication line, from another information processing device, conference identification information for identifying a remote conference to be attended and notification information including information on the state of a sound collection unit or information on the state of a sound output unit; a step of identifying, as a nearby device, an information processing device that includes conference identification information identical to conference identification information of a remote conference in which the information processing device is scheduled to participate, and that is closer to the information processing device than a predetermined distance, among the information processing devices that have transmitted the received notification information; The computer executes a determination step of comparing the state of the sound collection section in the identified nearby device with the state of the audio output section in the device itself, or comparing the state of the audio output section in the identified nearby device with the state of the sound collection section in the device itself, to determine whether or not there is a possibility of howling occurring.

[0007] A second aspect of the present invention provides a program according to the first aspect, further comprising a warning step of issuing a warning to that effect when it is determined in the determining step that there is a possibility of feedback occurring.

[0008] A program according to a third aspect of the present invention is the program according to the first or second aspect, wherein the information about the state of the sound collection unit includes information indicating whether the sound collection unit is in an on state or an off state, The information about the state of the audio output unit includes information indicating whether the audio output unit is in an on state or an off state, In the determination step, if both the sound collection unit in the identified nearby device and the sound output unit in the device itself are on, or if both the sound output unit in the identified nearby device and the sound collection unit in the device itself are on, it is determined that there is a possibility of howling occurring.

[0009] A fourth aspect of the program of the present invention is a program of the third aspect, in which, if it is determined in the determination step that there is a possibility of howling occurring, the computer is further caused to execute a suppression step of changing the sound collection unit or sound output unit of the device to an off state and suppressing the occurrence of howling.

[0010] A fifth aspect of the program of the present invention is the program of the third aspect, in which, if it is determined in the determination step that there is a possibility of howling occurring, the computer further executes a request step of requesting the proximity device to change the sound collection unit or the sound output unit to an off state to suppress the occurrence of howling.

[0011] A sixth aspect of the present invention provides a program according to the first or second aspect, wherein the information on the state of the sound collection unit includes information indicating the sound collection sensitivity of the sound collection unit, The information about the state of the audio output unit includes information indicating the output intensity of the audio output unit, In the determination step, it is determined whether or not there is a possibility of howling occurring based on the relationship between the sound collection sensitivity of the sound collection section of the identified nearby device and the output strength of the audio output section of the device itself, or the relationship between the output strength of the audio output section of the identified nearby device and the sound collection sensitivity of the sound collection section of the device itself.

[0012] A seventh aspect of the program of the present invention is a program of the sixth aspect, in which, if it is determined in the determination step that there is a possibility of howling occurring, the computer is further made to execute a suppression step of changing the sound collection sensitivity of the sound collection section or the output intensity of the audio output section of the device to suppress the occurrence of howling.

[0013] The program of the eighth aspect of the present invention is the program of the sixth aspect, in which, if it is determined in the determination step that there is a possibility of howling occurring, the computer further executes a request step of requesting the proximity device to change the sound collection sensitivity of the sound collection unit or the output intensity of the audio output unit to suppress the occurrence of howling.

[0014] A ninth aspect of the present invention provides a program according to any one of the first to eighth aspects, wherein in the specifying step, the received radio wave intensity of the wireless communication line is determined to be equal to or greater than a preset threshold value. End The information processing device is identified as an information processing device whose distance from the information processing device is shorter than a preset distance.

[0015] A tenth aspect of the program of the present invention is a program of the first aspect, in which, if it is determined in the determination step that there is a possibility of howling occurring, the computer is further made to execute a suppression step of suppressing the occurrence of howling by selecting either changing the state of the sound collection unit or sound output unit in the device itself, or changing the state of the sound collection unit or sound output unit in the nearby device.

[0016] An eleventh aspect of the program of the present invention is a program of the tenth aspect, in which, in the suppression step, priority is given to a device between the device itself and the nearby device that has later changed the state of its sound collection unit or sound output unit, and howling is suppressed by changing the state of the sound collection unit or sound output unit of a device that is not given priority.

[0017] A twelfth aspect of the program of the present invention is a program of the tenth aspect, in which, in the suppression step, priority is given to a device with a higher pre-set priority between the device itself and the nearby device, and the occurrence of howling is suppressed by changing the state of the sound collection unit or sound output unit of a device with a lower priority.

[0018] A 13th aspect of the program of the present invention is a program of the 10th aspect, in which the suppression step determines which device should be prioritized between the device itself and the nearby device based on characteristic information of the sound collection unit or characteristic information of the audio output unit used, and suppresses the occurrence of howling by changing the state of the sound collection unit or audio output unit of the device that is not prioritized.

[0019] A 14th aspect of the program of the present invention is a program of the 13th aspect, in which, in the suppression step, it is determined that, between the device itself and the nearby device, the device using an externally connected device as the sound collection unit or the device using an externally connected device as the device that should be prioritized, and the occurrence of howling is suppressed by changing the state of the sound collection unit or sound output unit of the device that is not prioritized.

[0020] An information processing device according to a fifteenth aspect of the present invention includes a processor, The processor: receiving, via a wireless communication line, from another information processing device, conference identification information for identifying a remote conference in which the user is participating and notification information including information on the state of the sound collection unit or information on the state of the audio output unit; Among the information processing devices that have transmitted the received notification information, an information processing device whose notification information includes conference identification information identical to conference identification information of the remote conference in which the own device is scheduled to participate and whose distance from the own device is shorter than a preset distance is identified as a nearby device; The state of the sound collection section in the identified nearby device is compared with the state of the sound output section in the device itself, or the state of the sound output section in the identified nearby device is compared with the state of the sound collection section in the device itself to determine whether or not there is a possibility of howling occurring. [Effects of the Invention]

[0021] According to the program of the first aspect of the present invention, when a remote conference is held by transmitting and receiving audio data between multiple information processing devices via a network, it is possible to suppress the occurrence of howling caused by two or more information processing devices being located in close proximity to each other.

[0022] According to the program of the second aspect of the present invention, a user who is holding a remote conference using an information processing device can know that there is a possibility that feedback will occur if the remote conference is held in the current state.

[0023] According to the program of the third aspect of the present invention, when a remote conference is held by transmitting and receiving audio data between multiple information processing devices via a network, it is possible to suppress the occurrence of howling caused by two or more information processing devices being located in close proximity to each other.

[0024] According to the program of the fourth aspect of the present invention, it is possible to suppress the occurrence of howling without the user having to perform an operation to turn off the sound collection unit or the sound output unit.

[0025] According to the program of the fifth aspect of the present invention, a user who is holding a remote conference using an information processing device can turn off the sound collection unit or the sound output unit in response to a request, thereby suppressing the occurrence of howling.

[0026] According to the program of the sixth aspect of the present invention, even when both the sound collection unit in the nearby device and the audio output unit in the device itself are on, or when both the audio output unit in the nearby device and the sound collection unit in the device itself are on, if the possibility of howling occurring is not high, it is possible not to determine that there is a possibility of howling occurring.

[0027] According to the program of the seventh aspect of the present invention, even if it is determined that there is a possibility of howling occurring, it is possible to suppress the occurrence of howling without turning off the sound collection section or sound output section of the device itself.

[0028] According to the program of the eighth aspect of the present invention, it is possible to suppress the occurrence of howling without changing the sound collection sensitivity of the sound collection section or the output intensity of the audio output section of the device itself.

[0029] According to the program of the ninth aspect of the present invention, it is possible to estimate the distance between another information processing device and the own device without needing the position information of the other information processing device and the own device, respectively.

[0030] According to the program of the 10th aspect of the present invention, when it is determined that there is a possibility of howling occurring, it becomes possible to switch between changing the state of the sound collection unit or sound output unit in the device itself or changing the state of the sound collection unit or sound output unit in a nearby device depending on the situation.

[0031] According to the program of the 11th aspect of the present invention, it is possible to prevent malfunctions such as audio no longer being collected or audio no longer being output in a device in which the state of the audio collection unit or audio output unit is subsequently changed.

[0032] According to the program of the 12th aspect of the present invention, it is possible to prevent the state of the sound collection unit or audio output unit in a device with a higher priority from being changed due to a change in the state of the sound collection unit or audio output unit in a device with a lower priority.

[0033] According to the program of the 13th aspect of the present invention, it is possible to set a device that changes the state of the sound collection unit or the sound output unit when it is determined that there is a possibility of howling occurring, based on the characteristics of the sound collection unit or the characteristics of the sound output unit.

[0034] According to the program of the 14th aspect of the present invention, when it is determined that there is a possibility of howling occurring, it is possible to prevent problems such as audio not being collected or not being output in a device that uses an externally connected device as a sound collection unit or sound output unit.

[0035] According to the information processing device of the 15th aspect of the present invention, when a remote conference is held by transmitting and receiving audio data between multiple information processing devices via a network, it is possible to suppress the occurrence of howling caused by two or more information processing devices being located in close proximity to each other. [Brief explanation of the drawings]

[0036] [Figure 1] 1 is a diagram illustrating a system configuration of a Web conference system according to an embodiment of the present invention. [Figure 2] FIG. 1 is a diagram for explaining the principle of howling. [Figure 3] FIG. 2 is a diagram illustrating a hardware configuration of a terminal device 20 according to an embodiment of the present invention. [Figure 4] 10 is a flowchart for explaining a process for suppressing the occurrence of howling, which is performed in a terminal device 20 according to an embodiment of the present invention. [Figure 5] FIG. 10 is a diagram showing how the terminal device 20A receives an advertising signal transmitted from the terminal device 20B. [Figure 6] 10A and 10B are diagrams for explaining a specific example of a process for determining whether or not there is a possibility of howling occurring. [Figure 7] FIG. 10 is a diagram showing an example of a warning display screen that is displayed when it is determined that there is a possibility of howling occurring. [Figure 8] FIG. 10 is a diagram showing an example of a warning display screen that is displayed when it is determined that there is a possibility of howling occurring. [Figure 9] 10 is a flowchart for explaining the processing in the terminal device 20 when, when it is determined that there is a possibility of howling occurring, processing is performed to suppress the occurrence of howling. [Figure 10] 10 is a flowchart showing details of the process of step S107 shown in the flowchart of FIG. 9. [Figure 11] 10A and 10B are diagrams illustrating an example of control in which the settings of other devices are given priority to suppress the occurrence of howling. [Figure 12] 10A and 10B are diagrams illustrating an example of control in which the setting of the device itself is prioritized to suppress the occurrence of howling. [Figure 13] 10 is a diagram showing how an advertising signal including the output intensity level of a speaker and the sound collection sensitivity level of a microphone is transmitted from the terminal device 20B to the terminal device 20A. FIG. [Figure 14] 10A and 10B are diagrams illustrating an example of control in which the occurrence of howling is suppressed by performing level adjustment, and the occurrence of howling is suppressed by prioritizing the settings of other devices. [Figure 15]10A and 10B are diagrams illustrating an example of control in which the occurrence of howling is suppressed by performing level adjustment, and the occurrence of howling is suppressed by prioritizing the settings of the device itself. [Figure 16] 10A and 10B are diagrams illustrating an example of control in which level adjustment is performed in both the own device and another device to suppress the occurrence of howling. [Figure 17] FIG. 10 is a diagram for explaining a specific control pattern example 1 in an actual usage situation. [Figure 18] 10A to 10C are diagrams for explaining howling suppression processing in each case of specific control pattern example 1 in actual use situations. [Figure 19] 10 is a flowchart for explaining the details of step S107 in the flowchart shown in FIG. 9 when attempting to suppress the occurrence of howling among three or more terminal devices 20. [Figure 20] FIG. 10 is a diagram for explaining a specific control pattern example 2 in an actual usage situation. [Figure 21] 10A to 10C are diagrams for explaining howling suppression processing in each case of specific control pattern example 2 in actual use situations. DETAILED DESCRIPTION OF THE INVENTION

[0037] Next, an embodiment of the present invention will be described in detail with reference to the drawings.

[0038] FIG. 1 is a diagram showing the system configuration of a Web conference system according to an embodiment of the present invention.

[0039] As shown in FIG. 1, a Web conference system according to an embodiment of the present invention is configured by interconnecting a plurality of terminal devices 20A to 20D and a Web conference management server 10 via the Internet 30.

[0040] The Web conference system shown in FIG. 1 is a system for transmitting and receiving audio data and image data between a plurality of information processing devices 20A to 20D via a network such as the Internet 30 to hold a Web conference, which is a remote conference.

[0041] The terminal devices 20A to 20D are, for example, information processing devices such as notebook personal computers used by users A to D, respectively. Therefore, users A to D can move to various locations and participate in the Web conference.

[0042] The Web conference management server 10 manages a Web conference held among a plurality of terminal devices 20A to 20D, issues a conference ID, which is conference identification information for identifying the Web conference, and controls the holding of the Web conference.

[0043] An application program for Web conferences (hereinafter abbreviated as an app) is installed in each of the terminal devices 20A to 20D. A Web conference is held by activating this Web conference app in the terminal devices 20A to 20D.

[0044] Here, the terminal devices 20A to 20D can each participate in a Web conference from different locations, but there are also cases where a meeting is held between locations in which multiple conference participants gather in a predetermined location such as a conference room to participate in a Web conference.

[0045] However, when multiple users participating in the same web conference gather in the same place, a phenomenon called feedback may occur.

[0046] Next, the principle behind howling will be explained with reference to FIG.

[0047] For example, consider a case where terminal device 20A and terminal device 20B, participating in the same Web conference, are present in the same conference room. In such a case, if the speaker of terminal device 20A is on and the microphone of terminal device 20B is on, the audio output from the speaker of terminal device 20A may be picked up by the microphone of terminal device 20B. When the Web conference starts, the audio picked up by the microphone of terminal device 20B is sent as audio data to terminal device 20A via Web conference management server 10 and output as audio from the speaker. As a result, a loop of audio information occurs between terminal devices 20A and 20B, and if the loop gain is 1 or greater, oscillation occurs, causing an abnormal audio phenomenon known as feedback.

[0048] Furthermore, when multiple participants gather in the same conference room to hold a web conference and audio is output from the conference speaker, howling can occur if the conference speaker and the microphone of the terminal device are close to each other.

[0049] When this type of feedback occurs, it can interfere with normal audio transmission and reception during a web conference. Therefore, if two or more terminal devices are located close to each other in the same conference room, the web conference cannot be conducted normally unless feedback is suppressed.

[0050] Therefore, in the Web conferencing system of this embodiment, by performing the control described below, the occurrence of howling caused by two or more terminal devices being located in close proximity when conducting a remote conference such as a Web conference is suppressed.

[0051] Next, the hardware configuration of the terminal device 20 of this embodiment is shown in Fig. 3. When the terminal devices 20A to 20D are referred to without specifying each other, they will be simply referred to as the terminal device 20.

[0052] 3, the terminal device 20 has a CPU 11, a memory 12, a storage device 13 such as a hard disk drive, a communication interface (abbreviated as IF) 14 for transmitting and receiving data to and from external devices via the Internet 30, a user interface (abbreviated as UI) device 15 including a touch panel or liquid crystal display and a keyboard, a microphone 16 functioning as a sound collection unit, a speaker 17 functioning as an audio output unit, and a short-range wireless communication unit 18 for transmitting and receiving data via a wireless communication line conforming to a standard such as Bluetooth (registered trademark). These components are connected to each other via a control bus 19.

[0053] The communication IF 14 may have a function of connecting to the Internet 30 via a wireless LAN communication line such as WiFi (registered trademark).

[0054] The CPU 11 is a processor that executes predetermined processing based on a control program stored in the memory 12 or the storage device 13, thereby controlling the operation of the terminal device 20. In this embodiment, the CPU 11 is described as reading and executing the control program stored in the memory 12 or the storage device 13, but it is also possible to provide the program to the CPU 11 by storing it on a storage medium such as a CD-ROM.

[0055] Next, a process for suppressing the occurrence of howling, which is performed when a Web conference application in the terminal device 20 in this embodiment attempts to start a Web conference, will be described with reference to the flowchart of FIG.

[0056] First, in step S101, the CPU 11 receives from another terminal device an advertisement signal, which is notification information including a conference ID that identifies the Web conference to be attended and information about the microphone status or information about the speaker status, via a short-range wireless communication line such as Bluetooth (registered trademark).

[0057] FIG. 5 shows how the terminal device 20A receives the advertising signal transmitted from the terminal device 20B.

[0058] 5, it can be seen that an advertising signal is being transmitted from terminal device 20B to terminal device 20A via a short-range wireless communication line. This advertising signal contains a conference ID, which is conference identification information, information about the speaker status (that the speaker is on), information about the microphone status (that the microphone is off), and device-specific information, namely, the device name of the own device. It is assumed here that the conference ID "1234A" represents a conference called "Conference A" that is about to be held.

[0059] By receiving such an advertising signal from the terminal device 20B, the terminal device 20A can identify the speaker state and microphone state of the terminal device 20B and the conference that user B is scheduled to participate in. Note that, although the following description will only describe the processing executed in the terminal device 20A, similar processing is also executed in other terminal devices such as the terminal device 20B.

[0060] Next, in steps S102 to S104, the CPU 11 of the terminal device 20A identifies, among the terminal devices 20 that have transmitted the received advertisement signals, the terminal devices 20 whose advertisement signals contain a conference ID that is the same as the conference ID of the Web conference in which the terminal device 20A is scheduled to participate, and whose distance from the terminal device 20A is shorter than a preset distance, as nearby devices.

[0061] Specifically, first, in step S102, CPU 11 determines whether the conference ID included in the received advertising signal matches the conference ID of the conference that the own device is scheduled to participate in. If the conference ID included in the received advertising signal does not match the conference ID of the conference that the own device is scheduled to participate in in step S102, CPU 11 ends the process and waits until another advertising signal is received.

[0062] Then, in step S102, if the conference ID included in the received advertisement signal matches the conference ID of the conference in which the device is scheduled to participate, CPU 11 measures the distance between the terminal device and the device itself based on the received radio wave strength of the short-range wireless communication that transmitted the advertisement signal.

[0063] Note that the distance between the terminal device that transmitted the advertising signal and the device itself may not be measured strictly, but may be determined based on a distance range indicating whether the distance is long or short depending on whether the received radio wave strength is above or below a certain threshold. Alternatively, the distance may be determined using three or more distance ranges in addition to the two types of distance ranges, long and short.

[0064] Furthermore, the method for measuring the distance between one's own device and another device is not limited to a measurement method based on the received radio wave strength in short-range wireless communication, but various measurement methods can be used, such as a position measurement method using GPS (Global Positioning System), a position measurement method using a beacon signal, and a measurement method based on the received radio wave strength in a wireless communication line other than short-range wireless communication.

[0065] Then, in step S104, CPU 11 determines whether the measured distance between the terminal devices is less than a threshold value. If it determines in step S104 that the measured distance between the terminal devices is equal to or greater than the threshold value, CPU 11 ends the process and waits until another advertising signal is received.

[0066] If it is determined in step S104 that the measured distance between the terminal devices is less than the threshold value, then in step S105, the CPU 11 determines whether or not there is a possibility that feedback will occur.

[0067] That is, the CPU 11 identifies a terminal device whose advertisement signal contains the same conference ID as the conference ID of the Web conference in which the own device is scheduled to participate and whose distance from the own device is shorter than a preset distance as a nearby device, and determines whether or not there is a possibility of howling occurring between the identified nearby device and the terminal device. End The terminal device is identified as a terminal device whose distance from the own device is shorter than a preset distance.

[0068] The CPU 11 compares the state of the microphone in the identified nearby device with the state of the speaker in the device itself, or compares the state of the speaker in the identified nearby device with the state of the microphone in the device itself, to determine whether or not there is a possibility of howling occurring.

[0069] Specifically, the microphone status information included in the advertising signal includes information indicating whether the microphone is on or off, and the speaker status information includes information indicating whether the speaker is on or off. CPU 11 then determines that there is a possibility of feedback occurring when both the microphone of the identified nearby device and the speaker of the device itself are on, or when both the speaker of the identified nearby device and the microphone of the device itself are on.

[0070] A specific example of the process of determining whether or not there is a possibility of feedback occurring will be described with reference to FIG.

[0071] 6 illustrates a case where terminal devices 20B to 20D are present around terminal device 20A. In Fig. 6, because the distance between terminal device 20A and terminal device 20C is long, terminal device 20C is not determined to be a nearby device. Furthermore, because the conference that user D of terminal device 20D plans to attend is conference B, the advertising signal transmitted from terminal device 20D includes a conference ID that is different from the conference ID of conference A, and therefore terminal device 20D is not determined to be a nearby device despite being in the vicinity of terminal device 20A.

[0072] However, the conference that user B of terminal device 20B is scheduled to attend is Conference B, and terminal device 20B is close to terminal device 20A and the distance between the devices is short, so it is determined to be a nearby device. Then, because the speaker of terminal device 20B is on and the microphone of terminal device 20A is on, CPU 11 of terminal device 20A determines that there is a possibility of feedback occurring if the Web conference is started in this state.

[0073] If it is determined in step S105 that there is no possibility of howling occurring between the adjacent device and the CPU 11, the CPU 11 ends the process and waits until another advertising signal is received.

[0074] Then, in step S105, if it is determined that there is a possibility that feedback will occur between the adjacent device and the adjacent device, CPU 11 warns the user that there is a possibility that feedback will occur.

[0075] Examples of warning display screens that are displayed when it is determined that there is a possibility of feedback occurring are shown in Figs. 7 and 8.

[0076] Referring to FIG. 7, the display screen of terminal device 20A displays the message "If you continue to hold a Web conference in this state, there is a possibility that feedback may occur between the terminal devices.", warning the user.

[0077] Also, referring to Figure 8, in addition to the warning described above, the message "Please turn off the microphone" is displayed, showing the user specific steps to take to prevent feedback from occurring.

[0078] In addition, terminal device 20A may not only display the warning display as described above on its own display screen, but may also request terminal device 20B, the other device that has determined that there is a possibility of howling occurring, to display a warning display on its display screen.

[0079] In the flowchart shown in FIG. 4, if there is a possibility of feedback occurring, the user is simply warned of that fact. However, if the CPU 11 determines that there is a possibility of feedback occurring, it is also possible to perform processing to suppress the occurrence of feedback.

[0080] FIG. 9 is a flowchart showing a process for suppressing the occurrence of howling when it is determined that there is a possibility of howling occurring.

[0081] The flowchart shown in Fig. 9 is obtained by replacing the process of step S106 with the process of step S107 in the flowchart shown in Fig. 4. Therefore, a description of the processes of steps S101 to S105 will be omitted.

[0082] If it is determined in step S105 that there is a possibility of feedback occurring, the CPU 11 executes processing to suppress the occurrence of feedback in step S107.

[0083] The details of the process in step S107 are shown in the flowchart of FIG.

[0084] First, in step S201, the CPU 11 determines whether to prioritize the settings of another terminal device, specifically the state of the speaker and microphone, or the settings of its own device.

[0085] Here, the terminal device 20A may be configured to switch between prioritizing the settings of its own device and the settings of another device. Alternatively, when it is determined that there is a possibility of feedback occurring, a selection screen may be displayed on the display screen, allowing the user to select whether to change the state of the microphone or speaker of the terminal device itself or to change the state of the microphone or speaker of another terminal device that is a nearby device, thereby suppressing the occurrence of feedback.

[0086] Alternatively, the occurrence of howling may be suppressed by giving priority to the device that changes the microphone or speaker state later between the device itself and nearby devices, and changing the microphone or speaker state of the device that is not given priority.

[0087] Furthermore, it is also possible to set a priority level in advance for each of the terminal devices 20, and give priority to the device with the higher priority level between the own device and nearby devices, and change the state of the microphone or speaker of the device with the lower priority level to suppress the occurrence of howling.

[0088] In addition, the device that should be prioritized between the device itself and nearby devices can be determined based on characteristic information of the microphone or speaker being used, and the occurrence of howling can be suppressed by changing the state of the microphone or speaker of the device that is not prioritized.

[0089] Specifically, the CPU 11 determines that, among its own device and nearby devices, the device using a microphone that is an externally connected device or a device using a speaker that is an externally connected device is the device that should be prioritized, and suppresses the occurrence of howling by changing the state of the microphone or speaker of the device that is not prioritized.

[0090] For example, if multiple users share a conference microphone or speaker in the same conference room, changing the microphone or speaker functions of the terminal device to which the shared microphone or speaker is connected could disrupt the web conference. Therefore, by determining that the terminal device using an externally connected microphone or speaker is the device that should be prioritized and controlling the settings of this terminal device to be maintained as much as possible, it is highly likely that the entire web conference will not be affected.

[0091] In addition, when the characteristic information of the microphone or speaker in each terminal device 20 is, for example, "microphone (for PC)" or "speaker (for PC)," it can be determined that the microphone or speaker inside the device is being used, and when it is "microphone (USB connection)," "microphone (for conference)," "speaker (external connection)," etc., it can be determined that the externally connected microphone or speaker is being used.

[0092] If it is determined in step S201 that the settings of the other device are to be prioritized as a result of the above-described determination, the CPU 11 switches the microphone or speaker of the own device to an off state in step S202 to suppress the occurrence of howling.

[0093] For example, FIG. 11 shows a control example in which the settings of other devices are given priority.

[0094] 11, the microphone of the terminal device 20A is in the on state and the speaker of the terminal device 20B is in the on state, so it is determined that there is a possibility of feedback occurring. Therefore, the terminal device 20A switches the state of its own microphone from the on state to the off state to suppress the occurrence of feedback. In this case, the setting of the terminal device 20B is maintained and not changed.

[0095] Furthermore, if, as a result of the above-described determination, it is determined in step S201 that the settings of the other device are not to be prioritized, CPU 11 requests in step S203 that the nearby device for which it has determined that howling will occur change its microphone or speaker to an off state to suppress the occurrence of howling.

[0096] Then, when the terminal device 20 receives the request to change the microphone or speaker to the off state, the terminal device 20 performs the process of changing the microphone or speaker to the off state, thereby suppressing the occurrence of howling.

[0097] For example, a control example in which the settings of the own device are given priority is shown in FIG.

[0098] In the control example shown in Fig. 12, the microphone of terminal device 20A is in an on state and the speaker of terminal device 20B is in an on state, so it is determined that there is a possibility of feedback occurring. Therefore, terminal device 20A transmits a request to terminal device 20B via a short-range wireless communication line to turn off the speaker. Upon receiving this request, terminal device 20B switches the state of its speaker from on to off to suppress the occurrence of feedback. In this case, the setting of terminal device 20A is maintained and not changed.

[0099] In the above, we have explained how to suppress the occurrence of howling by switching the microphone and speaker states between on and off, but it is also possible to suppress the occurrence of howling by adjusting the microphone's sound collection sensitivity level or the speaker's output intensity level.

[0100] Therefore, the following describes a case where the occurrence of howling is suppressed by adjusting the microphone sound collection sensitivity level and the speaker output intensity level. Note that in the following description, it is assumed that the microphone sound collection sensitivity level can be set within a range of 0 to 10, and the speaker output intensity level can be set within a range of 0 to 10. Here, when the microphone sound collection sensitivity level is 0, the microphone is in an off state, and when the speaker output intensity level is 0, the speaker is in an off state.

[0101] First, when suppressing the occurrence of howling by adjusting the microphone sound collection sensitivity level or the speaker output intensity level, the advertising signal includes, as information about the microphone status, information indicating the microphone sound collection sensitivity, for example, information about the microphone sound collection sensitivity level. Similarly, the advertising signal includes, as information about the speaker status, information indicating the speaker output intensity, for example, information about the speaker output intensity level.

[0102] FIG. 13 shows how an advertising signal including the speaker output intensity level and the microphone sound collection sensitivity level is transmitted from the terminal device 20B to the terminal device 20A.

[0103] Referring to Figure 13, it can be seen that the advertising signal sent from terminal device 20B to terminal device 20A includes, in addition to information such as the conference ID, information that the speaker output intensity level is "10" and information that the microphone sound collection sensitivity level is "0".

[0104] Then, the CPU 11 of the terminal device 20A determines whether or not there is a possibility of howling occurring based on the relationship between the sound collection sensitivity level of the microphone in the identified nearby device and the output intensity level of the speaker in the device itself, or the relationship between the output intensity level of the speaker in the identified nearby device and the sound collection sensitivity level of the microphone in the device itself.

[0105] Various determination methods can be used to determine when there is a possibility of feedback occurring in the relationship between the speaker output intensity levels and microphone sound collection sensitivity levels of the two terminal devices 20. For example, the simplest method is to determine that there is a possibility of feedback occurring when the sum of the speaker output intensity levels and microphone sound collection sensitivity levels is equal to or greater than a preset value, such as 14, or other determination methods may be used. Furthermore, the received radio wave intensity of the short-range wireless communication radio waves may also be taken into consideration when making the determination.

[0106] Furthermore, even when suppressing the occurrence of howling by adjusting the speaker output intensity level or the microphone sound collection sensitivity level, different control is performed depending on whether the settings of another device are prioritized or the settings of the device itself are prioritized.

[0107] FIG. 14 shows an example of control in which the occurrence of howling is suppressed by adjusting the level, and the occurrence of howling is suppressed by prioritizing the settings of other devices.

[0108] When suppressing the occurrence of howling by prioritizing the settings of the other device, the CPU 11 changes the sound collection sensitivity level of the microphone or the output intensity level of the speaker in the own device to suppress the occurrence of howling.

[0109] 14, it is determined that there is a possibility of feedback occurring because the sound collection sensitivity level of the microphone of the terminal device 20A is 8 and the output intensity level of the speaker of the terminal device 20B is 10. In this case, the CPU 11 of the terminal device 20A reduces the sound collection sensitivity level of the microphone of the terminal device 20A from 8 to 3, for example, to suppress the occurrence of feedback.

[0110] Next, FIG. 15 shows an example of control in which the occurrence of howling is suppressed by adjusting the level, and the occurrence of howling is suppressed by prioritizing the settings of the device itself.

[0111] When suppressing the occurrence of howling by prioritizing the settings of the terminal device 20A, the CPU 11 of the terminal device 20A requests the terminal device 20B, which is a nearby device, to change the microphone sound collection sensitivity level or the speaker output intensity level to suppress the occurrence of howling.

[0112] For example, in the control example shown in Fig. 15, the microphone sound collection sensitivity level of the terminal device 20A is 8, and the speaker output intensity level of the terminal device 20B is 10, so it is determined that there is a possibility of feedback occurring. In this case, the CPU 11 of the terminal device 20A transmits a request to the terminal device 20B via the short-range wireless communication line to set the speaker output intensity level to 4 or less. Then, the terminal device 20B reduces the speaker output intensity level of its own device from 10 to 4, for example, to suppress the occurrence of feedback.

[0113] When suppressing feedback by changing the microphone sensitivity level or the speaker output intensity level, it is also possible to suppress feedback by adjusting the levels in both the device itself and the other device. An example of control in this case is shown in Figure 16.

[0114] When adjusting the levels in both the own device and the other device in this way to suppress the occurrence of howling, the CPU 11 of the terminal device 20A requests the terminal device 20B, which is the nearby device, to change the microphone sound collection sensitivity level or the speaker output intensity level, and also changes the microphone sound collection sensitivity level or the speaker output intensity level in the own device.

[0115] For example, in the control example shown in FIG. 16, the microphone sound collection sensitivity level of terminal device 20A is 8, and the speaker output intensity level of terminal device 20B is 10, so it is determined that there is a possibility of feedback occurring. In this case, CPU 11 of terminal device 20A transmits a request to terminal device 20B via the short-range wireless communication line to set the speaker output intensity level to 6 or less. Then, terminal device 20B performs control to lower the speaker output intensity level of its own device, for example, from 10 to 6. Furthermore, CPU 11 of terminal device 20A lowers the microphone sound collection sensitivity level of its own device from 8 to 5. By adjusting the microphone sound collection sensitivity level and speaker output intensity level in both devices in this way, the occurrence of feedback is suppressed.

[0116] Next, specific examples of control patterns in actual use will be described with reference to FIGS.

[0117] First, a case will be described in which two terminal devices 20A and 20B are present close to each other in the same conference room, as shown in FIG.

[0118] 17, it is assumed that the speaker of the terminal device 20A is in an off state, and the microphone is initially in an off state but is changed to an on state midway, with the sound collection sensitivity level at that time being 9. It is also assumed that the speaker of the terminal device 20B is in an on state with an output intensity level of 10, and the microphone is in an on state with a sound collection sensitivity level of 10.

[0119] Fig. 18 shows what kind of howling suppression processing is selected in various cases in the state shown in Fig. 17. Note that the selection of the howling suppression processing shown in Fig. 18 is performed, for example, by displaying a selection screen to the user when it is determined that there is a possibility of howling occurring, and the user viewing this screen selects one of the options "give priority to the settings of the own device," "give priority to the settings of the other device," or "change the status of both the own device and the other device."

[0120] First, Case 1 shows a case where, for example, the user selects an option to "give priority to the settings of the other device" and gives priority to the settings of the terminal device 20B. In this Case 1, the microphone sound collection sensitivity level of the terminal device 20A is lowered from 9 to 2 to suppress the occurrence of howling.

[0121] Furthermore, case 2 shows a case where the user selects an option of "give priority to the settings of the own device" and gives priority to the settings of the own device, that is, terminal device 20A. In this case 2, terminal device 20A transmits a request to terminal device 20B to set the speaker output intensity level to 3 or less. As a result, terminal device 20B performs control to lower the speaker output intensity level from 10 to 3.

[0122] Furthermore, case 3 shows a case where, for example, the user selects an option "change the status of both the own device and the other device" to change the settings of both the own device, terminal device 20A, and the other device, terminal device 20B. In this case 3, the microphone sound collection sensitivity level is lowered from 9 to 4 in terminal device 20A, and a request is transmitted from terminal device 20A to terminal device 20B to lower the speaker output intensity level to 5 or less. As a result, control is performed in terminal device 20B to lower the speaker output intensity level from 10 to 5. In this way, level adjustments are performed in both terminal devices 20A and 20B, and the occurrence of howling is suppressed.

[0123] The above describes the control for suppressing howling that occurs between two terminal devices 20, but the following describes the control for suppressing howling that occurs when three or more terminal devices 20 are located close to each other.

[0124] For example, when suppressing the occurrence of howling among three or more terminal devices 20, the howling occurrence suppression process in step S107 in the flowchart shown in FIG. 9 is executed by the process shown in the flowchart in FIG.

[0125] First, in step S301, the CPU 11 determines whether to prioritize the settings of another terminal device, specifically the state of the speaker and microphone, or the settings of its own device.

[0126] If it is determined in step S301 that the setting of the other device is to be prioritized, the CPU 11 switches the microphone or speaker of the own device to an off state in step S302 to suppress the occurrence of howling.

[0127] If it is determined in step S301 that the settings of other devices are not to be prioritized, the CPU 11 determines in step S303 whether to change the settings of all devices. It is assumed that whether to change the settings of all devices or only the settings of the device that is most likely to cause feedback is selected and set in advance.

[0128] If it is determined in step S303 that the settings of all devices are to be changed, then in step S304, the CPU 11 requests all other devices that have been able to receive the advertising signal to change the state of their microphones or speakers.

[0129] If it is determined in step S303 that the settings of all devices will not be changed, that is, that only the settings of the other device that is most likely to cause howling will be changed, then in step S305, CPU 11 requests the other device that is most likely to cause howling to change the state of its microphone or speaker.

[0130] Next, a specific example of a control pattern in an actual usage situation where three terminal devices are located close to each other will be described with reference to FIGS.

[0131] First, a case will be described in which three terminal devices 20A to 20c are present close to each other in the same conference room, as shown in FIG.

[0132] 20, the speaker of the terminal device 20A is in an off state, and the microphone is initially in an off state but is changed to an on state midway, with the sound collection sensitivity level at that time being 9. Also, the speakers of the terminal devices 20B and 20C are in an on state with an output intensity level of 10, and the microphones are in an on state with a sound collection sensitivity level of 10.

[0133] Here, it is assumed that no howling occurs between terminal device 20B and terminal device 20C, that the distance between terminal device 20A and terminal device 20B is shorter than the distance between terminal device 20A and terminal device 20C, and that the probability of howling occurring between terminal device 20A and terminal device 20B is high. It is assumed that there is a probability of howling occurring between terminal device 20A and terminal device 20C, but that this probability is relatively low.

[0134] Fig. 21 shows what kind of howling suppression processing is selected in various cases in the state shown in Fig. 20. Note that the selection of the howling suppression processing shown in Fig. 21 is performed, for example, by displaying a selection screen to the user when it is determined that there is a possibility of howling occurring, and the user viewing this display selecting one of the options "give priority to the settings of the own device (change the settings of only the device most likely to cause howling)", "give priority to the settings of the own device (change the settings of all other devices)", "give priority to the settings of other devices", or "change the status of all devices, including the own device and other devices".

[0135] First, case 1 shows a case where, for example, the user selects the option "give priority to the settings of other devices" and gives priority to the settings of terminal devices 20B and 20C. In this case 1, the microphone sound collection sensitivity level of terminal device 20A is lowered from 9 to 2 to suppress the occurrence of feedback.

[0136] Furthermore, case 2 shows a case where the user selects an option of "give priority to the settings of the own device (change only the settings of the device most likely to cause howling)" and gives priority to the settings of the own device, that is, terminal device 20A. In this case 2, terminal device 20A transmits a request to terminal device 20B to set the speaker output intensity level to 3 or less. As a result, terminal device 20B performs control to lower the speaker output intensity level from 10 to 3.

[0137] Furthermore, case 3 shows a case where the user selects an option of "give priority to the settings of my own device (change the settings of all other devices)" and gives priority to the settings of terminal device 20A, which is my own device. In this case 3, terminal device 20A transmits a request to terminal devices 20B and 20C to set the speaker output intensity level to 4 or less. As a result, control is performed in terminal devices 20B and 20C to lower the speaker output intensity level from 10 to 4.

[0138] Furthermore, case 4 shows a case where, for example, the user selects an option to "change the status of all devices, including the user's own device and other devices," thereby changing the settings of terminal device 20A, which is the user's own device, and the status of all devices, including terminal devices 20B and 20C, which are other devices. In this case 4, the microphone sound collection sensitivity level is lowered from 9 to 3 in terminal device 20A, and a request is sent from terminal device 20A to terminal devices 20B and 20C to lower the speaker output intensity level to 5 or less. As a result, control is performed in terminal devices 20B and 20C to lower the speaker output intensity level from 10 to 5. In this way, level adjustment is performed in all devices, terminal devices 20A, 20B, and 20C, and the occurrence of howling is suppressed.

[0139] In the above embodiment, a case has been described in which the speaker output intensity and microphone sound collection sensitivity are set using relative numerical values ​​such as the output intensity level and sound collection sensitivity level, but the present invention can also be applied in the same way when the speaker output intensity and microphone sound collection sensitivity are set using absolute numerical values ​​such as sound pressure level (dB SPL (Sound Pressure Level)).

[0140] In each of the above embodiments, the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

[0141] Furthermore, the operations of the processor in each of the above embodiments may be performed not only by a single processor but also by multiple processors located at physically separate locations working together. Furthermore, the order of the operations of the processor is not limited to the order described in each of the above embodiments and may be changed as appropriate. [Explanation of symbols]

[0142] 10 Web conference management server 11 CPU 12 Memory 13 Storage device 14 Communication Interface 15 User Interface Device 16. Mike 17 Speaker 18 Near Field Wireless Communication Department 19 Control Bus 20A~20D Terminal equipment 30 Internet

Claims

1. a receiving step of receiving, via a wireless communication line, from another information processing device, conference identification information for identifying a remote conference in which the user is participating and notification information including information on the state of the sound collection unit or information on the state of the audio output unit; a step of identifying, as a nearby device, an information processing device that includes conference identification information identical to conference identification information of a remote conference in which the information processing device is scheduled to participate, and that is closer to the information processing device than a predetermined distance, among the information processing devices that have transmitted the received notification information; a determining step of comparing the state of the sound collection unit in the identified nearby device with the state of the audio output unit in the device itself, or comparing the state of the audio output unit in the identified nearby device with the state of the sound collection unit in the device itself, to determine whether or not there is a possibility of howling occurring; A program that causes a computer to execute the following.

2. 2. The program according to claim 1, further causing the computer to execute a warning step of issuing a warning to the effect that feedback may occur, if it is determined in the determining step that feedback may occur.

3. The information about the state of the sound collection unit includes information indicating whether the sound collection unit is in an on state or an off state, The information about the state of the audio output unit includes information indicating whether the audio output unit is in an on state or an off state, A program as described in claim 1 or 2, wherein in the judgment step, it is judged that there is a possibility of howling occurring when both the sound collection unit in the identified nearby device and the sound output unit in the device itself are on, or when both the sound output unit in the identified nearby device and the sound collection unit in the device itself are on.

4. The program of claim 3, further causing the computer to execute a suppression step of suppressing the occurrence of howling by changing the sound collection unit or sound output unit of the device to an off state if it is determined in the judgment step that there is a possibility of howling occurring.

5. The program of claim 3, further comprising a request step of requesting the proximity device to change the sound collection unit or sound output unit to an off state to suppress the occurrence of howling if it is determined in the determination step that there is a possibility of howling occurring.

6. The information about the state of the sound collection unit includes information indicating the sound collection sensitivity of the sound collection unit, The information about the state of the audio output unit includes information indicating the output intensity of the audio output unit, A program as described in claim 1 or 2, wherein the determination step determines whether or not there is a possibility of howling occurring based on the relationship between the sound collection sensitivity of the sound collection section of the identified nearby device and the output strength of the audio output section of the device itself, or the relationship between the output strength of the audio output section of the identified nearby device and the sound collection sensitivity of the sound collection section of the device itself.

7. The program of claim 6, further comprising causing the computer to execute a suppression step of suppressing the occurrence of howling by changing the sound collection sensitivity of the sound collection section or the output intensity of the audio output section of the device if it is determined in the judgment step that there is a possibility of howling occurring.

8. The program of claim 6, further comprising a request step of requesting the proximity device to change the sound collection sensitivity of the sound collection unit or the output intensity of the audio output unit to suppress the occurrence of howling if it is determined in the determination step that there is a possibility of howling occurring.

9. A program described in any one of claims 1 to 8, wherein in the identification step, an information processing device whose received radio wave strength of the wireless communication line is equal to or greater than a predetermined threshold is identified as an information processing device whose distance from the device itself is shorter than a predetermined distance.

10. The program of claim 1, further comprising causing the computer to execute a suppression step of suppressing the occurrence of howling by selecting either changing the state of the sound collection unit or sound output unit of the device itself, or changing the state of the sound collection unit or sound output unit of the nearby device, if the judgment step determines that there is a possibility of howling occurring.

11. The program of claim 10, wherein the suppression step prioritizes the device between the device itself and the nearby device that has later changed the state of its sound collection unit or sound output unit, and suppresses the occurrence of howling by changing the state of the sound collection unit or sound output unit of the device that is not prioritized.

12. The program of claim 10, wherein the suppression step prioritizes a device with a higher pre-set priority between the device itself and the nearby device, and suppresses the occurrence of howling by changing the state of the sound collection unit or sound output unit of a device with a lower priority.

13. The program of claim 10, wherein the suppression step determines which device should be prioritized between the device itself and the nearby device based on characteristic information of the sound collection unit or characteristic information of the audio output unit being used, and suppresses the occurrence of howling by changing the state of the sound collection unit or audio output unit of the device that is not prioritized.

14. The suppression step determines that, between the device itself and the nearby device, the device using an externally connected device as the sound collection unit or the device using an externally connected device as the device that should be prioritized, and suppresses the occurrence of howling by changing the state of the sound collection unit or the sound output unit of the non-priority device.

15. a processor; The processor: receiving, via a wireless communication line, from another information processing device, conference identification information for identifying a remote conference in which the user is participating and notification information including information on the state of the sound collection unit or information on the state of the audio output unit; Among the information processing devices that have transmitted the received notification information, an information processing device that includes conference identification information identical to conference identification information of the remote conference in which the own device is scheduled to participate in the notification information and is located closer to the own device than a preset distance is identified as a nearby device; The state of the sound collection unit in the identified nearby device is compared with the state of the sound output unit in the device itself, or the state of the sound output unit in the identified nearby device is compared with the state of the sound collection unit in the device itself, to determine whether or not there is a possibility of howling occurring. Information processing device.

Images