Electronic device, program, and system

Abstract

This electronic device comprises: a communication unit that communicates with a speaker's terminal; a speech output unit that outputs a speaker's speech signal received from the speaker's terminal by the communication unit as a speaker's speech; and a control unit that sets the volume of the speech outputted from the speech output unit. The control unit transmits information visually indicating the level of the speech at the position of a candidate to which the speech is to be transmitted, and can change the volume of the speech when the level of the speech is changed at the terminal.

Description

Electronic devices, programs, and systems CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to Japanese Patent Application No. 2021-115943, filed on July 13, 2021, the entire disclosure of which is incorporated herein by reference.

[0002] The present disclosure relates to an electronic device, a program, and a system.

[0003] In recent years, opportunities for so-called remote conferences, such as web conferences or video conferences, have been increasing. In remote conferences, electronic devices (or systems including electronic devices) are used to enable communication between participants in multiple locations. For example, consider a scenario in which a conference is held in an office, and at least one of the conference participants holds the remote conference at a remote home. In this case, audio and / or video of the conference held in the office is acquired by, for example, an electronic device installed in the office and transmitted to, for example, an electronic device installed in the participant's home. Furthermore, audio and / or video of the conference held at the participant's home is acquired by, for example, an electronic device installed in the participant's home and transmitted to, for example, an electronic device installed in the office. Using such electronic devices, a conference can be held without all participants having to gather in the same location.

[0004] Various technologies that can be applied to the above-mentioned remote conferences have been proposed. For example, Patent Document 1 discloses a device that displays a graphic representing the output range of directional sound output from a speaker, superimposed on an image captured by a camera. This device allows the output range of directional sound to be visually grasped.

[0005] JP 2010-21705 A

[0006] According to one embodiment, an electronic device includes: a communication unit that communicates with a terminal of a speaker; an audio output unit that outputs a voice signal of the speaker received from the terminal by the communication unit as the voice of the speaker; and a control unit that sets a volume of the voice output by the audio output unit. The control unit transmits to the terminal information visually indicating a level of the voice at a position of a candidate target to which the voice is transmitted, and can change the volume of the voice when the level of the voice is changed on the terminal.

[0007] A program according to one embodiment causes an electronic device to perform the following steps: communicating with a speaker's terminal; outputting the speaker's voice signal received from the terminal as the speaker's voice; setting the volume of the voice to be output in the outputting step; transmitting information to the terminal that visually indicates the level of the voice at the position of a candidate target to which the voice is to be transmitted; and, when the level of the voice is changed on the terminal, potentially changing the volume of the voice.

[0008] A system according to one embodiment includes an electronic device and a speaker's terminal that can communicate with each other. The electronic device includes: an audio output unit that outputs a voice signal of the speaker received from the terminal as the speaker's voice; and a control unit that sets the volume of the voice output by the audio output unit. The control unit transmits information to the terminal that visually indicates the level of the voice at the position of a candidate target to which the voice is to be transmitted, and controls the terminal to change the volume of the voice when the level of the voice is changed. The terminal includes: a sound collection unit that collects the voice of the speaker; and a control unit that transmits the voice signal of the speaker to the electronic device, receives from the electronic device information visually indicating the level of the voice at the position of a candidate target to which the voice is to be transmitted, and controls the terminal to send an input to change the level of the voice to the electronic device.

[0009] According to one embodiment, an electronic device is configured to be able to communicate with a terminal of a speaker and another electronic device that outputs the speaker's voice signal as the speaker's voice. The electronic device includes a control unit that sets the volume of the voice to be output and amplified by the other electronic device. The control unit transmits information to the terminal that visually indicates the level of the voice at the position of a candidate target to which the voice is to be transmitted, and can change the volume of the voice when the level of the voice is changed on the terminal, and controls the other electronic device to output the voice.

[0010] In addition, a program according to one embodiment causes an electronic device to execute the following steps: communicating with a terminal of a speaker and another electronic device that outputs the speaker's voice signal as the speaker's voice; setting the volume of the voice output by the other electronic device; transmitting information to the terminal that visually indicates the level of the voice at the position of a candidate target to which the voice is to be transmitted; changing the volume of the voice when the level of the voice is changed on the terminal; and controlling the other electronic device to output the voice.

[0011] Furthermore, a system according to one embodiment includes a terminal of a speaker, an electronic device, and another electronic device. The terminal and the electronic device are configured to be able to communicate with the other electronic device. The terminal includes: a sound collection unit that collects the speaker's voice; and a control unit that transmits the speaker's voice signal to the other electronic device, receives from the other electronic device information visually indicating the voice level at the position of a candidate target to which the voice is to be transmitted, and controls the other electronic device to transmit an input to change the voice level. The electronic device includes: a sound output unit that outputs the speaker's voice signal received from the other electronic device as the speaker's voice. The other electronic device includes: a control unit that sets the volume of the voice output by the electronic device; and the control unit transmits to the terminal information visually indicating the voice level at the position of a candidate target to which the voice is to be transmitted, and can change the volume of the voice when the voice level is changed on the terminal, and controls the electronic device to output the voice.

[0012] 14 is a diagram showing an example of a usage mode of a system including an electronic device and a terminal according to an embodiment. FIG. 15 is a functional block diagram schematically showing the configuration of an electronic device according to an embodiment. FIG. 16 is a functional block diagram schematically showing the configuration of a terminal according to an embodiment. FIG. 17 is a flowchart illustrating the operation of an electronic device according to an embodiment. FIG. 18 is a diagram showing an example of imaging by an electronic device according to an embodiment. FIG. 19 is a diagram showing an example of display by a terminal according to an embodiment. FIG. 20 is a flowchart illustrating the operation of an electronic device according to an embodiment. FIG. 21 is a diagram showing an example of display by a terminal according to an embodiment. FIG. 22 is a functional block diagram schematically showing the configuration of an electronic device according to another embodiment. FIG. 23 is a functional block diagram schematically showing the configuration of an electronic device according to another embodiment. FIG. 24 is a diagram showing an example of a usage mode of a system including an electronic device, a terminal, a server, etc. according to an embodiment. FIG. 25 is a functional block diagram schematically showing the configuration of a server etc. shown in FIG. 14. FIG. 26 is a flowchart illustrating the operation of the server etc. shown in FIG. 14.

[0013] In the present disclosure, an "electronic device" may refer to a device powered by power supplied from, for example, a power grid or a battery. In the present disclosure, a "user" may refer to a person (typically a human) who uses or may use an electronic device according to an embodiment, and a person who uses or may use a system including an electronic device according to an embodiment. In addition, in the present disclosure, a conference in which at least one participant participates by communication from a different location from the other participants, such as a web conference or video conference, is collectively referred to as a "remote conference."

[0014] Further improvements are desired in the functionality of electronic devices that enable communication between multiple locations, such as in remote conferences. An object of the present disclosure is to improve the functionality of electronic devices, programs, and systems that enable communication between multiple locations. According to one embodiment, it is possible to improve the functionality of electronic devices, programs, and systems that enable communication between multiple locations. Hereinafter, an electronic device according to one embodiment will be described in detail with reference to the drawings.

[0015] 1 is a diagram illustrating an example of a usage mode of an electronic device according to an embodiment. Hereinafter, a description will be given assuming a situation in which a participant Mg remotely participates in a conference held in a conference room MR from his / her home RL, as shown in FIG. 1. As shown in FIG. 1, participants Ma, Mb, Mc, Md, Me, and Mf participate in the conference in the conference room MR.

[0016] 1 , an electronic device 1 according to an embodiment may be installed in a conference room MR. Meanwhile, a terminal 100 that communicates with the electronic device 1 according to an embodiment may be installed in a home RL of a participant Mg. The location of the home RL of the participant Mg may be different from the location of the conference room MR. The location of the home RL of the participant Mg may be far away from the location of the conference room MR, or may be close to the location of the conference room MR.

[0017] 1 , an electronic device 1 according to an embodiment is connected to a terminal 100 according to an embodiment via, for example, a network N. The electronic device 1 according to an embodiment may be connected to the terminal 100 according to an embodiment via at least one of a wireless and a wired connection. In FIG. 1 , the state in which the electronic device 1 and the terminal 100 are connected wirelessly and / or via a wired connection via the network N is shown by a dashed line. In an embodiment, the electronic device 1 and the terminal 100 may be included in a remote conference system according to an embodiment.

[0018] In the present disclosure, the network N as shown in FIG. 1 may include, for example, various electronic devices and / or devices such as a server, as appropriate. The network N as shown in FIG. 1 may also include, for example, devices such as a base station and / or a repeater, as appropriate. In the present disclosure, for example, when the electronic device 1 and the terminal 100 "communicate," the electronic device 1 and the terminal 100 may communicate directly with each other. For example, when the electronic device 1 and the terminal 100 "communicate," the electronic device 1 and the terminal 100 may communicate via at least one of another device and / or a base station. For example, when the electronic device 1 and the terminal 100 "communicate," more specifically, the communication between the communication unit of the electronic device 1 and the communication unit of the terminal 100 may communicate.

[0019] The above-mentioned notation may include the same intention as above not only when the electronic device 1 and the terminal 100 "communicate" with each other, but also when one device "sends" information to the other device and / or when the other device "receives" information sent by the other device. Furthermore, the above-mentioned notation may include the same intention as above not only when the electronic device 1 and the terminal 100 "communicate" with each other, but also when any electronic device communicates with any other electronic device.

[0020] An electronic device 1 according to one embodiment may be arranged in a conference room MR, for example, as shown in FIG. 1 . In this case, the electronic device 1 may be arranged in a position where it can acquire audio and / or video of at least one of the conference participants Ma, Mb, Mc, Md, Me, and Mf. Furthermore, the electronic device 1 outputs audio and / or video of participant Mg, as described below. Therefore, the electronic device 1 may be arranged so that the audio and / or video of participant Mg output from the electronic device 1 reaches at least one of the conference participants Ma, Mb, Mc, Md, Me, and Mf.

[0021] The terminal 100 according to one embodiment may be placed in the home RL of the participant Mg, for example, in a manner as shown in Fig. 1. In this case, the terminal 100 may be placed in a position where it can acquire audio and / or video of the participant Mg. The terminal 100 may acquire audio and / or video of the participant Mg using a microphone, a headset, and / or a camera connected to the terminal 100.

[0022] Furthermore, as will be described later, the terminal 100 outputs audio and / or video of at least one of the participants Ma, Mb, Mc, Md, Me, and Mf of the conference in the conference room MR. Therefore, the terminal 100 may be arranged so that the audio and / or video output from the terminal 100 reaches the participant Mg. The audio output from the terminal 100 may be arranged so that it reaches the ears of the participant Mg via headphones, earphones, a headset, or the like, for example.

[0023] 1 shows only one example of a usage mode of the electronic device 1 and the terminal 100 according to an embodiment. The electronic device 1 and the terminal 100 according to an embodiment may be used in various other modes.

[0024] The remote conference system including the electronic device 1 and terminal 100 shown in FIG. 1 allows a participant Mg to act as if he or she were participating in a conference held in a conference room MR while at home RL. Furthermore, the remote conference system including the electronic device 1 and terminal 100 shown in FIG. 1 allows conference participants Ma, Mb, Mc, Md, Me, and Mf to feel as if the participant Mg were actually participating in the conference held in the conference room MR. That is, in the remote conference system including the electronic device 1 and terminal 100, the electronic device 1 placed in the conference room MR can serve as an avatar for the participant Mg. In this case, the electronic device 1 may function as a physical avatar representing the participant Mg. Furthermore, the electronic device 1 may function as a virtual avatar displaying an image of the participant Mg or an image, for example, of a character representing the participant Mg.

[0025] Next, the functional configuration of the electronic device 1 and the terminal 100 according to an embodiment will be described.

[0026] Fig. 2 is a block diagram that schematically shows the functional configuration of the electronic device 1 shown in Fig. 1. An example of the configuration of the electronic device 1 according to an embodiment will be described below.

[0027] The electronic device 1 according to an embodiment can be any of a variety of devices, but may be, for example, a specially designed device. The electronic device 1 according to an embodiment may also include, for example, a general-purpose smartphone, tablet, phablet, notebook computer (notebook PC or laptop), or computer (desktop) connected to the specially designed device.

[0028] 2, the electronic device 1 according to one embodiment may include a control unit 10, a storage unit 20, a communication unit 30, an imaging unit 40, a sound collection unit 50, an amplifier 60, an audio output unit 70, a direction adjustment unit 80, and a display unit 90. In one embodiment, the electronic device 1 may not include at least some of the functional units shown in FIG. 2, or may include components other than the functional units shown in FIG. 2.

[0029] The control unit 10 controls and / or manages the entire electronic device 1, including each functional unit constituting the electronic device 1. The control unit 10 may include at least one processor, such as a central processing unit (CPU) or a digital signal processor (DSP), to provide control and processing power for executing various functions. The control unit 10 may be implemented as a single processor, several processors, or individual processors. The processor may be implemented as a single integrated circuit (IC). The processor may be implemented as multiple communicatively connected integrated circuits and discrete circuits. The processor may also be implemented based on various other known technologies.

[0030] The control unit 10 may include one or more processors and memories. The processor may include a general-purpose processor that loads a specific program to execute a specific function, and a dedicated processor specialized for a specific process. The dedicated processor may include an application-specific integrated circuit (ASIC). The processor may include a programmable logic device (PLD). The PLD may include a field-programmable gate array (FPGA). The control unit 10 may be either a system-on-a-chip (SoC) or a system-in-a-package (SiP) in which one or more processors work together. The control unit 10 controls the operation of each component of the electronic device 1.

[0031] The control unit 10 may be configured to include, for example, at least one of software and hardware resources. Furthermore, in the electronic device 1 according to an embodiment, the control unit 10 may be configured by specific means in which software and hardware resources work together. The amplifier 60 described below may also be configured by at least one of software and hardware resources. Furthermore, in the electronic device 1 according to an embodiment, at least one of the other functional units may also be configured by specific means in which software and hardware resources work together.

[0032] In the electronic device 1 according to the embodiment, the operations such as control performed by the control unit 10 will be described in more detail later.

[0033] The storage unit 20 may function as a memory for storing various types of information. The storage unit 20 may store, for example, programs executed by the control unit 10 and results of processing executed by the control unit 10. The storage unit 20 may also function as a work memory for the control unit 10. As shown in FIG. 2 , the storage unit 20 may be connected to the control unit 10 via a wired and / or wireless connection. The storage unit 20 may include, for example, at least one of a random access memory (RAM) and a read-only memory (ROM). The storage unit 20 may be configured, for example, by a semiconductor memory or the like, but is not limited thereto, and may be any storage device. For example, the storage unit 20 may be a storage medium such as a memory card inserted into the electronic device 1 according to an embodiment. The storage unit 20 may be an internal memory of a CPU used as the control unit 10, or may be connected separately to the control unit 10.

[0034] The communication unit 30 has an interface function for wireless and / or wired communication with, for example, external devices. The communication method performed by the communication unit 30 in one embodiment may be a wireless communication standard. For example, the wireless communication standard includes cellular phone communication standards such as 2G, 3G, 4G, and 5G. For example, the cellular phone communication standard includes Long Term Evolution (LTE), Wideband Code Division Multiple Access (W-CDMA), CDMA2000, Personal Digital Cellular (PDC), Global System for Mobile communications (GSM), and Personal Handy-phone System (PHS). For example, the wireless communication standard includes Worldwide Interoperability for Microwave Access (WiMAX), IEEE 802.11, Wi-Fi, Bluetooth, Infrared Data Association (IrDA), and Near Field Communication (NFC). The communication unit 30 may include, for example, a modem whose communication method is standardized by the International Telecommunication Union Telecommunication Standardization Sector (ITU-T). The communication unit 30 can support one or more of the above communication standards.

[0035] The communication unit 30 may be configured to include, for example, an antenna for transmitting and receiving radio waves and an appropriate RF unit. The communication unit 30 may wirelessly communicate with, for example, a communication unit of another electronic device via an antenna. For example, the communication unit 30 may wirelessly communicate with the terminal 100 shown in FIG. 1. In this case, the communication unit 30 may wirelessly communicate with a communication unit 130 (described later) of the terminal 100. As such, in one embodiment, the communication unit 30 has a function of communicating with the terminal 100. The communication unit 30 may also be configured as an interface such as a connector for wired connection to an external device. The communication unit 30 can be configured using known technology for wireless communication, and therefore a more detailed description of the hardware and the like will be omitted.

[0036] 2, the communication unit 30 may be connected to the control unit 10 via a wired and / or wireless connection. Various pieces of information received by the communication unit 30 may be supplied to, for example, the storage unit 20 and / or the control unit 10. The various pieces of information received by the communication unit 30 may be stored in, for example, a memory built into the control unit 10. Furthermore, the communication unit 30 may transmit, for example, processing results by the control unit 10 and / or information stored in the storage unit 20 to the outside.

[0037] The imaging unit 40 may be configured to include an image sensor that captures images electronically, such as a digital camera. The imaging unit 40 may be configured to include an imaging element that performs photoelectric conversion, such as a CCD (Charge Coupled Device Image Sensor) or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The imaging unit 40 can capture, for example, an image of the surroundings of the electronic device 1. The imaging unit 40 may capture, for example, an image of the interior of a conference room MR shown in FIG. 1. In one embodiment, the imaging unit 40 may capture, for example, participants Ma, Mb, Mc, Md, Me, and Mf of a conference held in the conference room MR shown in FIG. 1.

[0038] The imaging unit 40 may convert the captured image into a signal and transmit it to the control unit 10. For this reason, the imaging unit 40 may be connected to the control unit 10 via a wired and / or wireless connection. Furthermore, a signal based on the image captured by the imaging unit 40 may be supplied to a functional unit of the electronic device 1, such as the storage unit 20 or the display unit 90. The imaging unit 40 is not limited to an imaging device such as a digital camera, and may be any device that captures an image of the inside of the conference room MR shown in FIG. 1 .

[0039] In one embodiment, the imaging unit 40 may capture, for example, still images of the state inside the conference room MR at predetermined time intervals (for example, 15 frames per second). Also, in one embodiment, the imaging unit 40 may capture, for example, continuous video of the state inside the conference room MR. Furthermore, the imaging unit 40 may be configured to include a fixed camera or a movable camera.

[0040] The sound collection unit 50 detects sounds or voices around the electronic device 1, including human voices. For example, the sound collection unit 50 may detect sounds or voices as air vibrations using, for example, a diaphragm and convert them into an electrical signal. Specifically, the sound collection unit 50 may include an acoustic device, such as a microphone, that converts sound into an electrical signal. In one embodiment, the sound collection unit 50 may detect the voices of at least one of the participants Ma, Mb, Mc, Md, Me, and Mf in the conference room MR shown in FIG. 1 . The voices (electrical signals) detected by the sound collection unit 50 may be input to, for example, the control unit 10. For this reason, the sound collection unit 50 may be connected to the control unit 10 via a wired and / or wireless connection.

[0041] The sound collection unit 50 may convert the collected sound or voice into an electrical signal and supply it to the control unit 10. The sound collection unit 50 may also supply the electrical signal (audio signal) into which the sound or voice has been converted to a functional unit of the electronic device 1, such as the storage unit 20. The sound collection unit 50 may be any device that detects sound or voice in the conference room MR shown in FIG.

[0042] The amplifier 60 appropriately amplifies the electrical signal of sound or voice (audio signal) supplied from the control unit 10 and supplies the amplified signal to the audio output unit 70. The amplifier 60 may be configured to include a device having a function of amplifying an electrical signal, such as an arbitrary amplifier. The amplifier 60 may amplify the electrical signal of sound or voice (audio signal) according to an amplification coefficient set by the control unit 10. The amplifier 60 may be connected to the control unit 10 by wire and / or wirelessly.

[0043] In one embodiment, the amplifier 60 may amplify the audio signal that the communication unit 30 receives from the terminal 100. Here, the audio signal received from the terminal 100 may be the audio signal of a speaker (e.g., participant Mg shown in FIG. 1 ) that is received by the communication unit 30 from the terminal 100 of the speaker.

[0044] The audio output unit 70 converts the electrical signal (audio signal) appropriately amplified by the amplifier 60 into sound and outputs it as sound or voice. The audio output unit 70 may be connected to the amplifier 60 via a wired and / or wireless connection. The audio output unit 70 may be configured to include a device having a function of outputting sound, such as an arbitrary speaker (loudspeaker). In one embodiment, the audio output unit 70 may be configured to include a directional speaker that transmits sound in a specific direction. Furthermore, the audio output unit 70 may be configured to be able to change the directionality of the sound.

[0045] In one embodiment, the audio output section 70 may output the audio signal of a speaker (for example, participant Mg shown in FIG. 1) amplified by the amplifier 60 as the voice of the speaker.

[0046] The direction adjustment unit 80 has a function of adjusting the direction of the sound or voice output by the audio output unit 70. The direction adjustment unit 80 may adjust the direction of the sound or voice output by the audio output unit 70 under the control of the control unit 10. For this reason, the direction adjustment unit 80 may be connected to the control unit 10 by wire and / or wirelessly. In one embodiment, the direction adjustment unit 80 may be configured to include a power source such as a servo motor that can change the direction of the audio output unit 70.

[0047] On the other hand, the direction adjustment unit 80 is not limited to having a function capable of changing the direction of the audio output unit 70. For example, the direction adjustment unit 80 may have a function capable of changing the orientation of the entire housing of the electronic device 1. In this case, the direction adjustment unit 80 may be configured to include a power source such as a servo motor that can change the orientation of the housing of the electronic device 1. By changing the orientation of the housing of the electronic device 1, it is possible to change the direction of the audio output unit 70 provided in the electronic device 1. Furthermore, by the direction adjustment unit 80 changing the orientation of the housing of the electronic device 1, it is possible to change the direction of the video (image) captured by the imaging unit 40.

[0048] The direction adjustment unit 80 may also be provided on a base or stand for placing the housing of the electronic device 1. In this case, the direction adjustment unit 80 may also be configured to include a power source such as a servo motor that can change the direction of the housing of the electronic device 1. As described above, the direction adjustment unit 80 may have a function that can change the direction or orientation of at least one of the audio output unit 70 and the electronic device 1.

[0049] Furthermore, for example, if the audio output unit 70 is configured to include a directional speaker, the direction adjustment unit 80 may have a function of adjusting (changing) the directivity of the sound or voice output by the audio output unit 70 .

[0050] In one embodiment, the direction adjustment unit 80 may adjust the direction of the voice of a speaker (e.g., participant Mg shown in FIG. 1 ) output by the audio output unit 70. The direction adjustment unit 80 may be configured to include any device as long as it has the function of adjusting the direction of the sound or voice that is ultimately output by the audio output unit 70. The direction of the voice adjusted by the direction adjustment unit 80 may be, for example, the left-right direction (horizontal direction) and / or the up-down direction (vertical direction).

[0051] The display unit 90 may be any display device, such as a liquid crystal display (LCD), an organic electroluminescence (EL) display (organic electroluminescence) panel, or an inorganic electroluminescence (EL) panel. The display unit 90 may display various types of information, such as characters, figures, or symbols. The display unit 90 may also display objects and icon images that constitute various GUIs, for example, to prompt the user to operate the electronic device 1.

[0052] Various data necessary for displaying on the display unit 90 may be supplied from, for example, the control unit 10 or the storage unit 20. Therefore, the display unit 90 may be connected to the control unit 10 or the like by wire and / or wirelessly. Furthermore, when the display unit 90 includes, for example, an LCD, it may be configured to include a backlight or the like as appropriate.

[0053] In one embodiment, the display unit 90 may display a video based on a video signal transmitted from the terminal 100. The display unit 90 may display, for example, a video of the participant Mg captured by the terminal 100 as a video based on a video signal transmitted from the terminal 100. By displaying the video of the participant Mg on the display unit 90 of the electronic device 1, for example, the participants Ma, Mb, Mc, Md, Me, and Mf shown in FIG. 1 can visually know the state of the participant Mg who is located away from the conference room MR.

[0054] The display unit 90 may directly display, for example, the video of the participant Mg captured by the terminal 100. On the other hand, the display unit 90 may also display, for example, an image (e.g., an avatar) that characterizes the participant Mg.

[0055] In one embodiment, the electronic device 1 may be a dedicated device as described above. Meanwhile, in one embodiment, the electronic device 1 may include, for example, an audio output unit 70 and a direction adjustment unit 80 among the functional units shown in FIG. 2 . In this case, the electronic device 1 may be connected to another electronic device to supplement at least some of the functions of the other functional units shown in FIG. 2 . Here, the other electronic device may be, for example, a general-purpose smartphone, tablet, phablet, notebook computer (notebook PC or laptop), or computer (desktop).

[0056] 3 is a block diagram showing a schematic configuration of the terminal 100 shown in FIG. 1. An example of the configuration of the terminal 100 according to one embodiment will be described below. As shown in FIG. 1, the terminal 100 may be a terminal used by the participant Mg at his / her home RL, for example. The electronic device 1 according to one embodiment has a function of outputting the voice input to the terminal 100 when the participant Mg speaks. Therefore, in such a situation, the terminal 100 may also be referred to as the "speaker's terminal" as appropriate.

[0057] 3, the terminal 100 according to an embodiment may include a control unit 110, a storage unit 120, a communication unit 130, an imaging unit 140, a sound collection unit 150, an audio output unit 170, and a display unit 190. In an embodiment, the terminal 100 may not include at least some of the functional units shown in FIG. 3, or may include components other than the functional units shown in FIG. 3.

[0058] The control unit 110 controls and / or manages the entire terminal 100, including each functional unit that constitutes the terminal 100. The control unit 110 may basically have a configuration based on the same concept as the control unit 10 shown in FIG.

[0059] The storage unit 120 may function as a memory that stores various types of information. The storage unit 120 may store, for example, programs executed by the control unit 110 and results of processing executed by the control unit 110. The storage unit 120 may also function as a work memory for the control unit 110. As shown in FIG. 3, the storage unit 120 may be connected to the control unit 110 via a wired and / or wireless connection. The storage unit 120 may basically have a configuration based on the same concept as the storage unit 20 shown in FIG. 2, for example.

[0060] The communication unit 130 has an interface function for wireless and / or wired communication. The communication unit 130 may communicate wirelessly with, for example, a communication unit of another electronic device via an antenna. For example, the communication unit 130 may communicate wirelessly with the electronic device 1 shown in FIG. 1. In this case, the communication unit 130 may communicate wirelessly with the communication unit 30 of the electronic device 1. As such, in one embodiment, the communication unit 130 has a function for communicating with the electronic device 1. As shown in FIG. 3, the communication unit 130 may be connected to the control unit 110 via a wired and / or wireless connection. The communication unit 130 may basically have a configuration based on the same concept as, for example, the communication unit 30 shown in FIG. 2.

[0061] The imaging unit 140 may be configured to include an image sensor that electronically captures images, such as a digital camera. The imaging unit 140 may capture images of the interior of the home RL shown in FIG. 1, for example. In one embodiment, the imaging unit 140 may capture images of participants Mg participating in a conference from the home RL shown in FIG. 1, for example. The imaging unit 140 may convert the captured images into signals and transmit them to the control unit 110. For this reason, the imaging unit 140 may be connected to the control unit 110 via a wired and / or wireless connection. The imaging unit 140 may basically have a configuration based on the same concept as the imaging unit 40 shown in FIG. 2, for example.

[0062] The sound collection unit 150 detects sounds or voices around the terminal 100, including human voices. For example, the sound collection unit 150 may detect sounds or voices as air vibrations using, for example, a diaphragm and convert them into an electrical signal. Specifically, the sound collection unit 150 may include an acoustic device, such as a microphone, that converts sound into an electrical signal. In one embodiment, the sound collection unit 150 may detect the voice of the participant Mg in the home RL shown in FIG. 1 . The voice (electrical signal) detected by the sound collection unit 150 may be input to, for example, the control unit 110. For this reason, the sound collection unit 150 may be connected to the control unit 110 via a wired and / or wireless connection. The sound collection unit 150 may basically be configured based on the same concept as the sound collection unit 50 shown in FIG. 2 .

[0063] The audio output unit 170 converts an electrical signal (audio signal) output from the control unit 110 into sound and outputs it as sound or voice. The audio output unit 170 may be connected to the control unit 110 via a wired and / or wireless connection. The audio output unit 170 may be configured to include a device having a function of outputting sound, such as a speaker (loudspeaker). In one embodiment, the audio output unit 170 may output a sound detected by the sound collection unit 50 of the electronic device 1. Here, the sound detected by the sound collection unit 50 of the electronic device 1 may be the sound of at least one of the participants Ma, Mb, Mc, Md, Me, and Mf in the conference room MR shown in FIG. 1. The audio output unit 170 may basically be configured based on the same concept as the audio output unit 70 shown in FIG. 2, for example.

[0064] The display unit 190 may be any display device, such as a liquid crystal display (LCD), an organic electro-luminescence (EL) display, or an inorganic electro-luminescence (EL) display. The display unit 190 may basically have a configuration based on the same concept as the display unit 90 shown in FIG. 2, for example. Various data required for display on the display unit 190 may be supplied from, for example, the control unit 110 or the storage unit 120. For this reason, the display unit 190 may be connected to the control unit 110 or the like by wire and / or wirelessly.

[0065] The display unit 190 may be, for example, a touch screen display equipped with a touch panel function that detects input by contact with the participant Mg's finger or stylus.

[0066] In one embodiment, the display unit 190 may display video based on a video signal transmitted from the electronic device 1. The display unit 190 may display video of participants Ma, Mb, Mc, Md, Me, and Mf, for example, captured by (the imaging unit 40 of) the electronic device 1, as video based on the video signal transmitted from the electronic device 1. By displaying video of participants Ma, Mb, Mc, Md, Me, and Mf on the display unit 190 of the terminal 100, for example, participant Mg shown in FIG. 1 can visually know what participants Ma, Mb, Mc, Md, Me, and Mf are doing in a conference room MR away from their home RL.

[0067] The display unit 190 may directly display, for example, images of the participants Ma, Mb, Mc, Md, Me, and Mf captured by the electronic device 1. On the other hand, the display unit 190 may also display images (e.g., avatars) that characterize the participants Ma, Mb, Mc, Md, Me, and Mf.

[0068] In one embodiment, the terminal 100 may be a dedicated device as described above. Meanwhile, in one embodiment, the terminal 100 may include, for example, some of the functional units shown in FIG. 3 . In this case, the terminal 100 may be connected to another electronic device to supplement at least some of the functions of the other functional units shown in FIG. 3 . Here, the other electronic device may be, for example, a general-purpose smartphone, tablet, phablet, notebook computer (notebook PC or laptop), or computer (desktop).

[0069] In particular, smartphones, laptops, and the like often have almost all of the functional units shown in Fig. 3. Therefore, in one embodiment, the terminal 100 may be a smartphone, a laptop, or the like. In this case, the terminal 100 may be a smartphone, a laptop, or the like, on which an application (program) for linking with the electronic device 1 is installed.

[0070] Next, a description will be given of the operation of the electronic device 1 and the terminal 100 according to an embodiment. The following description will be given assuming a situation in which a participant Mg participates in a remote conference held in a conference room MR from his / her home RL, as shown in FIG.

[0071] That is, an electronic device 1 according to one embodiment is installed in a conference room MR and detects the voices of at least one of the participants Ma, Mb, Mc, Md, Me, and Mf. The voices detected by the electronic device 1 are transmitted to a terminal 100 installed in the home RL of the participant Mg. The terminal 100 outputs the voices of at least one of the participants Ma, Mb, Mc, Md, Me, and Mf received from the electronic device 1. This allows the participant Mg to hear the voices of at least one of the participants Ma, Mb, Mc, Md, Me, and Mf.

[0072] Meanwhile, a terminal 100 according to an embodiment is installed in the home RL of the participant Mg and detects the voice of the participant Mg. The voice detected by the terminal 100 is transmitted to an electronic device 1 installed in the conference room MR. The electronic device 1 outputs the voice of the participant Mg received from the terminal 100. This allows at least one of the participants Ma, Mb, Mc, Md, Me, and Mf to hear the voice of the participant Mg.

[0073] Hereinafter, the operation of the electronic device 1 and the terminal 100 according to an embodiment will be described mainly from the perspective of the electronic device 1. The operation of the electronic device 1 and the terminal 100 according to an embodiment may be configured to include roughly three stages. That is, the operation of the electronic device 1 and the terminal 100 according to an embodiment may be configured to include: (1) a user interface (hereinafter also referred to as UI as appropriate) display phase; (2) a setting change phase; and (3) an audio output phase. Each phase will be further described below.

[0074] 4 is a flowchart mainly illustrating the (1) UI display phase of the operation of the electronic device 1 and the terminal 100 according to an embodiment. FIG. 4 is a flowchart illustrating the operation of the electronic device 1 according to an embodiment in the (1) UI display phase.

[0075] It is assumed that the electronic device 1 and the terminal 100 are connected wirelessly and / or wired when the operation shown in Fig. 4 starts. The time when the operation shown in Fig. 4 starts is a time after the electronic device 1 and the terminal 100 are connected, and may be, for example, a time before the start of a remote conference. In other words, the time when the operation shown in Fig. 4 starts may be, for example, a time when a remote conference is being prepared. The operation shown in Fig. 4 allows each user to check or understand the initial settings or current settings of the electronic device 1.

[0076] 4 starts, the control unit 10 of the electronic device 1 acquires the positions of candidates (hereinafter also referred to as "transmission target candidates") to which the voice (voice of participant Mg) received from the terminal 100 is to be transmitted in the conference room MR (step S11). Here, the transmission target candidates may be participants Ma, Mb, Mc, Md, Me, and Mf in the situation shown in Fig. 1. That is, in step S11, the control unit 10 acquires the positions of participants Ma, Mb, Mc, Md, Me, and Mf of the remote conference in the conference room MR.

[0077] In step S11, the position of the transmission target candidate may be stored, for example, as a predetermined position in the memory unit 20. For example, if the positions of chairs and the like in the conference room MR are predetermined, the control unit 10 can acquire the positions of the transmission target candidate in advance. Also, even if the positions of chairs and the like in the conference room MR are predetermined, if the positions of the transmission target candidate have not been acquired in advance, the control unit 10 may acquire the positions via, for example, the communication unit 30. Also, if the positions of the transmission target candidate have not been acquired in advance, the control unit 10 may detect a user input of the positions via an input device such as a keyboard.

[0078] Furthermore, in step S11, the control unit 10 may detect the position of the transmission target candidate using, for example, the imaging unit 40. The imaging unit 40 is capable of capturing an image of the surroundings of the electronic device 1. Therefore, the imaging unit 40 can capture video of the conference participants Ma, Mb, Mc, Md, Me, and Mf in the conference room MR. If the imaging unit 40 includes a movable camera, the control unit 10 may capture an image of the surroundings of the electronic device 1 while changing the direction of the imaging unit 40. Furthermore, if the direction adjustment unit 80 is capable of changing the orientation of the housing of the electronic device 1, the control unit 10 may capture an image of the surroundings of the electronic device 1 while changing the direction of the electronic device 1.

[0079] Fig. 5 is a diagram showing a portion of the conference room MR captured by the imaging unit 40. For example, if the direction adjustment unit 80 can change the orientation of the electronic device 1 by 360° in the horizontal direction, the imaging unit 40 can capture a 360° image of the conference room MR, such as that shown in Fig. 5. Fig. 5 may show a portion of the 360° image of the conference room MR captured by the imaging unit 40. As shown in Fig. 5, it is assumed that conference participants Ma, Mb, and Mc are captured in the image captured by the imaging unit 40.

[0080] In step S11, the control unit 10 may acquire the positions of conference participants Ma, Mb, Mc, etc. (transmission target candidates) from an image such as that shown in FIG. 5. In this case, the control unit 10 may first extract the transmission target candidates from the image such as that shown in FIG. 5 using existing technology such as facial recognition. Then, the control unit 10 may estimate the actual position of the transmission target candidates in, for example, the conference room MR from the position of the transmission target candidates in the angle of view based on information such as the position and direction of the electronic device 1 when the imaging unit 40 captured the transmission target candidates. In this way, various known techniques can be adopted as a method for estimating the position of a target from a captured image.

[0081] In step S11, the control unit 10 estimates the position of participant Ma as coordinates (Xa, Ya, Za), the position of participant Mb as coordinates (Xb, Yb, Zb), and the position of participant Mc as coordinates (Xc, Yc, Zc), as shown in FIG. 5. Similarly, in step S11, the control unit 10 may estimate the coordinates of each of participants Md, Me, and Mf, for example, based on images of these participants. For simplicity of explanation, only participants Ma, Mb, and Mc will be described below among participants Ma, Mb, Mc, Md, Me, and Mf of the conference shown in FIG. 1.

[0082] After the positions of the communication target candidates are acquired in step S11, the control unit 10 calculates or acquires the voice level at the position of each communication target candidate (step S12). That is, the control unit 10 calculates or acquires information indicating the voice level of the participant Mg, which is received from the terminal 100 and output from the electronic device 1, at the position of each communication target candidate in the conference room MR.

[0083] In step S12, the control unit 10 may calculate or acquire the sound level at each position based on, for example, the position of the electronic device 1, the direction of the sound output unit 70, and the position of each transmission target candidate. Here, the direction of the sound output unit 70 may be the direction of the directivity of the sound output from the sound output unit 70. Furthermore, the "sound level" may be any of various indicators that allow the transmission target candidate to be auditorily recognized. For example, the "sound level" may be the level of sound pressure.

[0084] In step S12, the control unit 10 may obtain the sound level at the position of each transmission target candidate from data previously obtained through demonstration experiments, etc. Such data may be stored in advance in the storage unit 20, for example, or may be obtained from outside the electronic device 1 via the communication unit 30 as necessary.

[0085] On the other hand, in step S12, the control unit 10 may calculate or estimate the sound level at the position of each transmission target candidate from various data. For example, the control unit 10 may calculate or estimate the sound level at the position of each transmission target candidate based on the position and direction of the electronic device 1 (or the sound output unit 70), the sound pressure of the sound output by the sound output unit 70, and the position of each transmission target candidate. Also, in step S12, the control unit 10 may calculate or estimate the sound level at each position within a predetermined range around each transmission target candidate from various data.

[0086] After the audio level is calculated or acquired in step S12, the control unit 10 transmits information visually indicating the audio level to the terminal 100 (step S13). In step S13, the control unit 10 may generate information visually indicating the audio level and transmit the information from the communication unit 30 of the electronic device 1 to the communication unit 130 of the terminal 100.

[0087] Here, the information visually indicating the audio level may be, for example, information that visually suggests to what extent the audio of participant Mg output by electronic device 1 is audibly recognizable at the position of each candidate transmission target.

[0088] 6 is a diagram showing an example in which the information visually indicating the audio level transmitted in step S13 is received by the terminal 100 and displayed on the display unit 190. By seeing the information as shown in FIG. 6 displayed on the display unit 190 of the terminal 100, the participant Mg participating in the conference from his / her home RL can know in advance to which participants his / her speech will be transmitted and to what extent.

[0089] Participants Ma and Mb are highlighted within area A12 shown in Fig. 6 . This may indicate that the voice of participant Mg output by the electronic device 1 is audibly recognizable to participant Ma and Mb within area A12. Furthermore, within area A11 (excluding area A12) shown in Fig. 6 , the voice of participant Mg output by the electronic device 1 is barely audibly recognizable. Furthermore, outside area A11 shown in Fig. 6 , participant Mc is not highlighted. This may indicate that the voice of participant Mg output by the electronic device 1 is almost or completely audibly recognizable to participant Mc outside area A11.

[0090] 6 visually indicates whether each of the communication target candidates can recognize the voice of the participant Mg depending on whether the images of the participants Ma, Mb, and Mc are highlighted. However, in one embodiment, the manner of visually indicating whether the voice of the participant Mg can be recognized is not limited to whether the image of each of the communication target candidates is highlighted.

[0091] For example, the control unit 10 may visually indicate whether the voice of the participant Mg is recognizable in each of the communication target candidates by using different shades of colors to display the images of the participants Ma, Mb, and Mc. Furthermore, for example, the control unit 10 may visually indicate whether the voice of the participant Mg is recognizable in each of the communication target candidates by using different transparency levels when displaying the images of the participants Ma, Mb, and Mc. Furthermore, for example, the control unit 10 may visually indicate whether the voice of the participant Mg is recognizable in each of the communication target candidates by using different sizes when displaying the images of the participants Ma, Mb, and Mc. Furthermore, the control unit 10 may visually indicate whether the voice of the participant Mg is recognizable in each of the communication target candidates by using different modes when displaying the images of the participants Ma, Mb, and Mc.

[0092] In this way, the control unit 10 transmits information visually indicating the level of the speaker's (participant Mg's) voice at the position of the candidate target to which the speaker's voice is transmitted to the terminal 100. This allows the terminal 100 to display, on the display unit 190, the level of the speaker's voice at the position of the candidate target to which the speaker's voice is transmitted.

[0093] By looking at the display on the display unit 190 of the terminal 100 as shown in Fig. 6, the participant Mg can understand that his / her voice is audibly recognizable (i.e., audible) to the participants Ma and Mb. Also, by looking at the display on the display unit 190 of the terminal 100 as shown in Fig. 6, the participant Mg can understand that his / her voice is audibly inaudible (i.e., inaudible) to the participant Mc.

[0094] As described above, in the (1) UI display phase, the participant Mg can know from the UI whether or not his / her voice can be heard by other conference participants.

[0095] 7 is a flowchart mainly illustrating the (2) setting change phase of the operation of the electronic device 1 and the terminal 100 according to an embodiment. FIG. 4 is a flowchart illustrating the operation of the electronic device 1 according to an embodiment in the (2) setting change phase.

[0096] 7 starts, the remote conference may or may not have started between the electronic device 1 and the terminal 100. By the operation shown in Fig. 7, the participant Mg, who is the speaker, can select a participant whose voice is audibly recognizable to the participant Mg (i.e., whose voice the participant Mg can hear).

[0097] 7 starts, the control unit 10 determines whether or not a command to change the audio level has been input from the terminal 100 (step S21). For example, in step S21, assume that the participant Mg inputs, using the terminal 100, a command to change the audio level at the position of the transmission target candidate. The terminal 100 then transmits the input to the communication unit 30 of the electronic device 1 via the communication unit 130. In this case, the control unit 10 may determine that a command to change the audio level has been input from the terminal 100.

[0098] When a command to change the audio level is input from the terminal 100 in step S21, the control unit 10 may transmit to the terminal 100 information for displaying a screen such as that shown in Fig. 8 on the display unit 190 of the terminal 100. In this case, the control unit 10 may transmit information for displaying the screen such as that shown in Fig. 8 from the communication unit 30 to the communication unit 130 of the terminal 100.

[0099] Fig. 8 is a diagram showing a screen on the display unit 190 of the terminal 100 on which an input for changing the audio level at the position of the communication target candidate is made. Fig. 8 is a diagram showing sliders on the screen of the display unit 190 shown in Fig. 6 that can change the audio level for each communication target candidate. The control unit 110 may display the screen shown in Fig. 8 when the participant Mg makes a predetermined input on the terminal 100, for example, when the participant Mg touches the display unit 190 of the terminal 100, or when the participant Mg touches any of the communication target candidates on the display unit 190 of the terminal 100. The participant Mg can change the audio level at the position of each communication target candidate by performing a touch operation on the slider displayed at the position of each communication target candidate.

[0100] For example, a participant Mg can change the audio level at the position of participant Ma by operating a slider Sa corresponding to participant Ma displayed on the display unit 190. Also, a participant Mg can change the audio level at the position of participant Mb by operating a slider Sb corresponding to participant Mb displayed on the display unit 190. Also, a participant Mg can change the audio level at the position of participant Mc by operating a slider Sc corresponding to participant Mc displayed on the display unit 190.

[0101] In FIG. 8, the slider Sa corresponding to participant Ma is at its maximum. In this case, this indicates that the voice of participant Mg is sufficiently audibly recognizable to participant Ma (i.e., sufficiently audible). Also in FIG. 8, the slider Sb corresponding to participant Mb is also at its maximum. In this case, this indicates that the voice of participant Mg is sufficiently audibly recognizable to participant Mb (i.e., sufficiently audible). Also in FIG. 8, the slider Sc corresponding to participant Mc is at its minimum. In this case, this indicates that the voice of participant Mg is not audibly recognizable to participant Mc (i.e., inaudible).

[0102] If no input to change the audio level is received from the terminal 100 in step S21, the control unit 10 may wait until an input to change the audio level is received. Alternatively, if no input to change the audio level is received from the terminal 100 in step S21, the control unit 10 may end the operation shown in Fig. 7. On the other hand, if an input to change the audio level is received from the terminal 100 in step S21, the control unit 10 calculates or acquires an amplification coefficient and an audio direction (step S22).

[0103] In step S22, the control unit 10 calculates or acquires an amplification coefficient and a voice direction for achieving the changed voice level. Here, the "amplification coefficient" may be the amplification coefficient used when the amplifier 60 amplifies the speaker's voice signal. Furthermore, the "voice direction" may be the direction of the speaker's voice adjusted by the direction adjustment unit 80. Here, the control unit 10 may acquire the amplification coefficient and the voice direction for achieving the changed voice level from data previously obtained through demonstration experiments, etc. Such data may be pre-stored in the storage unit 20, for example, or may be acquired from outside the electronic device 1 via the communication unit 30 as needed. Furthermore, the control unit 10 may calculate the amplification coefficient and the voice direction for achieving the changed voice level from various data. For example, the control unit 10 may calculate the amplification coefficient and the voice direction for achieving the changed voice level based on the position and direction of the electronic device 1 (or the voice output unit 70), the sound pressure of the voice output by the voice output unit 70, the position of each transmission target candidate, and the like.

[0104] When the amplification factor and the sound direction are calculated or acquired in step S22, the control unit 10 controls at least one of the amplifier 60 and the direction adjustment unit 80 so that the amplification factor and the sound direction are realized (step S23). If at least one of the amplification factor and the sound direction is already realized in step S23, the control unit 10 does not need to control at least one of the amplifier 60 and the direction adjustment unit 80.

[0105] After the amplifier 60 and / or the direction adjustment unit 80 are controlled in step S23, the control unit 10 transmits information visually indicating the changed audio level to the terminal 100 (step S24). In step S13, the control unit 10 may generate information visually indicating the changed audio level and transmit the information from the communication unit 30 of the electronic device 1 to the communication unit 130 of the terminal 100. Here, the information visually indicating the audio level may be information visually suggesting to what extent the audio of the participant Mg output by the electronic device 1 is audibly recognizable at the position of each communication target candidate, as shown in step S13 of FIG.

[0106] In step S24, the control unit 10 may calculate or estimate the sound level at the position of each transmission target candidate from various data. For example, the control unit 10 may calculate or estimate the sound level at the position of each transmission target candidate based on the position and direction of the electronic device 1 (or the sound output unit 70), the sound pressure of the sound output by the sound output unit 70, and the position of each transmission target candidate. In addition, in step S24, the control unit 10 may calculate or estimate the sound level at each position within a predetermined range around each transmission target candidate from various data.

[0107] 9 is a diagram showing an example in which the information visually indicating the audio level transmitted in step S24 is received by the terminal 100 and displayed on the display unit 190. By seeing the information as shown in FIG. 9 displayed on the display unit 190 of the terminal 100, the participant Mg participating in the conference from his / her home RL can know in advance to which participants his / her speech will be conveyed and to what extent.

[0108] Participant Ma is highlighted within area A22 shown in Fig. 9 . This may indicate that the voice of participant Mg output by the electronic device 1 is audibly recognizable to participant Ma within area A22. Furthermore, within area A21 (excluding area A22) shown in Fig. 9 , the voice of participant Mg output by the electronic device 1 is barely audibly recognizable. Furthermore, participants Mb and Mc are not highlighted outside area A21 shown in Fig. 9 . This may indicate that the voice of participant Mg output by the electronic device 1 is almost or completely audibly recognizable to participants Mb and Mc outside area A21.

[0109] As shown in Fig. 8, the participant Mg can operate the slider Sb corresponding to the participant Mb displayed on the display unit 190 of the terminal 100 so that it is, for example, minimized. As a result, as shown in Fig. 9, the participant Mg can visually understand that the slider Sb corresponding to the participant Mb displayed on the display unit 190 of the terminal 100 has been minimized. That is, in the situation shown in Fig. 8, the voice of the participant Mg was audible to the participant Mb. However, in the situation shown in Fig. 9, the voice of the participant Mg can be made inaudible to the participant Mb.

[0110] For example, depending on the sound pressure and / or direction of the sound output from the sound output unit 70, it may be impossible to arbitrarily determine whether each communication target candidate is audibly recognizable (i.e., whether it is audible or inaudible). In such cases, the control unit 10 may control the slider Sb corresponding to the participant Mb shown in FIG. 8 so that it does not move, or may control the slider Sb to return to a certain position after being moved. Furthermore, for example, when a participant Mg attempts to move the slider Sb corresponding to the participant Mb shown in FIG. 8, the control unit 10 may control the slider Sb so that it is difficult for the participant Mg to perform the movement. Furthermore, for example, when a participant Mg moves the slider Sb corresponding to the participant Mb shown in FIG. 8, the control unit 10 may control another slider, such as the slider Sa corresponding to the participant Ma, to move in conjunction with the slider Sb.

[0111] 10 is a flowchart mainly illustrating the (3) audio output phase of the operation of the electronic device 1 and the terminal 100 according to an embodiment. Fig. 10 is a flowchart illustrating the operation of the electronic device 1 according to an embodiment in the (3) audio output phase.

[0112] 10 starts, the operations shown in Fig. 4 and Fig. 7 may have been completed, and a remote conference may have started between the electronic device 1 and the terminal 100. By the operation shown in Fig. 10, the participant Mg, who is the speaker, can speak so that his or her voice is audibly recognizable, as set in Fig. 7.

[0113] 10 starts, the control unit 10 determines whether or not a voice input has been received from the terminal 100 (step S31). Here, the voice input from the terminal 100 may be an input based on the voice of the participant Mg detected by the sound collection unit 150 of the terminal 100.

[0114] If there is a voice input from the terminal 100 in step S31, the control unit 10 controls the amplifier 60 so that the voice input is amplified according to the amplification coefficient calculated or acquired in step S22 of Fig. 7 (step S32). Also, in step S32, the control unit 10 controls the voice output unit 70 to output the voice amplified according to the amplification coefficient.

[0115] When the sound is output in step S32, the control unit 10 transmits information visually indicating the level of the sound to be output to the terminal 100 (step S33). In step S33, the control unit 10 may generate information visually indicating the level of the sound output from the electronic device 1, and transmit the information from the communication unit 30 of the electronic device 1 to the communication unit 130 of the terminal 100. Here, the information visually indicating the level of the sound may be, for example, information visually suggesting to what extent the sound of the participant Mg output by the electronic device 1 is audibly recognizable at the position of each communication target candidate, as shown in step S24 of FIG.

[0116] In step S33, the control unit 10 may calculate or estimate the sound level at the position of each transmission target candidate from various data. For example, the control unit 10 may calculate or estimate the sound level at the position of each transmission target candidate based on the position and direction of the electronic device 1 (or the sound output unit 70), the sound pressure of the sound output by the sound output unit 70, and the position of each transmission target candidate. In addition, in step S33, the control unit 10 may calculate or estimate the sound level at each position within a predetermined range around each transmission target candidate from various data.

[0117] 11 is a diagram showing an example in which the information visually indicating the voice level transmitted in step S33 is received by the terminal 100 and displayed on the display unit 190. By seeing the information as shown in FIG. 11 displayed on the display unit 190 of the terminal 100, the participant Mg participating in the conference from his / her home RL can understand to which participants the speech of the participant Mg is being conveyed and to what extent.

[0118] Participant Ma is highlighted within area A22 shown in Fig. 11 . This may indicate that the voice of participant Mg output by the electronic device 1 is audibly recognizable to participant Ma within area A22. Furthermore, within area A21 (excluding area A22) shown in Fig. 11 , the voice of participant Mg output by the electronic device 1 is barely audibly recognizable. Furthermore, participants Mb and Mc are not highlighted outside area A21 shown in Fig. 11 . This may indicate that the voice of participant Mg output by the electronic device 1 is almost or completely audibly recognizable to participants Mb and Mc outside area A21.

[0119] As described above, in the electronic device 1 according to one embodiment, the control unit 10 sets an amplification coefficient used when the amplifier 60 amplifies the speaker's voice signal. At the same time, the control unit 10 sets the direction of the speaker's voice that is adjusted by the direction adjustment unit 80. The control unit 10 can also change at least one of the amplification coefficient and the voice direction based on an input to change the speaker's voice level at the terminal 100. Here, the "amplification coefficient" may be the amplification coefficient used when the amplifier 60 amplifies the speaker's voice signal. The "voice direction" may be the direction of the speaker's voice that is adjusted by the direction adjustment unit 80.

[0120] In the electronic device 1 according to an embodiment, the audio output unit 70 may output, as the speaker's voice, a voice signal of the speaker that the communication unit 30 receives from the terminal 100. In addition, in an embodiment, the control unit 10 may set the volume of the speaker's voice that the audio output unit 70 outputs. In this case, the control unit 10 may be configured to change the volume of the speaker's voice that the audio output unit 70 outputs when the level of the speaker's voice is changed in the terminal 100. In an embodiment, the volume of the speaker's voice that the audio output unit 70 outputs may be changed by, for example, changing the amplification coefficient using the amplifier 60.

[0121] As described above, the electronic device 1 can output the voice of the participant Mg who is at home RL in the conference room MR. Furthermore, the electronic device 1 can transmit the voice to only the candidate communication targets to which the participant Mg wishes to communicate in the conference room MR. That is, according to the electronic device 1 of one embodiment, the speaker, the participant Mg, can select participants whose voice is auditorily recognizable to the speaker (i.e., who can hear the participant Mg's voice). Therefore, according to the electronic device 1 of one embodiment, it is possible to improve the functionality of an electronic device that enables communication between multiple locations.

[0122] (Other Embodiments) As described above, it is possible that it may not be possible to arbitrarily distinguish whether or not each candidate communication target is audibly recognizable (i.e., whether or not it is audible) depending on the sound pressure and / or direction of the sound output from the sound output unit 70. That is, it is possible that the voice of the participant Mg output from the sound output unit 70 may be heard by a person among the candidate communication targets who is not desired to be a communication target (hereinafter also referred to as a "non-communication target"). In such a case, the voice of the participant Mg may be masked by outputting various sounds, such as noise, toward the non-communication target.

[0123] 12 is a block diagram showing a schematic functional configuration of an electronic device 2 according to another embodiment. An example of the configuration of the electronic device 2 according to the embodiment will be described below, focusing on differences from the electronic device 1 described above.

[0124] 12 includes a second output unit 72. The second output unit 72 outputs a predetermined auditory effect. The predetermined auditory effect may be any of various sounds or sounds, such as environmental sounds, noises, operating sounds of the electronic device 2, sound effects that attract human attention, or sounds different from the voice of the participant Mg.

[0125] The second output unit 72 may be provided inside or outside the housing of the electronic device 2. When the second output unit 72 is provided inside the housing of the electronic device 2, a mechanism capable of changing the direction of the second output unit 72 and / or the directivity of the auditory effect output from the second output unit 72 may be provided. When the second output unit 72 is provided outside the housing of the electronic device 2, a mechanism capable of changing the direction of the second output unit 72 and / or the directivity of the auditory effect output from the second output unit 72 may be provided. Furthermore, when the second output unit 72 is provided outside the housing of the electronic device 2, (a plurality of) second output units 72 may be provided corresponding to each of the candidate transmission targets, such as participants Ma, Mb, Mc, Md, Me, and Mf. Furthermore, the second output unit 72 may be located at a position different from the audio output unit 70 so as to generate a different sound effect from that of the audio output unit 70.

[0126] For example, if the voice of participant Mg output in step S32 of Fig. 10 is transmitted to a non-transmission target, or if there is a risk of this, the electronic device 2 may output a predetermined auditory effect from the second output unit 72 toward the non-transmission target. For example, in the situation shown in Fig. 8, even if participant Mg, designating participant Mb as a non-transmission target, lowers the slider Sb displayed on the display unit 190 of the terminal 100 to a certain extent, it is possible that the audio level at the position of participant Mb does not fall below a predetermined level. In such a case, the control unit 10 may control the second output unit 72 to output a predetermined auditory effect instead of or in addition to lowering the level of the audio output from the audio output unit 70.

[0127] In this case, for example, when the control unit 10 receives an input to change the speaker's voice level at the terminal 100 at step S21 in FIG. 7 , the control unit 10 may control, at step S23 or the like, to output a predetermined auditory effect from the second output unit 72. In this way, the control unit 10 may control the auditory effect output from the second output unit 72 based on the input to change the speaker's voice level at the terminal 100. This reduces the risk that the voice of the participant Mg will be transmitted to non-communication targets. Therefore, the electronic device 2 according to one embodiment can improve the functionality of the electronic device that enables communication between multiple locations.

[0128] In still another embodiment, the second output unit 72 included in the electronic device 2 may output ultrasonic waves to create a predetermined auditory effect. The second output unit 72 may output ultrasonic waves to a predetermined part of the non-communication target or a predetermined part around the non-communication target, thereby drawing the attention of the non-communication target to the reflection of the ultrasonic waves, and as a result, the attention of the non-communication target may be less likely to be directed to the voice of the participant Mg.

[0129] Other Embodiments In the above-described cases, a predetermined visual effect may be output instead of or in addition to the predetermined auditory effect.

[0130] 13 is a block diagram showing a schematic functional configuration of an electronic device 3 according to another embodiment. An example of the configuration of the electronic device 3 according to the embodiment will be described below, focusing on differences from the electronic device 1 or 2 described above.

[0131] Compared to the electronic device 1 shown in Fig. 2 , the electronic device 3 shown in Fig. 13 includes a third output unit 93. The third output unit 93 outputs a predetermined visual effect. Here, the predetermined visual effect may be, for example, light such as an LED or a laser beam. The predetermined visual effect may also be in various forms, such as the aforementioned light emitting for only a moment or flashing at a predetermined speed.

[0132] The third output unit 93 may be provided inside the housing of the electronic device 3 or outside the housing of the electronic device 3. When the third output unit 93 is provided inside the housing of the electronic device 3, a mechanism capable of changing the direction of the third output unit 93 and / or the directivity of the visual effect output from the third output unit 93 may be provided. When the third output unit 93 is provided outside the housing of the electronic device 3, a mechanism capable of changing the direction of the third output unit 93 and / or the directivity of the visual effect output from the third output unit 93 may be provided. Furthermore, when the third output unit 93 is provided outside the housing of the electronic device 3, (a plurality of) third output units 93 may be provided corresponding to each of the transmission target candidates, such as participants Ma, Mb, Mc, Md, Me, and Mf.

[0133] For example, if the voice of participant Mg output in step S32 of Figure 10 is transmitted to a non-transmission target, or if there is a risk of this being the case, the electronic device 3 may output a predetermined visual effect from the third output unit 93 toward the non-transmission target.

[0134] In this case, when the control unit 10 receives an input to change the speaker's voice level at the terminal 100 in step S21 of FIG. 7, for example, the control unit 10 may control, in step S23 or the like, to output a predetermined auditory effect from the third output unit 93. In this way, the control unit 10 may control the auditory effect output from the third output unit 93 based on the input to change the speaker's voice level at the terminal 100. This reduces the risk that the voice of the participant Mg will be transmitted to non-communication targets. Therefore, according to the electronic device 3 of one embodiment, it is possible to improve the functionality of the electronic device that enables communication between multiple locations.

[0135] Although the embodiments of the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art would easily be able to make various modifications or alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are within the scope of the present disclosure. For example, functions included in each component or step can be rearranged so as not to cause logical inconsistencies, and multiple components or steps can be combined or divided into one. Although the embodiments of the present disclosure have been described mainly in terms of an apparatus, the embodiments of the present disclosure can also be realized as a method including steps executed by each component of the apparatus. The embodiments of the present disclosure can also be realized as a method, a program executed by a processor included in the apparatus, or a storage medium on which a program is recorded. It should be understood that these are also encompassed within the scope of the present disclosure.

[0136] The above-described embodiment is not limited to implementation as the electronic device 1. For example, the above-described embodiment may be implemented as a control method for a device such as the electronic device 1. Furthermore, for example, the above-described embodiment may be implemented as a program executed by a device such as the electronic device 1. Furthermore, such a program is not necessarily limited to being executed only by the electronic device 1, and may be executed by another electronic device such as a smartphone linked to the electronic device 1.

[0137] The above-described embodiment can be implemented from various perspectives. For example, in one embodiment, the electronic device 1 may be implemented as a system including the electronic device 1 and the terminal 100. Furthermore, in one embodiment, the electronic device 1 may be implemented as another electronic device (e.g., a server or a control device) capable of communicating with the electronic device 1 and the terminal 100. In this case, at least some of the functions and / or operations of the electronic device 1 described in the above-described embodiment may be performed by another electronic device, such as a server. More specifically, the other electronic device, such as a server, may set (instead of the electronic device 1) an amplification coefficient used when amplifying an audio signal to be output as audio by the electronic device 1. In this case, the electronic device 1 can amplify the audio signal according to the amplification coefficient set by the other electronic device, such as the server, and output the amplified audio signal as audio. Furthermore, in one embodiment, the electronic device 1 may be implemented as a program executed by the other electronic device, such as a server. Furthermore, in one embodiment, the electronic device 1 may be implemented as a system including the electronic device 1, the terminal 100, and the other electronic device, such as the server.

[0138] For example, the other electronic devices (e.g., a server or a control device) described above may be components such as the electronic device 200 shown in FIG. 14 . In this case, the electronic device 200 may include, for example, a control unit 210, a storage unit 220, and a communication unit 230, as shown in FIG. 15 . The control unit 210 may have the same configuration and / or function as the control unit 10 and / or the control unit 110. The storage unit 220 may have the same configuration and / or function as the storage unit 20 and / or the storage unit 120. The communication unit 230 may have the same configuration and / or function as the communication unit 30 and / or the communication unit 130. Furthermore, the control unit 210 of the electronic device 200 may perform, for example, an operation as shown in FIG. 16 . The processing in each step shown in FIG. 16 may be the same as or similar to the corresponding step described in FIGS. 4 , 7 , 10 , etc.

[0139] REFERENCE SIGNS LIST 1 Electronic device 10 Control unit 20 Memory unit 30 Communication unit 40 Imaging unit 50 Sound collection unit 60 Amplifier 70 Audio output unit 72 Second output unit 80 Direction adjustment unit 90 Display unit 93 Third output unit 100 Terminal 110 Control unit 120 Memory unit 130 Communication unit 140 Imaging unit 150 Sound collection unit 170 Audio output unit 190 Display unit 200 Electronic device

Claims

1. A communication unit that communicates with the terminal of the speaker; An audio output unit that outputs the voice signal of the speaker received by the communication unit from the terminal as the voice of the speaker; A control unit that sets the volume of the voice output by the audio output unit; Comprising: The control unit transmits information visually indicating the level of the voice at the position of the candidate for the target to which the voice is transmitted to the terminal, and when the level of the voice is changed in the terminal, the control unit can change the volume of the voice, an electronic device.

2. Comprising a direction adjustment unit that adjusts the direction of the voice output by the audio output unit; The control unit sets the direction of the voice adjusted by the direction adjustment unit, and when the level of the voice is changed in the terminal, the control unit can change at least one of the volume of the voice and the direction of the voice. The electronic device according to claim 1.

3. When the level of the voice is changed for each candidate for the target to which the voice is transmitted in the terminal, the control unit can change at least one of the volume of the voice and the direction of the voice for each candidate for the target to which the voice is transmitted. The electronic device according to claim 1 or 2.

4. The control unit changes at least one of the volume of the voice and the direction of the voice so that the level of the voice becomes a predetermined level or less at the position of a predetermined candidate among the candidates for the target to which the voice is transmitted. The electronic device according to claim 1.

5. Comprising a second output unit that outputs a predetermined auditory effect; When the level of the voice is changed in the terminal, the control unit controls the auditory effect output from the second output unit. The electronic device according to claim 1.

6. The second output unit outputs a predetermined sound wave. The electronic device according to claim 5.

7. The second output unit outputs a predetermined ultrasonic wave. The electronic device according to claim 5.

8. The second output unit is disposed at a position different from the audio output unit. The electronic device according to claim 5.

9. Comprising a third output unit that outputs a predetermined visual effect; When the level of the voice is changed in the terminal, the control unit controls the visual effect output from the second output unit. The electronic device according to claim 5.

10. The third output unit outputs a predetermined light. The electronic device according to claim 9.

11. In an electronic device, A step of communicating with the terminal of the speaker; Outputting the voice signal of the speaker received from the terminal as the voice of the speaker; Setting the volume of the voice output in the outputting step; Transmitting, to the terminal, information visually indicating the level of the voice at the position of the candidate for the target to whom the voice is transmitted; Changing the volume of the voice when the level of the voice is changed in the terminal; A program for causing the above to be executed.

12. A system including an electronic device and a speaker's terminal capable of communicating with each other, wherein the electronic device includes a voice output unit that outputs the voice signal of the speaker received from the terminal as the voice of the speaker, and a control unit that sets the volume of the voice output by the voice output unit, and the control unit transmits, to the terminal, information visually indicating the level of the voice at the position of the candidate for the target to whom the voice is transmitted, and controls to be able to change the volume of the voice when the level of the voice is changed in the terminal; wherein the terminal includes a sound collection unit that collects the voice of the speaker, and a control unit that transmits the voice signal of the speaker to the electronic device, and receives, from the electronic device, information visually indicating the level of the voice at the position of the candidate for the target to whom the voice is transmitted, and controls to transmit an input for changing the level of the voice to the electronic device; A system.

13. An electronic device communicable with a speaker's terminal and another electronic device that outputs the voice signal of the speaker as the voice of the speaker, including a control unit that sets the volume of the voice output by the other electronic device, wherein the control unit transmits, to the terminal, information visually indicating the level of the voice at the position of the candidate for the target to whom the voice is transmitted, can change the volume of the voice when the level of the voice is changed in the terminal, and controls the other electronic device to output the voice.

14. In an electronic device, communicating with a speaker's terminal and another electronic device that outputs the voice signal of the speaker as the voice of the speaker; setting the volume of the voice output by the other electronic device; transmitting, to the terminal, information visually indicating the level of the voice at the position of the candidate for the target to whom the voice is transmitted; changing the volume of the voice when the level of the voice is changed in the terminal; a step of controlling the other electronic device to output the voice; A program for causing the above to be executed.

15. A system including a speaker's terminal, an electronic device, and another electronic device, wherein the terminal and the electronic device are configured to be communicable with the other electronic device; The terminal includes a sound collection unit that collects the voice of the speaker; a control unit that transmits the voice signal of the speaker to the other electronic device and receives information visually indicating the level of the voice at a position of a candidate for a target to which the voice is transmitted from the other electronic device, and controls to transmit an input for changing the level of the voice to the other electronic device; and The electronic device includes a voice output unit that outputs the voice signal of the speaker received from the other electronic device as the voice of the speaker; and The other electronic device includes a control unit that sets the volume of the voice output by the electronic device; The control unit transmits information visually indicating the level of the voice at a position of a candidate for a target to which the voice is transmitted to the terminal, can change the volume of the voice when the level of the voice is changed in the terminal, and controls the electronic device to output the voice; A system.